Securonix (securonix.com) is the market leader in entity and user behavior analytics offering risk mitigation, incident response and rapid detection of sophisticated insider threats and cyber-attacks. Securonix helps their clients across the globe address their cyber security needs including the detection of advanced persistent threats (APTs), insider threats, privilege account misuses, and enterprise and online fraud. Our security analytics platform is used by some of the largest organizations in the financial, healthcare, pharmaceutical, manufacturing, and federal sectors.
The Spark Performance Engineer is to help drive performance analysis, resiliency initiatives, optimization and scalability of spark jobs running on the Big Data clusters.
The ideal candidate should have a good knowledge of performance engineering and have experience in the Big Data Administrator role with a keen focus on performance and resilience. The candidate will be responsible for analyzing, debugging, measuring and recreating specific user/tenant issues in order to extract actionable data and model future capacity and scalability needs and will have a chance to learn about the security data sets along with the SIEM industry.
- Ability to clearly characterize system and Spark Job performances.
- Passion for distilling large volumes of data, analyze scalability, capacity and performance issues in Big Data Platforms.
- Experience with Data Science and Big Data technologies, Hadoop, Hive, HBase, Solr, Spark, Kafka, Jupyter, Elasticsearch.
- Deep technical understanding of Spark Processing Framework.
- Ability to analyze and debug Spark applications written in Scala, Python and Java.
- Implement the spark performance tuning and resource optimization wherever required.
- Perform benchmarking/stress tests and document the best practices for different applications whenever required.
- Proactively work with multiple tenants on improving the application performance and ensure they are robust, resilient and scalable.
- Experience in tuning the spark applications on Kafka, HBase, Hive, HDFS and any other technologies.
- JVM Profiling and tuning experience.
- Application and architecture code profiling.
- Infrastructure and application performance tuning.
- Unix or Linux experience.
- Good understanding of OS Concepts & Resources Management.
- General operational expertise such as good troubleshooting skills, understanding of system’s capacity, bottlenecks, Memory, CPU, OS, storage, and networks.
- Proactively monitoring system performance and capacity planning.
- Automation experience (Shell/Python/Ansible).
- Good Understanding of Yarn Framework and Schedulers.
- Should have working experience of using Spark data frame and data sets
- Ability to interpret business requirements and logically think through to developing technical solutions
- Work with DevOps to build automated data pipelines
- Analyze and understand data sources & APIs
- Work with cloud and data engineers to define robust architecture in cloud setup pipelines and workflows
Required Knowledge / Skills / Experience:
- Bachelors or Master’s degree in Computer Science or equivalent experience
- At least 2+ years of IT experience in Big Data/Hadoop environments
- Must have working knowledge of YARN, HBase, Hive, Spark, Flume, Kafka etc.
- Knowledge of programming & scripting languages a plus –Python, R, Perl, Hive, MapReduce, SQL etc.
- Experience with Open Source technologies ELK, Presto, Nodejs.
- Knowledge of Monitoring solutions like Dynatrace, Splunk.
- Spark and Java Applications Development Experience.
- Experience in automating deployments and monitoring/alerting tasks (preferably Ansible).
- Experience in working cloud environments, AWS, OCI etc.
- Experience with Performance, Load, Stress, and Scalability Testing and relevant tooling
- Experience of application software development life cycle. Should be able to influence application projects and provide guidance if needed.
- Strong Problem Solving and creative thinking skills
- Ability to multi-task and manage multiple assignments simultaneously
- Effective oral and written communications
- Experience working with geographically distributed teams
Securonix, Inc. provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, gender, sexual orientation, gender identity, national origin, age, disability, genetic information, marital status, amnesty or status as a covered veteran in accordance with applicable federal, state and local laws. Securonix complies with applicable state and local laws governing nondiscrimination in employment in every location in which the company has facilities. This policy applies to all terms and conditions of employment, including hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation and training.
Securonix expressly prohibits any form of unlawful employee harassment based on race, color, religion, gender, sexual orientation, national origin, age, genetic information, disability or veteran status. Improper interference with the ability of Securonix employees to perform their expected job duties is absolutely not tolerated.