Job Description
US Work Authorization Requirement:
Candidates must be legally authorized to work in the United States without employer sponsorship. This includes, but is not limited to, U.S. Citizens, Permanent Residents, and other individuals with valid U.S. work authorization.
Job Description:
As part of the Network Analytics team, the Sr Data Engineer role is crucial in executing advanced analytic techniques to enhance security and operational efficiency of client enterprise networks. This position focuses on constructing robust ETL pipelines using Spark to ensure data availability and quality for analytics tasks. The role requires expertise in data engineering, leveraging AWS for scalable solutions and enhancing operational efficiency. The role also requires expertise in statistical analysis and machine learning, utilizing generative AI to build data-driven, automated systems that enhance operational efficiency.
Roles & Responsibilities:
Design, develop, and maintain scalable ETL pipelines to ensure data quality and availability
Implement monitoring and alerting solutions to ensure data pipeline reliability and performance
Develop and manage deployment pipelines to facilitate continuous integration and delivery of data engineering solutions
Implement data integration solutions to support analytics and reporting needs
Execute the complete analytics lifecycle for problem solving, including:
- Algorithm traditionalization
- Model validation
- Model prototyping
- Data exploration
- Data grooming
Survey varied data sources for analytic relevance, including:
- External sources accessed via API
- Flat files
- Relational databases
- Distributed file systems
Interpret, synthesize and communicate results of analyses to effect action and changes within the organization
Collaborate across teams to integrate analytic products with existing production architecture, develop, execute, and evaluate courses of action, and socialize results
Help teach and explain techniques and tools used to a broad set of business
Required Qualifications:
Skills/Abilities and Knowledge
- Ability to read, write, speak and understand English
- Strong communication and presentation skills
- Expertise in data engineering languages such as Scala (preferred) or Java, with proficiency in Python
- Experience with BigData tools, particularly Spark
- Proficiency in building and managing ETL pipelines
- Expert-level quantitative analysis skills including interpretation of model results, consideration of causality, treatment of multicollinearity
- The ability to work in compiled, high-performance languages (e.g., Scala, Java, C++)
- Experience with relational databases
- Strong understanding of relational databases and SQL, and familiarity with NoSQL databases
- Broad experience and solid theoretical foundation on the modeling process using a
- variety of algorithmic techniques, including Machine Learning, and Graph/Network Analytics
- Data pre-processing, exploratory data analysis using a variety of techniques
- Basic understanding of data architecture, data warehouse, and data marts
- Demonstrated ability and desire to continually expand skill set, and learn from and teach others
Preferred Qualifications:
- Experience with Linux-based operating systems
- Knowledge of other relevant techniques such as text analysis and text mining
- Bachelors degree (or relevant experience) in computer science, mathematics, statistics, physics, operations research, or other quantitatively-focused fields
- Masters: 6+ Years Work Experience
- Bachelors: 8+ Years Work Experience
