Data Engineer (Data Warehouse, ADF, Stored Procedures)

Job Category: Engineer
Job Type: Remote
Job Location: Minneapolis Minnesota
Compensation: Depends on Experience
W2: W2-Contract Only; Kindly note that applications on a C2C basis will not be considered for this role.
JPS-3035 | Posted On: 03/31/2025 | Closes On: 04/06/2025
Description:

100% Telecommute

Work Hours: 9am-5pm CST

Project:

  • As a member of the Data Management team, the Data Engineer supports the Alabama EDS by developing and maintaining workflows, identifying, and resolving data quality issues, and optimizing processes to improve performance. 
  • The Data Engineer will also support intrastate agencies by monitoring automated data extracts and working directly with state partners to create new extracts based on business specifications.

Responsibilities:

  • Develop and manage effective working relationships with other departments, groups, and personnel with whom work must be coordinated or interfaced
  • Efficiently communicate with ETL architect while understanding the requirements and business process knowledge in order to transform the data in a way that’s geared towards the needs of end users
  • Assist in the overall architecture of the ETL Design, and proactively provide inputs in designing, implementing, and automating the ETL flows
  • Investigate and mine data to identify potential issues within ETL pipelines, notify end-users and propose adequate solutions
  • Developing ETL pipelines and data flows in and out of the data warehouse using a combination of Azure Data Factory and Snowflake toolsets
  • Developing idempotent ETL process design so that interrupted, incomplete, or failed processes can be rerun without errors using ADF dataflows and Pipelines
  • Ability to work in Snowflake Virtual Warehouses as needed in Snowflake and automate data pipelines using Snowpipe for tedious ETL problems  
  • Capturing changes in data dimensions and maintaining versions of them using Stream sets in snowflake and scheduling them using Tasks
  • Optimize every step of the data movement not only limited to source and during travel but also when it’s at rest in the database for accelerated responses
  • Must have the ability to build a highly efficient orchestrator that can schedule jobs, execute workflows, perform Data quality checks, and coordinate dependencies among tasks
  • Responsible for testing of ETL system code, data design, and pipelines and data flows. Root cause analysis on all processes and resolving production issues are also a part of the process and routine tests on databases and data flow and pipeline testing
  • Responsible for documenting the implementations, and test cases as well as responsible for building deployment documents needed for CI/CD

Ideal Background: Data Engineer with Healthcare (Medicaid) and Microsoft Azure based experience with Snowflake and Azure Data Factory

TOP REQUIREMENTS:

  • 5+ years of Data engineering experience with a focus on Data Warehousing
  • 2+ years of experience creating pipelines in Azure Data Factory (ADF)
  • 3+ years of experience creating stored procedures with Oracle PL/SQL, SQL Server T-SQL, or Snowflake SQL

Required:

  • 5+ years of Data engineering experience with a focus on Data Warehousing
  • 2+ years of experience creating pipelines in Azure Data Factory (ADF)
  • 5+ years developing ETL using Informatica PowerCenter, SSIS, Azure Data Factory, or similar tools.
  • 5+ years of experience with Relational Databases, such as Oracle, Snowflake, SQL Server, etc.
  • 3+ years of experience creating stored procedures with Oracle PL/SQL, SQL Server T-SQL, or Snowflake SQL
  • 2+ years of experience with GitHub, SVN, or similar source control systems
  • 2+ years of experience processing structured and un-structured data.
  • Experience with HL7 and FHIR standards, and processing files in these formats.
  • 3+ years analyzing project requirements and developing detailed specifications for ETL requirements.
  • Excellent problem-solving and analytical skills, with the ability to troubleshoot and optimize data pipelines.
  • Ability to adapt to evolving technologies and changing business requirements.
  • Bachelors or Advanced Degree in a related field such as Information Technology/Computer Science, Mathematics/Statistics, Analytics, Business

Preferred:

  • 2+ years of batch or PowerShell scripting
  • 2+ years of experience with Python scripting.
  • 3+ years of data modeling experience in a data warehouse environment
  • Experience or familiarity with Informatica Intelligent Cloud Services (specifically Data Integration)
  • Experience designing and building APIs in Snowflake and ADF (e.g. REST, RPC)
  • Experience with State Medicaid / Medicare / Healthcare applications
  • Azure certifications related to data engineering or data analytics.