ETL Developer

Responsibilities

• Formulate and define scope and objectives.
• Devise or modify procedures to solve complex problems considering computer equipment capacity and limitations, operating time, and form of desired results.
• Prepare detailed specifications from which programs will be written and design, code, test, debug, and document those programs.

Requirements

• A BS/BA in Engineering, Physics, Computer Science, Information Technology, Mathematics or equivalent, and at least four (4) years’ experience in the field.
• Eight (8) years of related experience is equivalent to a BS/BA in Engineering, Physics, Computer Science, Information Technology, Mathematics or related discipline.
• Strong understanding of Data warehousing techniques and Star/Snowflake models.
• Must have experience with AWS Glue, Data Catalog, Kafka, AWS Lambda, Python, Oracle SQL and Snowflake.
• Must have experience with Data Transformation and Data Integration.
• Must have high level of expertise in writing SQL queries on oracle and Snowflake.
• Experienced with orchestrating event trigger with AWS Lambda or Python to trigger workflows and process.
• Experienced with streaming of data with Kafka and reporting, analyzing and error correction with streams.
• Experienced with utilizing AWS Glue Data quality to evaluate, monitor and enforce data integrity across pipelines.
• Experienced with managing AWS Glue Data Catalog to store, discover and maintain metadata.
• Experienced in building and maintaining data pipelines with AWS Glue.
• Experienced with building and maintaining AWS Data Lake in S3.
• Experienced with Tuning data processing units and configuring workers.

Qualifications

• Candidate must be approved through the DEA’s Background Investigation process for a Public Trust clearance.

Benefits

• Health care
• Dental
• Vision
• Life insurance
• 401(k)
• Education assistance
• Paid time off including PTO, holidays, and any other paid leave required by law

Pay

The salary offered will depend on several factors including, but not limited to, relevant experience, skills, education, geographic location, internal equity, business needs, and other factors permitted by law.

Schedule

N/A

Skills

• Data warehousing techniques and Star/Snowflake models
• AWS Glue, Data Catalog, Kafka, AWS Lambda, Python, Oracle SQL and Snowflake
• Data Transformation and Data Integration
• Writing SQL queries on oracle and Snowflake
• Orchestrating event trigger with AWS Lambda or Python to trigger workflows and process
• Streaming of data with Kafka and reporting, analyzing and error correction with streams
• Utilizing AWS Glue Data quality to evaluate, monitor and enforce data integrity across pipelines
• Managing AWS Glue Data Catalog to store, discover and maintain metadata
• Building and maintaining data pipelines with AWS Glue
• Building and maintaining AWS Data Lake in S3
• Tuning data processing units and configuring workers