Data Engineer

Location: Charlotte, North Carolina | Job type: Contract | Work hours: 9am to 6pm

About the role

The primary role of the Data Engineer is to function as a critical member of a data team by designing data integration solutions that deliver business value in line with the company's objectives. They are responsible for the design and development of data/batch processing, data manipulation, data mining, and data extraction/transformation/loading into large data domains using Python/Pyspark and AWS tools.

Responsibilities

• Design and develop data integration solutions using Python and Pyspark.
• Work with AWS cloud services like Glue, Lambda, MSK (Kafka), S3, Step functions, RDS, and EKS.
• Utilize databases such as Postgres, SQL Server, Oracle, and Sybase.
• Write and optimize SQL queries, stored procedures, views, functions, and triggers.
• Develop and maintain data pipelines and ETL processes.
• Implement CI/CD DevOps processes using tools like GitHub and Jenkins.
• Apply data modeling and warehousing concepts in an Agile/SCRUM environment.
• Leverage AI tools effectively for data engineering tasks.
• Demonstrate strong problem-solving and analytical skills.

Requirements

• Minimum 8 years of hands-on data engineering experience in development using Python and Pyspark.
• Hands-on experience with AWS services (Glue, Lambda, MSK/Kafka, S3, Step functions, RDS, EKS).
• Experience with databases like Postgres, SQL Server, Oracle, and Sybase.
• Strong SQL database programming skills, including performance tuning, relational model analysis, queries, stored procedures, views, functions, and triggers.
• Experience in design (mapping specifications, HLD, LLD) and development (coding, unit testing).
• Knowledge of UNIX scripting and Oracle SQL/PL-SQL.
• Experience with data models, data mining, data analysis, and data profiling.
• Experience working with REST APIs.
• Familiarity with workload automation tools like Control-M or Autosys.
• Good knowledge of CI/CD DevOps tools such as Bitbucket, GitHub, and Jenkins.
• Strong experience with Agile/SCRUM methodology.
• Experience with other ETL tools (DataStage, Informatica, Pentaho, etc.).
• Knowledge of MDM, data warehousing, and data analytics.

Qualifications

• Bachelor’s degree in a relevant field.

Pay

$60 - 66 per hour

Benefits

• Comprehensive benefits package, including medical, prescription, dental, vision, AD&D, and life insurance offerings.
• Short-term disability coverage.
• 401K plan (all benefits based on eligibility).