AWS Data Engineer

About the Role

Design and implement modern data engineering tools to manage petabytes of data using offline and online data integrations. Execute the entire data engineering life cycle, including problem formulation, data acquisition and assessment, data ingestion, feature selection and engineering, data model development and fine-tuning, performance measurement, and delivery of consumption modules (applications, dashboards, or APIs).

Responsibilities

• Build highly optimized and scalable data pipelines (ETL) using batch and stream processing frameworks (Spark).
• Develop and optimize SQL queries, procedures, and scripts.
• Implement data warehousing, data lake, and analytic processes and methodologies.
• Design and build monitoring and alerting dashboards to identify operational issues in data pipelines.
• Optimize data and compute capacity resources for on-demand infrastructure scaling, cluster utilization, and application SLAs.
• Work on security compliance and contribute to data and service access controls.
• Implement best practices for disaster recovery and service reliability.

Requirements

• Experience with data integration (ETL/ELT), data quality, and multi-dimensional data modeling.
• Proficiency in Amazon Redshift, AWS Glue, AWS CloudTrail, Python, SQL, Amazon Kinesis Data Streams, S3 Standard, Amazon S3 Glacier, AWS Key Management Service, and NoSQL databases.