ETL Data Engineer

About The Role

Manage and Lead complex initiatives within Technology and contribute to large scale data processing framework initiatives related to enterprise strategy deliverables
Build and maintain optimized and highly available data pipelines that facilitate deeper analysis and reporting
Review and analyze moderately complex business, operational or technical challenges that require an in-depth evaluation of variable factors
Oversee the data integration work, including developing a data model, maintaining a data warehouse and analytics environment, and writing scripts for data integration and analysis
Resolve moderately complex issues and lead teams to meet data engineering deliverables while leveraging solid understanding of data information policies, procedures and compliance requirements
Collaborate and consult with colleagues and managers to resolve data engineering issues and achieve strategic goals.

Required Qualifications

• 8+ years of Data Engineering experience, or equivalent demonstrated experience.
• 7+ years of Strong Experience of Ab Initio, Spark Core, Spark SQL, Data Frames, Datasets
• 2+ Experience on Python or other scripting languages
• Hands-on experience with GCP or other cloud platforms (AWS/Azure)
• 4+ years of data warehouse experience, SQL experience (Teradata, SQL, PL/SQL).
• Understand and write the complex SQL queries and have in-depth understanding of oracle database.
• Strong experience with Teradata, Hadoop, Hive, Autosys, Airflow, BigQuery SparkFlow.
• Good understanding of distributed system and parallel processing, spark architecture in depth.
• Ability to conduct code reviews with focus on testability and code coverage.
• Must have executed end to end projects in data engineering.
• Must have experience working in an Agile environment using Scrum.
• Excellent in communication.

Skills

• Data warehouse
• AbInitio
• ETL
• Spark
• SQL
• Data engineering
• Data integration
• Data