Lead Data Engineer
Role Overview
We are seeking a Lead Data Engineer to build and operate enterprise-grade data pipelines using Microsoft Azure as the orchestration layer and Snowflake as the enterprise data lake/warehouse. This role focuses on reliable ETL/ELT, push/pull data integration patterns, and scalable platform engineering in a GxP / validated environment.
Key Responsibilities
The Lead Data Engineer will be responsible for developing and maintaining ETL/ELT pipelines for ingesting data from various sources into our data warehouse.
Design and build batch and incremental data pipelines using Azure-native orchestration services to move data into and out of Snowflake.
Support push and pull architectures across files, APIs, enterprise applications, and external partners.
Develop curated, consumption-ready datasets in Snowflake using standardized, reusable patterns.
Implement data quality, and reconciliation controls within pipelines.
Follow CI/CD and controlled deployment practices across Dev / QA / Prod environments.
Ensure solutions meet GxP expectations for data integrity, traceability, auditability, and change control.
Partner with Quality, Validation, and platform teams to support compliance and audit readiness.
Data Catalog, Metadata & Governance Enablement:
Publish and maintain data assets, schemas, and lineage in the enterprise data catalog.
Partner with Data Governance and Business teams to support certified datasets, ownership, and usage transparency.
Work Style
Work with an end-to-end ownership mindset, innovate and drive initiatives through completion.
Optimize data storage and retrieval to ensure efficient performance and scalability.
Collaborate with data architects, data analysts and data scientists to understand their data needs and ensure that the data infrastructure supports their requirements.
Ensure data quality and integrity through data validation and testing.
Implement and maintain security protocols to protect sensitive data.
Stay up-to-date with emerging trends and technologies in data engineering and analytics.
Community lead to enable adoption of data and technology strategy.
Knowledgeable in evolving trends in Data platforms and Product based implementation.
Comfortable working in a fast-paced environment with minimal oversight.
Prior experience working in an Agile/Product based environment.
Data engineering, pipelines, AI/ML development, requirements gathering with Business Partners.
Technical delivery execution (build, test, deploy).
Technical estimates and feasibility.
Resource planning within Data & AI team.
Qualify/Validate system and pipelines.
FUT/SIT completion.
Technical Project Management.
Requirements
Bachelor’s degree in Computer Science, Engineering, Information Systems, or related field (Master’s preferred).
7-10 years of hands-on data engineering experience.
Total 15+ years of IT experience
Strong experience with Microsoft Azure orchestration/integration services.
Strong experience working with Snowflake as a data lake/warehouse.
Proven experience with ETL/ELT and push/pull integration patterns.
Experience operating in regulated or validated (GxP) environments.
Strong SQL and data engineering fundamentals.
Experience with enterprise data platforms and multi-domain data architectures.
Familiarity with CI/CD, role-based access control, and secure data sharing.
Experience in Data Catalog tools and implementation.
Experience integrating ERP, manufacturing, or regulated systems.
Strong programming skills (Python, SQL; Java/Scala a plus)
Experience with data warehousing and lakehouse architectures.
Hands-on experience with ETL/ELT tools and orchestration frameworks.
Proficiency with relational and non-relational databases.
Strong understanding of data modeling, normalization, and performance tuning.
Cloud data platforms (e.g., Azure Synapse, Snowflake, BigQuery, Redshift).
Data processing frameworks (e.g., Spark).
Orchestration tools (e.g., Airflow).
Version control and CI/CD practices.