Azure Senior Data Lead

Responsibilities

• Data Architect / Senior Data Engineer with 12+ years of experience designing and modernizing enterprise data platforms across banking, healthcare and retail domains
• Architected scalable ETL/ELT pipelines using Python, PySpark, Databricks, AWS Glue and Azure Data Factory, supporting high-volume transactional and regulatory data processing.
• Led enterprise-scale Data Architecture initiatives, defining logical and physical data models, governance standards and cloud-native platform blueprints across AWS and Azure environments.
• Designed and implemented Medallion (Bronze/Silver/Gold) Lakehouse architectures using Delta Lake, S3, ADLS Gen2, Snowflake, Redshift and Synapse Analytics.
• Engineered large-scale distributed processing workloads using Apache Spark, PySpark, Databricks, EMR, Hive and HDFS, processing billions of records for enterprise analytics.
• Orchestrated complex data workflows using Apache Airflow, Databricks Workflows, AWS Step Functions and Azure Data Factory triggers, ensuring SLA-driven pipeline execution.
• Strong hands-on experience in Advanced SQL, including complex joins, CTEs, window functions, stored procedures, indexing strategies, partitioning and execution plan optimization across Snowflake, PostgreSQL and Oracle.
• Built real-time streaming architectures using Apache Kafka, AWS Kinesis, Azure Event Hub and Service Bus, supporting fraud detection, claims monitoring and operational telemetry.

Requirements

Required:

• 12+ years of experience in data engineering and architecture
• Experience with ETL/ELT pipelines using Python, PySpark, Databricks, AWS Glue and Azure Data Factory
• Experience with Medallion (Bronze/Silver/Gold) Lakehouse architectures using Delta Lake, S3, ADLS Gen2, Snowflake, Redshift and Synapse Analytics
• Experience with large-scale distributed processing workloads using Apache Spark, PySpark, Databricks, EMR, Hive and HDFS
• Experience with complex data workflows using Apache Airflow, Databricks Workflows, AWS Step Functions and Azure Data Factory triggers
• Strong hands-on experience in Advanced SQL, including complex joins, CTEs, window functions, stored procedures, indexing strategies, partitioning and execution plan optimization across Snowflake, PostgreSQL and Oracle
• Experience with real-time streaming architectures using Apache Kafka, AWS Kinesis, Azure Event Hub and Service Bus

Qualifications

Preferred:

• Experience with Azure Data Factory
• Experience with Medallion (Bronze/Silver/Gold) Lakehouse architectures
• Experience with Apache Spark, PySpark, Databricks, EMR, Hive and HDFS
• Experience with Apache Airflow, Databricks Workflows, AWS Step Functions and Azure Data Factory triggers
• Experience with complex data workflows
• Experience with real-time streaming architectures

Skills

• Advanced SQL
• ETL/ELT Pipelines
• Medallion (Bronze/Silver/Gold) Lakehouse Architectures
• Azure Data Factory
• Apache Spark, PySpark, Databricks, EMR, Hive and HDFS
• Apache Airflow, Databricks Workflows, AWS Step Functions and Azure Data Factory triggers
• Complex Data Workflows
• Real-Time Streaming Architectures

Benefits

• Medical
• Dental
• Vision
• Pharmacy
• Life
• Accidental Death & Dismemberment
• Disability Insurance
• Employee Assistance Program
• 401(k) Retirement Plan
• 10 Days of Paid Time Off per Year (some positions are eligible for need-based leave with no designated number of leave days per year)
• 10 Paid Holidays per Year

Pay

The exact pay rate will vary based on skills, experience, and location and will be determined by the third-party employer.

Schedule

Full-time / Contract