Senior Data Engineer

About the role

Design, build, and maintain the data pipelines, transformation layers, and data models that power the enterprise lakehouse. This role serves as a technical anchor on the data engineering team, delivering robust ELT/ETL solutions and mentoring junior engineers.

Responsibilities

• Design and implement scalable batch and streaming data pipelines using Apache Spark, Kafka, and Flink
• Build and maintain the Bronze/Silver/Gold medallion architecture within the lakehouse (Delta Lake / Iceberg)
• Develop and optimize complex SQL and PySpark transformations for large-scale datasets
• Integrate structured, semi-structured, and unstructured data sources into the lakehouse
• Collaborate with data architects to evolve the physical and logical data models
• Implement data quality checks and monitoring using Great Expectations or dbt tests
• Write Infrastructure-as-Code for pipeline environments (Terraform, Helm)
• Participate in code reviews and enforce engineering standards and best practices
• Troubleshoot pipeline failures, performance bottlenecks, and data incidents
• Mentor junior and mid-level data engineers and contribute to internal knowledge sharing

Requirements

• 6+ years of data engineering experience with a track record of enterprise-scale delivery
• Expert proficiency in Python and SQL; PySpark experience required
• Hands-on experience with Apache Spark, Delta Lake, or Apache Iceberg
• Experience with orchestration tools: Apache Airflow, Prefect, or Dagster
• Strong knowledge of cloud data services: AWS Glue, Azure Data Factory, GCP Dataflow
• Proficiency with version control (Git), CI/CD pipelines, and containerization (Docker/Kubernetes)
• Experience with dbt (data build tool) for transformation layer management
• Bachelor's degree in Computer Science, Engineering, or related technical field

Preferred Qualifications

• Experience with Databricks, Snowflake, or Apache Hudi
• Knowledge of streaming architectures and Apache Kafka
• Certifications: Databricks Certified Data Engineer, AWS Data Analytics Specialty

Pay

Base salary range: $130,000–$150,000 per year. A discretionary incentive program may be provided as part of the compensation package, in addition to a full range of medical and other benefits, dependent on full-time employment status.