Senior Data Platform Engineer (Python)

About the role

We are looking for a Senior Data Platform Engineer to design, build, and operate scalable data pipelines and lakehouse architectures across batch and streaming workloads. You will work across Python, SQL, Airflow, Spark/PySpark, dbt, and modern AWS data services, with a strong focus on data quality, observability, and repeatable delivery.

Responsibilities

• Build and maintain scalable data pipelines in Python and SQL, orchestrated with Airflow, and delivered into S3-backed data lakes and Redshift/Athena warehouses.
• Design and implement lakehouse architectures using modern table formats such as Apache Iceberg and Delta Lake.
• Own data quality and reliability by implementing validation frameworks, data contracts, and observability practices including monitoring, alerting, SLIs, and SLOs.
• Optimize large-scale batch and streaming workloads with Spark/PySpark and dbt, balancing performance, cost, and maintainability across GBs to TBs of data.
• Drive infrastructure as code and CI/CD best practices for data workflows so deployments are repeatable, testable, and observable.
• Collaborate with analytics, product, and engineering stakeholders to turn raw data into dependable platform capabilities.

Requirements

• 5+ years of experience designing and building production ETL/ELT pipelines at scale.
• Hands-on expertise with Python, advanced SQL, and data modeling.
• Production experience with Airflow, data transformation, and Apache Spark/PySpark optimization.
• Practical AWS experience across S3 and data warehouses, plus CI/CD practices including Git, testing, and deployment automation.
• Experience building data quality frameworks, data contracts, and schema evolution strategies.
• Nice to have: modern table formats, medallion architecture, dbt, streaming platforms, and data observability tooling.