Senior Data Engineer – Platform Foundation

Key Responsibilities

• Design and implement reusable ingestion components using dlt and dbt-core, covering both structured and unstructured data sources, handling high-volume, append-heavy, and schema-drifting patterns.
• Own the Airflow platform end-to-end: extend and maintain DAGs and shared operators, handle deployments and version upgrades, and provide hands-on support to consuming teams.
• Ensure incremental loading strategies, data quality checks, and lineage metadata are first-class outputs of every pipeline.
• Platform Simplification & Architecture:
  • Identify and eliminate redundant ingestion patterns across consuming teams, drive standardization onto shared Platform Foundation components.
  • Collaborate with Solution Architects to evolve the platform architecture in response to new data sources and shifting business requirements.
  • Support data product exposure: define and implement governed interfaces that make data reliably accessible to internal consumers.
  • Contribute to Terraform-managed infrastructure; participate in multi-cloud (AWS / Azure) deployment patterns.
• AI Tooling & Developer Productivity:
  • Actively use and evaluate AI-assisted development tools (GitHub Copilot, Claude Code, etc.) to accelerate platform Foundation delivery.
  • Champion AI tooling adoption within the squad; share best practices and guardrails around AI-generated code review.
  • Explore AI-powered capabilities (RAG pipelines, LLM-assisted data cataloguing) for internal platform documentation and self-service enablement.
• DevOps & Reliability:
  • Maintain and improve CI/CD pipelines (TeamCity, GitHub Actions) for platform Foundation components.
  • Define and enforce observability standards: DAG/Task-level alerting, SLA tracking.
  • Participate in on-call rotation for critical ingestion pipelines; drive post-incident improvements.
• Team Enablement & Stakeholder Management:
  • Produce platform Foundation documentation, runbooks, and enablement materials for consuming squads.
  • Translate ambiguous or moving business requirements into concrete technical designs — comfortable challenging scope when needed.
  • Mentor mid-level engineers; participate in hiring and technical assessments.

Basic Qualifications

• Bachelor's degree in Computer Science, Engineering, Mathematics, Information Systems, or a related field.
• Minimum 5 years in data engineering roles, with at least 2 years in a senior / platform-level position.
• Proven track record building production ingestion and transformation pipelines at scale.
• Experience contributing to a shared platform or internal developer tooling consumed by multiple teams.

Core Technical Skills

• Python: idiomatic, testable, production-grade code — not just scripting.
• dbt-core: advanced modelling (custom materializations), testing, documentation, packages.
• Apache Airflow: DAG design patterns, custom operators, dynamic task mapping, SLA management.
• Cloud data platforms: comfortable with one or more major cloud warehouses (Snowflake, BigQuery, Databricks, Microsoft Fabric).
• SQL: complex analytical queries, window functions, query profiling.
• Git, CI/CD: trunk-based development, automated testing gates, pipeline-as-code.
• Understands the limits of AI-generated code — applies rigorous review, not blind trust.
• Interest in LLM-powered data tooling (RAG pipelines, Cortex, semantic layers) is a plus.

Our Benefits

• Comprehensive Health & Well-being Coverage.
• Family Building Benefit.
• Generous Paid Time Off.
• Competitive Retirement Savings Plans.
• Income Protection & Insurance Options.
• Company Vehicle Lease Program.
• Tuition reimbursement.
• Student loan refinancing programs.
• 18 paid volunteer hours each year to make a difference in your community.