Technical Lead-Data Engineer

About the role

We are looking for a highly skilled Technical Lead – Data Engineering to guide our engineering team in building and scaling a robust, modern data platform. In this role, you will be the bridge between architectural blueprints and engineering execution. You will remain deeply hands-on with our core stack—Databricks, dbt, and Apache Airflow—while mentoring engineers, conducting rigorous code reviews, and ensuring the delivery of high-quality data products.

Responsibilities

• Technical Leadership & Delivery
  • Lead a team of data engineers to execute sprint goals, managing code quality and delivery timelines.
  • Act as the subject matter expert (SME) for the team on Databricks, dbt, and Airflow best practices.
  • Enforce engineering standards, including code modularity, documentation, and version control.
  • Conduct comprehensive code reviews to ensure scalability, security, and performance.
• Hands-on Engineering & Optimization
  • Build and maintain production-grade data pipelines using PySpark, Delta Live Tables (DLT), and Spark SQL on Databricks.
  • Develop complex dbt models, custom macros, and tests to transform raw data into analytics-ready layers.
  • Author and schedule sophisticated Apache Airflow DAGs, utilizing dynamic task mapping and custom operators.
  • Optimize pipeline performance by troubleshooting bottlenecked Spark jobs, Z-Ordering Delta tables, and tuning Airflow schedulers.
• DataOps & Governance
  • Implement and manage CI/CD pipelines for data deployments using toolsets like GitHub Actions or Azure DevOps.
  • Enforce data governance policies, access controls, and lineage tracking via Databricks Unity Catalog.
  • Integrate automated data quality testing and alerting mechanisms across dbt and Airflow workflows.
• Mentorship & Collaboration
  • Mentor and upskill junior and mid-level data engineers through pair programming and workshops.
  • Translate architectural designs into actionable, granular engineering tasks and JIRA tickets.
  • Collaborate closely with data architects, product managers, and downstream data analysts.

Requirements

• 10+ years of professional experience in data engineering and backend software development.
• 2+ years of experience in a technical lead, team lead, or mentoring capacity.

Skills

• Databricks: Strong hands-on experience with Delta Lake, Unity Catalog, and optimizing PySpark workloads.
• dbt: Proficiency with dbt Core or Cloud, including advanced macros, packages, and custom testing.
• Airflow: Solid experience writing complex, reliable DAGs, handling task failures, and managing dependencies.
• Languages: Advanced proficiency in Python and expert-level SQL writing.
• Cloud Platforms: Hands-on experience deploying data solutions on at least one cloud provider (AWS, Azure, or GCP).
• Leadership Skills:
  • Proven ability to guide engineering sprint velocity and resolve technical blockers for a team.
  • Excellent verbal and written communication skills to articulate technical trade-offs.

Pay

$130,000–$200,000 a year