Technical Lead

Role Overview

We are looking for a highly skilled Lead Data Engineer with strong hands-on experience in Databricks, dbt, and Python, and a proven track record of leading and executing data platform migration and modernization programs.

Key Responsibilities

• Data Engineering & Development
• Design, build, and maintain scalable data pipelines using Databricks, PySpark, SQL, and Python.
• Develop and optimize ELT/ETL processes for large-scale data ingestion and transformation.
• Implement robust data quality, monitoring, and reconciliation frameworks.
• Build reusable data engineering components and frameworks.
• Data Platform Migration & Modernization
• Lead migration of legacy data platforms, data warehouses, and ETL ecosystems to Databricks Lakehouse.
• Transform existing ETL workloads to modern ELT patterns using dbt.
• Analyze source environments and define migration approaches, roadmap, and execution strategy.
• Drive code conversion, performance optimization, and workload modernization efforts.
• Databricks Engineering
• Develop Solutions Leveraging Databricks Lakehouse Platform
• Delta Lake
• Unity Catalog
• Databricks Workflows
• Structured Streaming
• Medallion Architecture (Bronze, Silver, Gold)
• Optimize Spark jobs and Databricks workloads for performance and cost efficiency.
• dbt Development
• Build and maintain dbt models, macros, tests, and documentation.
• Implement incremental processing and reusable transformation frameworks.
• Establish data lineage, testing, and CI/CD best practices.
• Leadership & Collaboration
• Lead a team of data engineers and developers.
• Perform code reviews and enforce engineering standards.
• Collaborate with architects, product owners, business analysts, and stakeholders.
• Mentor junior engineers and drive adoption of best practices.

Pay

Salary Range: $120,000 - $200,000 per year