AWS Databricks Engineering Lead (1146996)

Location: Irvine, CA

About the role

We are seeking an experienced AWS Databricks Engineering Lead to design, build, and oversee enterprise-scale cloud-native data platforms supporting Asset Management business functions. This role will lead platform engineering, cloud infrastructure, DevOps automation, platform reliability, security, and operational excellence across the Enterprise Data Office (EDO).

Responsibilities

• Lead the design, implementation, and management of enterprise data platforms.
• Define platform engineering standards, best practices, and automation frameworks.
• Architect and administer Databricks platforms, including Delta Lake, Unity Catalog, DLT, MLflow, and Structured Streaming.
• Drive AWS cloud engineering initiatives utilizing S3, ECS, EKS, Lambda, Glue, and CloudWatch.
• Develop Infrastructure-as-Code solutions using Terraform.
• Build and manage CI/CD pipelines using GitHub Actions, GitLab CI/CD, Jenkins, and Azure DevOps.
• Support containerized environments leveraging Docker and Kubernetes.
• Establish governance, security, monitoring, and operational controls across environments.
• Implement Site Reliability Engineering (SRE) practices, including SLAs, SLOs, observability, and incident management.
• Drive platform performance, scalability, cost optimization, and continuous improvement.

Requirements

• Strong hands-on experience with Databricks, dbt, Airflow, AWS, Terraform, and Python.
• Experience leading cloud platform engineering and DevOps initiatives.
• Expertise in CI/CD, Infrastructure as Code, Docker, and Kubernetes.
• Experience supporting Asset Management or Investment Management organizations.
• Strong knowledge of platform reliability, security, governance, and operational best practices.

Pay

$130,000.00 USD Annually - $140,000.00 USD Annually