Senior Data Engineer

Responsibilities

• Data Engineering and Operations (60%): Develop, deploy, and maintain platform components, including: data connections, data pipelines, Databricks Lakeflow jobs, related GitHub Actions and Databricks Declarative Automation Bundle (DABs) pipelines, and various monitoring frameworks.
• Create and maintain data engineering templated project repositories enabled and simplified for the internal platform team with custom CLI.
• Maintain team documentation for the internal platform and for external business users of the platform (i.e. developers).
• Mentor junior data engineers within the internal team, including reviewing PRs and ensuring code quality are standardized and rigorously reviewed on a platform level.
• Partner with other members of the internal team, IT, data science, and analytics engineering teams to support end-to-end solution delivery, including building and promoting data pipelines, provisioning the related Databricks DAB infrastructure for data ingestion and transformation, and supporting DevOps pipelines (GitHub Actions) and processes.
• Provide support and guidance across the SDLC, including CI/CD, release management, and environment promotion.
• Provide ad hoc support for application deployment for Databricks Apps space and Azure App Services.
• Help evaluate emerging Azure and Databricks services and AI/ML capabilities to drive innovation and continuous improvement.
• Contribute to data engineering platform governance, including data ethics, compliance, and security best practices.
• Mentor technical teams and promote architectural standards and reusable patterns.

Qualifications

• 6+ years of experience in a combination of Data Engineering, DevOps, and App Development roles.
• Experience configuring and maintaining Databricks Workspaces, ACLs, and other Workspace artifacts.
• Experience working with Databricks (Lakehouse architecture, Spark, Unity Catalog, Delta tables).
• Experience with DevOps, and SDLC practices in a regulated enterprise environment.
• Experience creating GitHub Actions Workflows for code promotion across environments.
• Experience creating data ingestion and data transformation pipelines in Pyspark, preferably within Databricks.

Preferred Qualifications

• Experience with data masking, synthetic data generation, and secure data provisioning.
• Familiarity with AI governance frameworks, including model traceability, explainability, and bias mitigation.
• Knowledge of cost optimization strategies and platform usage transparency.
• Experience with orchestration and event-driven architecture.
• Kubernetes experience.
• Databricks and Terraform expertise.

Benefits

The salary range for this role is $120,000 to $170,000 or the hourly equivalent. Pay is based on several factors including but not limited to education, work experience, certifications, etc. In addition to your salary, Turnberry Solutions offers benefits such as a comprehensive healthcare package (medical, dental, vision), disability and group term life insurance, health and flexible spending accounts, a utilization bonus, 401(k) with match, flexible time off for salaried employees, parental leave for salaried employees, and flexible work arrangements (all benefits are subject to eligibility requirements).