Databricks Platform Administrator

About the role

We are seeking an Azure Databricks Platform Support Engineer to ensure the reliability, performance, and security of our Databricks environment. The role focuses on day-to-day platform administration, production troubleshooting, and enabling data engineering and analytics teams across the organisation.

Responsibilities

• Administer and support Azure Databricks workspaces, compute, jobs, notebooks, and SQL Warehouses.
• Troubleshoot and resolve production incidents by analysing job logs, cluster events, error messages, and audit information.
• Manage Unity Catalog governance and ADLS Gen2 integration, including permissions and access controls.
• Configure and maintain compute policies, instance pools, autoscaling, auto-termination, and cost controls.
• Support workspace networking and connectivity, including VNet injection, private endpoints, and Private Link.
• Maintain operational documentation, runbooks, and change records for the platform.

Requirements

• Hands-on experience with Azure Databricks platform support or administration.
• Good understanding of Databricks workspaces, compute, jobs, notebooks, SQL Warehouses, libraries, and permissions.
• Experience troubleshooting Databricks production issues.
• Working knowledge of Unity Catalog and ADLS Gen2 integration.
• Understanding of Azure networking basics: VNet, subnet, NSG, route table, NAT Gateway, firewall, private endpoint, Private Link, and DNS.
• Familiarity with compute policies, instance pools, runtime versions, autoscaling, auto-termination, and cost controls.
• Ability to read job logs, cluster events, error messages, and audit information.
• Strong communication and documentation skills.

Preferred Skills

• Experience with VNet-injected Databricks workspaces and Secure Cluster Connectivity.
• Experience with Azure Key Vault, Azure Monitor, Log Analytics, and diagnostic logs.
• Knowledge of PySpark, Spark SQL, Delta Lake, and Medallion Architecture.
• Experience with Databricks CLI, REST APIs, Terraform, Bicep, Azure DevOps, or GitHub.
• ITIL-style incident, problem, and change management experience.

Tools & Technologies

• Azure Databricks
• Unity Catalog
• Delta Lake
• Apache Spark / PySpark / Spark SQL
• ADLS Gen2
• Azure Key Vault
• Azure Monitor / Log Analytics
• Azure Networking
• Databricks CLI / REST APIs
• Azure DevOps / GitHub

Pay

$55/hr on W2

Schedule

Onsite in Dallas, TX