DevOps Engineer

Responsibilities

Develop and maintain CI/CD pipelines in Azure DevOps to deploy notebooks, jobs, and workflows to Azure Databricks.

Automate job runs in Databricks using Databricks CLI, REST API, or DevOps tasks.

Maintain infrastructure-as-code practices (e.g., using Terraform).

Monitor and troubleshoot pipeline failures, notebook issues, and deployment inconsistencies.

Collaborate with Data Engineers and Platform teams to maintain development, staging, and production environments.

Implement best practices for version control, testing, and notebook promotion in a multi-environment setup.

Store and manage Databricks secrets, tokens, and configuration securely within Azure Key Vault and DevOps variable groups.

Understand and apply DevOps practices in a data platform context.

Requirements

Bachelor’s degree in computer science, Information Technology, or related field, or equivalent combination of education and experience.

Minimum of three (3) years of experience in DevOps engineering or a related role, with a focus on Microsoft Azure, including other cloud-based data integration and associated technologies.

Proficiency with data analytics and AI platforms and automating notebook execution.

Proficiency in scripting languages for automation and various programming.

Qualifications

Comprehensive Benefits - Medical, dental, and vision insurance, employee assistance program, employer-paid and voluntary life insurance, disability insurance, plus health and flexible spending accounts

Financial & Retirement Support – Competitive compensation, 401k with employer match, and financial wellness resources

Time Off & Leave – Paid holidays, flexible vacation time/PSSL, and paid parental leave

Wellness & Growth – Work life assistance resources, physical wellness perks, mental health support, employee referral program, and BenefitHub for employee discounts