Senior Engineer, ML Engineering (Databricks)

Responsibilities

• Manage and support ML and AI model lifecycle using Databricks and MLflow: training, tracking, packaging, deployment, monitoring, agentic AI safety and generative AI guardrails
• Build and maintain robust and scalable monitoring pipelines for ML model performance, agentic and generative AI guardrails and safety, and LLMs evaluators to ensure SLA adherence, scalability, reproducibility, and reliability
• Implement CI/CD pipelines for ML and AI workflows, including testing, version control, and rollback strategies
• Support governance and compliance requirements: model registry usage, auditability, and reproducibility
• Document processes, automation frameworks, and operational runbooks

Requirements

Strong hands-on experience with Databricks: Workflows, Delta Lake, Delta Live Tables, Asset Bundles, Unity Catalog, Genie Code.

Proficiency with MLflow for experiment tracking, model registry, and deployments.

Experience deploying models to Databricks Model Serving, REST APIs, or external endpoints.

Solid understanding of CI/CD tools (e.g., Databricks Asset Bundles, Jenkins, etc.).

Competency with shell scripting in a Linux environment

Nice to have: Exposure to contemporary big data and AI stacks and technologies: Yarn, AWS, Spark, Databricks, distributed computing, chatGPT, Claude, Copilot

Solid understanding of data structures and algorithms

Some experience with real-time processing

Some experience with production support

Solid verbal and written communication skills

Team player and ability to work well with others in an intellectually challenging environment

Qualifications

Desired Skills & Expertise

• Strong hands-on experience with Databricks: Workflows, Delta Lake, Delta Live Tables, Asset Bundles, Unity Catalog, Genie Code.
• Proficiency with MLflow for experiment tracking, model registry, and deployments.
• Experience deploying models to Databricks Model Serving, REST APIs, or external endpoints.
• Solid understanding of CI/CD tools (e.g., Databricks Asset Bundles, Jenkins, etc.).
• Competency with shell scripting in a Linux environment
• Nice to have: Exposure to contemporary big data and AI stacks and technologies: Yarn, AWS, Spark, Databricks, distributed computing, chatGPT, Claude, Copilot
• Solid understanding of data structures and algorithms
• Some experience with real-time processing
• Some experience with production support
• Solid verbal and written communication skills
• Team player and ability to work well with others in an intellectually challenging environment

Benefits

Competitive benefits, including a range of Financial, Health and Lifestyle benefits to choose from

Flexible working options, including home working, flexible hours and part time options, depending on the role requirements – please ask!

Unlimited PTO policy, floating holidays, volunteering days and a day off for your birthday

Learning and development tools to assist with your career development

Regular social events and networking opportunities

Collaborative, supportive culture, including an active DE&I program

Employee Assistance Program which provides expert third-party advice on well-being, relationships, legal and financial matters, as well as access to counselling services