Data & ML Engineer

General Function

The Data & ML Engineer is a self-sufficient engineering professional responsible for designing, building, and operating scalable, secure, and reliable data and machine learning platforms primarily on Azure, with exposure to multi-cloud environments (AWS, GCP) where applicable.

Expertise in programming languages such as Python or Scala, experience with data processing frameworks like Spark, and familiarity with container orchestration tools such as Kubernetes are essential for this role. Proficiency with CI/CD pipelines, DevOps, and MLOps practices is expected to ensure robust deployment and operationalization of analytics and AI solutions.

This role complements the Applied Data Scientist by owning the engineering foundations required to operationalize analytics and AI solutions.

Major Duties and Responsibilities

• Managing the design, development, and operation of large-scale data ingestion, transformation, and storage pipelines

• Managing ML infrastructure, CI/CD, DevOps, and MLOps pipelines to support model training and deployment

• Managing platform performance, cost optimization, reliability, and availability

• Managing data security, governance, and regulatory compliance across platforms

• Managing collaboration with Applied Data Scientists to productionize models

• Designing and organizing ETL/ELT workflows using Azure Data Factory and orchestration tools

• Structuring Lakehouse, Azure Data Lake, and Synapse environments for scalable analytics

• Organizing data formats, schemas, and versioning (Delta, Parquet, JSON, CSV)

• Structuring reusable data pipelines and ML components to accelerate delivery

• Organizing monitoring, logging, and alerting for data and ML pipelines

• Leading engineering best practices for scalable data and ML platforms

• Driving automation-first and infrastructure-as-code approaches

• Guiding solution design to ensure performance, resilience, and cost efficiency

• Leading troubleshooting and root-cause analysis for data and ML pipeline issues

• Mentoring engineers on cloud-native, big data, and MLOps practices

Basic Qualification

• Bachelor’s degree in Computer Science, Engineering, or a related field

• Proven experience as a Data Engineer, ML Engineer, or Platform Engineer

• Strong hands-on experience with Azure cloud services and big data platforms

• Proficiency in Python, SQL, Scala, and scripting languages

• Strong experience building production-grade data pipelines

• Demonstrated ability to independently own and deliver complex data and ML engineering solutions end-to-end

PREFERRED QUALIFICATIONS

• Master’s degree in Computer Science, Engineering, or related discipline

• Experience with Azure Databricks, Spark, Synapse, and MLFlow

• Experience with Docker, AKS, APIs, and containerized ML workloads

• Experience with orchestration tools such as Azure Data Factory or Airflow

• Exposure to SAP CDC and enterprise data integration

• Experience working in agile, fast-paced, cross-functional environments

Tech and Behavioral Skills

• Strong engineering ownership mindset with minimal supervision

• Ability to translate analytical requirements into scalable engineering solutions

• Strong collaboration skills with data scientists and business-facing teams

• Excellent problem-solving and troubleshooting capabilities

• Focus on reliability, scalability, and operational excellence