Lead Machine Learning Engineer

We are seeking a Senior Lead / Lead ML Platform Engineer to architect and own the technical direction for our Training and Inference infrastructure. This high-leverage role is designed for an expert who understands the deep technical stack required to shift ML models from research to global production. You will be responsible for the "engine room" of the Applied Machine Learning Group (AMLG), ensuring that our MLEs can train massive models efficiently and serve them with sub-millisecond reliability.

About the role

The ML Platform Lead is the force-multiplier for every other ML pod. In this role, you will directly shape:

• The Training Foundation: Establishing AnyScale/Ray as the standard for distributed compute, enabling MLEs to train models on petabytes of data without managing infrastructure.
• Inference at Scale: Architecting the serving layer that handles billions of requests per day, optimizing for both p99 latency and GPU utilization.
• Operational Excellence: Setting the organizational standards for how ML models are deployed, monitored, and scaled across the enterprise.

Responsibilities

• Own the long-term architectural direction for the Training and Inference domains, ensuring the platform scales 10x over a 1–3 year horizon.
• Lead the implementation and optimization of Ray/AnyScale, providing a unified compute layer for batch processing, model training, and reinforcement learning.
• Design and maintain K8s-based inference servers (e.g., Triton, TorchServe, or vLLM) optimized for GPU memory management and high throughput.
• Navigate the trade-offs between different GPU instances (A100s, H100s, T4s), optimizing for cost, availability, and performance.
• Solve high-leverage problems that affect multiple pods (e.g., Entry, Session, Presentation), establishing reusable patterns for CI/CD, model versioning, and canary deployments.
• Define and enforce SLIs/SLOs for the platform, ensuring that infrastructure failures never interrupt the user-facing personalization experience.
• Act as a technical mentor to senior engineers across the ML Platform and Applied ML pods, raising the bar for system design and operational rigor.

Qualifications

Basic Qualifications

• 6-8+ years of experience in ML Infrastructure, Platform Engineering, or high-scale Backend Engineering.
• Extensive experience with Kubernetes (K8s) and serving frameworks for large-scale ML models.
• Strong knowledge of GPU architecture, CUDA, and optimizing ML workloads for hardware acceleration.
• Proven track record of owning the technical direction for a major domain and driving impact across multiple teams.

Preferred Qualifications

• Experience with Infra-as-Code (Terraform/Pulumi) and building automated MLOps pipelines.
• Deep expertise with Ray (AnyScale) or similar distributed compute frameworks.
• Familiarity with ML observability tools (Prometheus, Grafana, Weights & Biases, or MLFlow).
• Experience managing multi-cloud or hybrid-cloud ML environments.
• Deep knowledge of Python and C++ for performance-critical systems.

What Success Looks Like

In your first 6–12 months, you will:

• Unify the Compute Layer: Successfully transition the majority of AMLG training workloads to a governed AnyScale/Ray environment.
• Optimize Inference ROI: Measurably improve GPU utilization and reduce inference costs through better auto-scaling and server optimization.
• Establish Durable Standards: Author the "Gold Standard" for ML deployments that is adopted by at least three other pods in the organization.
• Reduce Systemic Risk: Implement a self-healing infrastructure layer that significantly reduces manual intervention for cluster-related failures.

Benefits

• Medical, dental, and vision coverage.
• 401(k) plan with company match.
• Life insurance and disability benefits.
• Tuition assistance program.
• Generous paid time off (PTO) or, if applicable, as dictated by the appropriate Collective Bargaining Agreement.
• Bonus eligibility.
• Attractive compensation and comprehensive benefits packages.
• Opportunities for both on-site and virtual engagement events.
• Unique opportunities to build community and make meaningful connections.

Pay

Hiring salary range: $157,000.00 - $235,000.00. The range applies to New York, California, Colorado, Washington state, and most other geographies. Starting pay depends on geographic location, market demands, experience, training, and education.