DirectJobHub
Jobs · Engineering · Washington

Principal Engineer, Cluster Orchestration

CoreWeave · Bellevue, WA · 6 days ago
Engineering$206k–$303k/yrFull-time

About the role

The AI infrastructure behind CoreWeave's largest GPU clusters determines workload placement, resource sharing, and system performance under constant pressure. As a Principal Engineer in AI Infrastructure, you will lead the design and evolution of orchestration systems, including Kubernetes, Slurm, SUNK, and related systems.

Responsibilities

  • Define long-term architecture for CoreWeave’s orchestration platforms across Kubernetes, Slurm, SUNK, Kueue, and related systems.

  • Make design decisions that balance performance, reliability, cost, and operational complexity.

  • Lead the evolution of Kubernetes-native control planes, including SUNK and custom operators.

  • Design systems that support workload admission, validation, and rollout, including model onboarding flows.

  • Identify and remove scaling limits across schedulers, control planes, registries, networking, and storage.

  • Set standards for reliability, observability, and operational readiness across orchestration services.

  • Define Service Level Objectives (SLOs), alerting, and incident response practices for platform-critical systems.

  • Ensure systems behave predictably during failures, peak load, and rapid growth.

  • Write and review production code for Kubernetes controllers, schedulers, admission logic, and internal tooling.

  • Measure and improve scheduling latency, container startup time, image distribution, and cold-start performance.

  • Lead architecture and design reviews across infrastructure teams.

  • Mentor senior and staff engineers and help grow technical leaders.

  • Influence platform, infrastructure, security, and product teams through clear technical judgment.

  • Engage with customers and open-source communities on deep technical topics when needed.

Requirements

  • 15+ years of experience building and operating large-scale distributed systems.

  • Deep, practical knowledge of Kubernetes and Slurm internals.

  • Experience running GPU-heavy platforms for AI training, inference, or HPC workloads.

  • Strong background in Go and cloud-native systems development.

  • Proven ability to set technical direction across teams without direct authority.

  • Comfortable making high-impact technical decisions in complex systems.

  • Bachelor’s or Master’s degree in a relevant field, or equivalent experience.

Preferred Qualifications

  • Experience with systems such as Kueue, Kubeflow, Argo Workflows, Ray, Istio, or Knative.

  • Background in ML platform engineering, model onboarding, or lifecycle management.

  • Strong understanding of scheduling strategies, pre-emption, quota enforcement, and elastic scaling.

  • Track record of operating highly reliable systems with clear SLOs and incident processes.

  • Contributions to Kubernetes, ML infrastructure, or related open-source projects.

  • Experience mentoring senior engineers and raising engineering standards.

Similar jobs