Sr. Engineering Manager, Inference
CoreWeave · Bellevue, WA · 1 mo ago
On-siteEngineering$188k–$303k/yrFull-time
About the role
CoreWeave is looking for a Sr. Engineering Manager to lead the core team for our next-generation Inference Platform.
What You'll Do
As a Sr. Engineering Manager for the Inference team, you will lead a team of senior and staff engineers building and operating CoreWeave's Kubernetes-native inference platform.
What We're Looking For
- 8+ years of experience building and operating large-scale distributed systems or cloud services
- 3+ years of engineering management experience, including managing senior and staff-level engineers
- Track record owning a production service end to end including availability, latency, capacity, and cost
- Deep expertise in Kubernetes at production scale, including orchestration, scheduling, and service design
- Strong understanding of distributed systems, networked services, and performance under load; able to lead architecture and code review without owning the critical path
- Experience defining and defending SLIs/SLOs and error budgets, and building the operational practice behind them
- Experience with multi-tenant platforms: isolation, quota, noisy-neighbor mitigation, and fair sharing
- Excellent communication and stakeholder management; able to create clarity in ambiguous or fast-scaling environment
- Demonstrated track record of growing and promoting engineers, including into senior and staff-level roles
- Experience running performance calibration and making accurate leveling decisions
- Experience managing underperformance directly and fairly, as well as retaining top performers
- Experience building and owning a hiring bar and interview process for senior engineering roles
Platform & Service Expertise
- Experience operating low-latency, high-throughput request-serving systems against strict P95/P99 targets
- Working knowledge of autoscaling, load balancing, admission control, and traffic management at scale
- Experience with capacity planning and demand forecasting for constrained or expensive resources
- Familiarity with GPU-backed workloads and the scheduling and utilization trade-offs they introduce
- Progressive delivery practices including canarying, feature gating, safe rollback for always-on services
Nice to Have
- Experience operating a customer-facing cloud service or developer platform, including API and control-plane design
- Familiarity with inference-serving stacks such as vLLM, Triton, TensorRT-LLM, Ray Serve, or TorchServe
- Experience operating multi-region fleets at a cloud provider or hyperscaler
- Familiarity with observability stacks (Prometheus, Grafana, OpenTelemetry)
- Experience scaling teams, tooling, and processes in a high-growth environment
About You
- You love to build frictionless products for developers
- You're curious about AI and MLOps tooling
- You're an expert in building inference systems that scale for production workloads
Benefits
- Medical, dental, and vision insurance - 100% paid for by CoreWeave
- Company-paid Life Insurance
- Voluntary supplemental life insurance
- Short and long-term disability insurance
- Flexible Spending Account
- Health Savings Account
- Tuition Reimbursement
- Ability to Participate in Employee Stock Purchase Program (ESPP)
- Mental Wellness Benefits through Spring Health
- Family-Forming support provided by Carrot
- Paid Parental Leave
- Flexible, full-service childcare support with Kinside
- 401(k) with a generous employer match
- Flexible PTO
- Catered lunch each day in our office and data center locations
- A casual work environment
- A work culture focused on innovative disruption
Pay
The base salary range for this role is $188,000 to $303,000. The starting salary will be determined based on job-related knowledge, skills, experience, and market location. In addition to base salary, our total rewards package includes a discretionary bonus, equity awards, and a comprehensive benefits program (all based on eligibility).