ML Engineer (Senior/Staff)

Responsibilities

• Own architecture for inference serving and GPU scheduling — Kubernetes operators, autoscaling, and dynamic capacity across vLLM/SGLang deployments on cloud and customer-managed infrastructure.
• Design and calibrate eval systems for model, prompt, and agent changes, including golden datasets, LLM-as-judge pipelines, and regression gates wired into CI.
• Apply physics-informed ML and enterprise AI expertise to the hardest client and platform problems, drawing on the team's research depth.
• Set technical standards for ML infrastructure and evaluation practice across the org, and mentor engineers working in this space.
• Partner closely with the inference platform, gateway, and evals-focused engineers to keep architecture coherent as the platform grows.

Qualifications

• 3+ years of experience with ML infrastructure and inference serving — vLLM, SGLang, TensorRT-LLM, or comparable systems — at production scale.
• Strong background in evaluation and reliability engineering for ML/LLM systems, or the seniority to build this practice from scratch.
• Solid Kubernetes experience, ideally including GPU-specific scheduling constraints (node pools, autoscaling under GPU bottlenecks).
• A track record of technical leadership at a staff or senior level — setting direction, not just executing tickets.

Bonus Qualifications

• Advanced ML/AI frameworks and techniques (e.g., PyTorch Lightning, JAX, HuggingFace, ONNX optimizations)
• Lower-level or performance-focused languages for ML acceleration (e.g., C++, Rust, CUDA)
• Large-scale data and distributed training paradigms (e.g., Spark, Ray, Horovod, Dask)
• Advanced data infrastructure (e.g., vector/graph databases, feature stores, data lakes)

Pay

Compensation Range: $140K - $220K

Schedule

Full-time