Inference Optimization ML Engineer

What You'll Do

• Own inference performance end-to-end — diagnose and improve latency, throughput, and efficiency of large foundation models in production
• Build systematic performance attribution: latency decomposition (compute vs. memory bandwidth vs. I/O), bottleneck identification, and prioritization across model families
• Apply and develop optimization techniques including quantization, pruning, distillation, operator fusion, and model compilation (e.g., TensorRT, torch.compile, XLA)
• Optimize attention mechanisms, KV caching, and memory layouts for large multimodal models (vision, video, language, proprioception)
• Work with kernel-level tooling (e.g., CUDA, Triton) to identify hotspots and implement or tune custom kernels where needed
• Build benchmarking and regression detection infrastructure: latency baselines, throughput curves, and automated detection of performance regressions across model versions
• Collaborate closely with research engineers to translate model innovations into optimized, deployment-ready implementations

What We're Looking For

• 3+ years of experience in inference optimization, ML systems, or a closely related field
• Deep hands-on experience with modern ML stacks (PyTorch required; JAX a plus)
• Strong understanding of compute, memory bandwidth, and I/O bottlenecks in large model inference
• Experience with model optimization techniques: quantization (INT8/FP8/AWQ), distillation, pruning, and compilation
• Familiarity with inference serving frameworks (e.g., Triton, TensorRT, vLLM, TorchServe)
• Exceptional debugging and measurement ability: turn "inference is slow" into clear bottlenecks, experiments, and validated improvements
• High ownership mindset and comfort in a fast-moving environment

Nice To Have (But Not Required)

• GPU kernel or compiler-level experience (CUDA, Triton, graph capture, operator fusion)
• Experience with multimodal or video model inference (variable-length sequences, packing/bucketing)
• Familiarity with edge/cloud hybrid deployment patterns and on-robot inference constraints
• Experience with speculative decoding, continuous batching, or other LLM serving optimizations
• Background in streaming or low-latency systems relevant to real-time robot control