Member of Technical Staff - Inference

What you’ll do

• Modify and extend state-of-the-art inference engines like vLLM and SGLang.
• Understand every microsecond of GPU time during a forward pass; be able to explain every kernel launch on an NSys profile.
• Design and implement exotic parallelism schemes to work with 'interesting' hardware topologies.
• Write custom GPU kernels to excel in specific regimes, such as cascade attention.

What we’re looking for

• A strong understanding of LLM mechanics (KV cache, mixture-of-experts, prefill vs. decode phases).
• An interest in MLSys research (speculative decoding, sparse attention).
• Familiarity with modern, tile-based GPU programming (Triton, CUTLASS, ThunderKittens), or interest in learning these.