Software Engineer - Model Performance

Baseten powers mission-critical inference for dynamic AI companies by combining applied AI research, flexible infrastructure, and developer tooling. The company is seeking a Software Engineer focused on ML performance to optimize and productionize techniques for large language model inference, investigate performance issues in ML libraries and infrastructure, and own projects from idea to production. Responsibilities Implement, refine, and productionize cutting-edge techniques (quantization, speculative decoding, KV cache reuse, chunked prefill and LoRA) for ML model inference and infrastructureDeep dive into underlying codebases of TensorRT, PyTorch, TensorRT-LLM, vLLM, sglang, CUDA, and other libraries to debug ML performance issuesApply and scale optimization techniques across a wide range of ML models, particularly large language modelsCollaborate with a diverse team to design and implement innovative solutionsOwn projects from idea to production