Senior Software Engineer, Quantized Inference

About the role

NVIDIA is seeking a Senior Software Engineer to accelerate the discovery and deployment of efficient inference recipes for Large Language Models (LLMs). The role involves translating recipe specifications into functionally correct, performant code, collaborating with partner inference teams, and optimizing throughput and interactivity on target workloads.

Responsibilities

• Implement quantized and sparse recipes in inference engines (vLLM, TRT-LLM, SGLang)
• Own model export pipelines (ModelOpt, Megatron-LM HuggingFace), ensuring quantized checkpoints serialize correctly for downstream serving
• Build prototypes and benchmarking harnesses to evaluate recipe throughput/interactivity before full optimization
• Develop data analysis tooling and visualizations for numerics debugging
• Improve developer productivity across the team: CI, build systems, training infrastructure, pipeline friction
• Participate in code reviews and incorporate feedback

Requirements

• Proficient in Python; familiarity with C++
• Strong software engineering fundamentals: concise, well-tested code; fluent with AI-assisted tooling
• Experience with ML accelerators with a basic understanding of how certain ML layers affect execution time
• Familiarity with PyTorch internals (custom ops, autograd, export) or equivalent framework
• Experience reading, modifying, or contributing to a large open-source codebase
• MS/PhD in Computer Science or related field, or equivalent experience
• 4+ years in a relevant software engineering role
• Demonstrated ability to move fast with ambiguous requirements, with strong written and verbal communication

Qualifications

• Experience contributing to inference serving frameworks (vLLM, TRT-LLM, SGLang) or Triton kernel development
• Track record of debugging numerical issues across mixed-precision boundaries
• Deep experience with model compression techniques: PTQ, QAT, structured/unstructured sparsity

Skills

• Python programming
• C++ programming
• Software engineering fundamentals
• AI-assisted tooling
• ML accelerators
• PyTorch internals (custom ops, autograd, export)
• Large open-source codebases
• Model compression techniques: PTQ, QAT, structured/unstructured sparsity

Benefits

• Base salary range: $152,000 - $241,500 for Level 3, and $184,000 - $287,500 for Level 4
• Eligibility for equity and benefits

Pay

• Base salary determined based on location, experience, and comparable positions

Schedule

• Full-time position

Benefits

• Comprehensive health insurance
• Retirement savings plan
• Flexible work arrangements
• Professional development opportunities