Manager, Large Language Model Inference

About the role

The TensorRT inference platform is the backbone of modern AI, delivering the industry's fastest and most efficient deployment of cutting-edge deep learning models on every NVIDIA GPU. With demand for AI exploding, particularly in the realm of large language models (LLMs) and vision language models (VLMs), we are significantly expanding our team.

Responsibilities

• Lead and grow a team responsible for specialized kernel development, runtime optimizations, and frameworks for LLM inference.
• Drive the design, development, and delivery of production inference software, targeting NVIDIA's next-generation enterprise and edge hardware platforms.
• Integrate cutting-edge technologies developed at NVIDIA and offer an intuitive developer experience for LLM deployment.
• Lead software development execution, with responsibility for project planning, milestone delivery, and cross-functional coordination.

Requirements

• MS, PhD, or equivalent experience in Computer Science, Computer Engineering, AI, or a related technical field.
• 7+ overall years of overall software engineering experience, including 3+ years of technical leadership experience.
• Proven ability to lead and scale high-performing engineering teams, especially across distributed and cross-functional groups.
• Strong background in C++ or Python, with expertise in software design and delivering production-quality software libraries.
• Demonstrated expertise in large language models (LLM) and/or vision language models (VLM).

Qualifications

• Deep understanding of GPU architecture, CUDA programming, and system-level performance tuning.
• Background in LLM inference or working with frameworks such as TensorRT-LLM, vLLM, or SGLang.
• Passion for building scalable, user-friendly APIs and enabling developers in the AI ecosystem.
• A proven track record of growing and managing a team that encourages idea sharing, empowers team members, and provides opportunities for professional growth.

Skills

• Experience with large language models (LLM) and/or vision language models (VLM).
• Expertise in software design and delivering production-quality software libraries.
• Strong background in C++ or Python.
• Understanding of GPU architecture, CUDA programming, and system-level performance tuning.
• Experience with LLM inference or working with frameworks such as TensorRT-LLM, vLLM, or SGLang.

Benefits

• NVIDIA is widely considered to be one of the technology world’s most desirable employers.
• We have some of the most forward-thinking and hardworking people in the world working with us.
• Due to outstanding growth, our best-in-class teams are rapidly growing.
• We are committed to fostering a diverse work environment and proud to be an equal opportunity employer.

Pay

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is $184,000 - $287,500 for Level 2, and $224,000 - $356,500 for Level 3.

Schedule

Applications for this job will be accepted at least until March 26, 2026.

Contact

This posting is for an existing vacancy. NVIDIA uses AI tools in its recruiting processes. NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.