Manager, Large Language Model Inference

About the role

The TensorRT inference platform is the backbone of modern AI, delivering the industry's fastest and most efficient deployment of cutting-edge deep learning models on every NVIDIA GPU. With demand for AI exploding, particularly in the realm of large language models (LLMs) and vision language models (VLMs), we are significantly expanding our team.

Responsibilities

• Lead and grow a team responsible for specialized kernel development, runtime optimizations, and frameworks for LLM inference.
• Drive the design, development, and delivery of production inference software, targeting NVIDIA's next-generation enterprise and edge hardware platforms.
• Integrate cutting-edge technologies developed at NVIDIA and offer an intuitive developer experience for LLM deployment.
• Lead software development execution, with responsibility for project planning, milestone delivery, and cross-functional coordination.

Requirements

• MS, PhD, or equivalent experience in Computer Science, Computer Engineering, AI, or a related technical field.
• 7+ overall years of overall software engineering experience, including 3+ years of technical leadership experience.
• Proven ability to lead and scale high-performing engineering teams, especially across distributed and cross-functional groups.
• Strong background in C++ or Python, with expertise in software design and delivering production-quality software libraries.
• Demonstrated expertise in large language models (LLM) and/or vision language models (VLM).

Qualifications

• Deep understanding of GPU architecture, CUDA programming, and system-level performance tuning.
• Background in LLM inference or working with frameworks such as TensorRT-LLM, vLLM, or SGLang.
• Passion for building scalable, user-friendly APIs and enabling developers in the AI ecosystem.
• A proven track record of growing and managing a team that encourages idea sharing, empowers team members, and provides opportunities for professional growth.

Skills

• Excellent communication and interpersonal skills.
• Ability to work effectively in a fast-paced, dynamic environment.
• Experience with version control systems like Git.
• Knowledge of cloud platforms and services.

Benefits

NVIDIA offers a comprehensive benefits package including health insurance, retirement plans, and paid time off. We also provide opportunities for professional development and continuous learning.

Pay

Base salary range: $184,000 - $287,500 for Level 2, and $224,000 - $356,500 for Level 3.

Schedule

Full-time position.