Engineering Manager, Deep Learning Inference
What You'll Be Doing
Lead, mentor, and scale a high-performing engineering team focused on deep learning inference and GPU-accelerated software.
Guide the strategy, roadmap, and execution of NVIDIA's OSS inference frameworks engineering.
Partner with internal compiler, libraries, and research teams to deliver end-to-end optimized inference pipelines across NVIDIA accelerators.
Oversee performance tuning, profiling, and optimization of large-scale models for LLM, multimodal, and generative AI applications.
Guide engineers in adopting best practices for CUDA, Triton, CUTLASS, and multi-GPU communications (NIXL, NCCL, NVSHMEM).
Represent the team in roadmap and planning discussions, ensuring alignment with NVIDIA’s broader AI and software strategies.
Foster a culture of technical excellence, open collaboration, and continuous innovation.
What We Need To See
- MS, PhD, or equivalent experience in Computer Science, Electrical/Computer Engineering, or a related field.
- 6+ overall years of software development experience, including 3+ years in technical leadership or engineering management.
- Strong background in C/C++ software design and development; proficiency in Python is a plus.
- Hands-on experience with GPU programming (CUDA, Triton, CUTLASS) and performance optimization.
- Proven record of deploying or optimizing deep learning models in production environments.
- Experience leading teams using Agile or collaborative software development practices.
Ways To Stand Out From The Crowd
- Significant open-source contributions to deep learning or inference frameworks such as PyTorch, vLLM, SGLang, Triton, or TensorRT-LLM.
- Deep understanding of multi-GPU communications (NIXL, NCCL, NVSHMEM) and distributed inference architectures.
- Expertise in performance modeling, profiling, and system-level optimization across CPU and GPU platforms.
- Proven ability to mentor engineers, guide architectural decisions, and deliver complex projects with measurable impact.
- Publications, patents, or talks on LLM serving, model optimization, or GPU performance engineering.
Pay & Benefits
With highly competitive salaries and a comprehensive benefits package, NVIDIA is widely considered to be one of the technology world's most desirable employers. The base salary range is 224,000 USD - 356,500 USD for Level 3, and 272,000 USD - 431,250 USD for Level 4. You will also be eligible for equity and benefits.