Senior Deep Learning Software Engineer, Inference

About the role

NVIDIA seeks a Senior Software Engineer specializing in Deep Learning Inference to join our growing team. As a key contributor, you will help design, build, and optimize GPU‑accelerated software that powers today’s most sophisticated AI applications. Our team develops and maintains high‑performance open‑source frameworks at the forefront of efficient large‑scale model serving and inference, and you will play a central role in improving these platforms, facilitating smooth deployment and serving of groundbreaking language models. You’ll work closely with the deep learning community to implement the latest algorithms for public release in inference frameworks, focusing on identifying and driving performance improvements for state‑of‑the‑art LLM and Generative AI models across NVIDIA accelerators—from datacenter GPUs to edge SoCs—using open‑source tools and plugins such as CUTLASS, OAI Triton, NCCL, and CUDA kernels.

Responsibilities

• Performance optimization, analysis, and tuning of deep learning models in various domains such as LLM, multimodal, and generative AI.
• Scaling performance of deep learning models across different architectures and types of NVIDIA accelerators.
• Contributing features and code to NVIDIA’s inference libraries, including vLLM, SGLang, FlashInfer, and other LLM software solutions.
• Working with cross‑collaborative teams across frameworks, NVIDIA libraries, and inference optimization to devise innovative solutions.
• Implementing and optimizing model serving pipelines using open‑source tools and plugins such as CUTLASS, OAI Triton, NCCL, and CUDA kernels.

Requirements

• Master’s or PhD degree (or equivalent experience) in a relevant field such as Computer Engineering, Computer Science, EECS, or AI.
• 5+ years of relevant software development experience.
• Excellent C/C++ programming and software design skills.
• Familiarity with Agile methodologies; Python experience is a plus.
• Prior experience with training, deploying, or optimizing the inference of deep learning models in production (a plus).
• Background in performance modeling, profiling, debugging, code optimization, or architectural knowledge of CPU and GPU (a plus).
• GPU programming experience with CUDA, OAI Triton, or CUTLASS (a plus).

Skills

• Proficiency in C/C++ and Python.
• Experience with GPU programming frameworks (CUDA, OAI Triton, CUTLASS).
• Knowledge of performance analysis, profiling, and debugging tools.
• Familiarity with multi‑GPU communication libraries (NCCL, NVSHMEM).
• Experience contributing to open‑source deep learning projects (e.g., PyTorch, vLLM, SGLang).
• Strong software design and Agile development practices.

Benefits

NVIDIA offers highly competitive salaries and a comprehensive benefits package, including equity eligibility, and is widely regarded as one of the technology world’s most desirable employers.

Pay

The base salary range is $152,000 USD – $241,500 USD for Level 3 and $184,000 USD – $287,500 USD for Level 4, with eligibility for equity and additional benefits.