Senior Software Engineer, Machine Learning Inference

About the Role

NVIDIA is seeking talented and motivated engineers to join the TensorRT team in developing the industry-leading deep learning inference software for NVIDIA AI accelerators. As a Senior Software Engineer in the TensorRT team, you will be responsible for designing and implementing inference software optimizations to power AI applications on NVIDIA GPUs.

Responsibilities

• Design, develop and optimize NVIDIA TensorRT and TensorRT-LLM to supercharge inference applications for datacenter, workstations, and PCs.
• Develop software in C++, Python, and CUDA for seamless and efficient deployment of state-of-the-art LLMs and Generative AI models.
• Collaborate with deep learning experts and GPU architects throughout the company to influence Hardware and Software design for inference.

Qualifications

• BS, MS, PhD or equivalent experience in Computer Science, Computer Engineering or a related field.
• 4+ years of software development experience on a large codebase or project.
• Strong proficiency in C++ (required), Rust or Python programming languages.
• Experience in developing Deep Learning Frameworks, Compilers, or System Software.
• Excellent problem-solving skills and passion to learn and work effectively in a fast-paced, collaborative environment.
• Strong communication skills and the ability to articulate complex technical concepts.

Preferred Skills

• Experience in developing inference backends and compilers for GPUs.
• Knowledge of Machine Learning techniques and GPU programming with CUDA or OpenCL.
• Background in working with LLM inference frameworks like TensorRT-LLM, vLLM, SGLang.
• Experience working with deep learning frameworks like TensorRT, PyTorch, JAX.
• Knowledge of close-to-metal performance analysis, optimization techniques, and tools.

Pay

Base salary range is 152,000 USD - 241,500 USD for Level 3, and 184,000 USD - 287,500 USD for Level 4. You will also be eligible for equity and benefits.

Schedule

Full time