Senior Inference Engineer, AIConfigurator for Dynamo

About the role

The Senior Inference Engineer will build and evolve AIConfigurator, a system that optimizes deployment configurations for large-scale Language Model (LLM) inference on NVIDIA platforms. This role integrates GPU systems, model serving, performance modeling, and production software engineering.

Responsibilities

• Build and evolve AIConfigurator's core optimization engine for LLM serving, including configuration search, SLA-aware ranking, efficiency and latency estimation, and Pareto frontier analysis.
• Create production-quality Python/Rust APIs, CLIs, SDK surfaces, and web workflows that help users generate strong deployment configurations for NVIDIA GPU clusters.
• Develop configuration generation systems that emit backend-specific artifacts for Dynamo, Kubernetes, TensorRT-LLM, vLLM, and SGLang deployments.
• Collaborate with inference runtime, performance, benchmarking, and product groups to ensure simulated results correspond with actual deployment performance on H100, H200, B200, GB200, and upcoming NVIDIA platforms.
• Improve model, hardware, and backend support by integrating performance databases, profiling data, support matrices, and validation tools.
• Drive software quality through maintainable architecture, schema development, tests, documentation, and automation suitable for open-source and production users.
• Convert intricate inference ideas like prefill/decode disaggregation, tensor parallelism, pipeline parallelism, expert parallelism, batching, and KV cache behavior into dependable software abstractions.

Requirements

• Bachelor’s, Master’s, or PhD in Computer Science, Computer Engineering, Electrical Engineering, Applied Math, or a related field, or equivalent experience.
• 10+ years of relevant software engineering experience.
• Strong Python/Rust engineering skills, including production APIs, CLI tools, packaging, testing, debugging, and maintainable software development.
• Experience with GPU computing, distributed systems, ML infrastructure, or high-performance model serving.
• Understanding of LLM inference concepts such as batching, latency, efficiency, memory constraints, parallelism strategies, and serving SLAs.
• Experience working with data-driven performance analysis, benchmarking, simulation, optimization, or managing resource needs.
• Ability to collaborate across research, runtime, platform, and customer-facing engineering teams.
• Strong written and verbal communication skills, with the ability to explain sophisticated technical tradeoffs clearly.

Qualifications

• Practical experience working directly with TensorRT-LLM, vLLM, SGLang, Triton Inference Server, Dynamo, Kubernetes, or comparable serving platforms.
• Experience improving LLM deployments on NVIDIA GPUs, especially H100, H200, B200, GB200, or multi-node GPU clusters.
• Familiarity with disaggregated serving, prefill/decode separation, KV cache management, NCCL/NIXL/NVSHMEM communication, or expert-parallel MoE inference.
• Open-source project experience, technical writing experience, or prior ownership of developer-facing tools.
• Agentic AI solutions to solve sophisticated technical problems.

Benefits

Base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is $184,000 - $287,500 for Level 4, and $224,000 - $356,500 for Level 5. Eligible for equity and benefits.

Pay

Base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is $184,000 - $287,500 for Level 4, and $224,000 - $356,500 for Level 5.

Schedule

Not specified.

Application Instructions

Applications for this job will be accepted at least until June 16, 2026.

Company Information

NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.