Manager, Software Engineering - Production AI Inference

What You'll Be Doing

Lead the team responsible for shipping production-ready LLM NIMs, including planning, new model onboarding, validated serving recipes, release readiness, and post-release follow-through.

Build a predictable operating model for the team through roadmap planning, a weekly execution rhythm, launch checklists, clear ownership boundaries, collaborator communication, and issue management.

Own project execution by anticipating schedule, staffing, and dependency risks. Adapt plans under pressure and collaborate with peer managers to dynamically prioritize engineering timelines to remain agile in the fast-paced AI industry.

Drive continuous improvement in production workflows through RCCA and partner feedback, removing unnecessary and redundant work while keeping the team passionate about production outcomes.

Build and maintain a world-class AI inference engineering team by building an innovative culture, setting clear expectations, maintaining active feedback loops, and mentoring engineers and emerging leaders.

What We Need To See

• 10+ overall years building production software, including 3+ years of managing software engineering teams.
• Experience delivering production software with strong quality, reliability, and release expectations.
• Experience driving process improvements, and improving operational efficiency.
• Excellent communication and collaborator management; ability to influence executive leadership across product, research, security, and operations.
• Deep understanding of AI/ML fundamentals, innovative model architectures, inference engine/kernel, performance optimization strategies, accelerated computing, large-scale distributed systems, and security hardening.
• A degree in Computer Science, Computer Engineering, or a related field (BS or MS) or equivalent experience.

Ways To Stand Out From The Crowd

• Built and managed globally distributed organizations; established durable engineering processes that significantly improved quality and velocity across multiple teams.
• Recognized industry leader with contributions to open-source ecosystems (i.e vLLM, SGLang, TensorRTLLM, Dynamo, Triton, PyTorch), technical publications, or talks in containers, Kubernetes, GPU, or inference communities.
• Drove measurable performance improvements for large-scale LLM inference systems, including latency, throughput, GPU utilization, cost efficiency, and performance regression prevention across production releases.
• Hands-on experience with core GPU technologies such as CUDA, cuDNN, CUTLASS, cuBLAS, NCCL, NIXL, NVLink, and GPUDirect RDMA.
• Hands-on experience delivering enterprise or government-ready AI software, including FedRAMP, air-gapped deployments, regulated environments, security hardening, compliance evidence, and production support expectations.