Software Development Engineer II, SageMaker AI Inference

About the role

This position focuses on improving LLM inference latency and throughput through kernel tuning, performance profiling, and production-ready inference systems for SageMaker AI.

Responsibilities

• Design and implement kernel-level optimizations and inference pipeline improvements that reduce latency and increase throughput for LLM serving workloads
• Profile, benchmark, and tune model inference performance across hardware and software stacks, identifying bottlenecks and delivering measurable gains
• Write well-organized, maintainable code and participate actively in code reviews, providing constructive feedback to peers and mentoring newer engineers on the team
• Collaborate with applied scientists, ML engineers, and infrastructure teams to translate model requirements into production-ready inference solutions
• Evaluate emerging tools, frameworks, and generative AI techniques for inference optimization, recommending adoption where they improve efficiency or quality