Sr. ML Kernel Performance Engineer, AWS Neuron, Annapurna Labs

About the role

The Acceleration Kernel Library team is at the forefront of maximizing performance for AWS's custom ML accelerators. Working at the hardware-software boundary, our engineers craft high-performance kernels for ML functions, ensuring every FLOP counts in delivering optimal performance for our customers' demanding workloads.

Responsibilities

• Design and implement high-performance compute kernels for ML operations, leveraging the Neuron architecture and programming models
• Analyze and optimize kernel-level performance across multiple generations of Neuron hardware
• Conduct detailed performance analysis using profiling tools to identify and resolve bottlenecks
• Implement compiler optimizations such as fusion, sharding, tiling, and scheduling
• Work directly with customers to enable and optimize their ML models on AWS accelerators
• Collaborate across teams to develop innovative kernel optimization techniques

Requirements

• 5+ years of non-internship professional software development experience
• 5+ years of programming with at least one software programming language experience
• 5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience
• 5+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
• Experience as a mentor, tech lead or leading an engineering team

Qualifications

• Bachelor's degree in computer science or equivalent
• 6+ years of full software development experience
• Expertise in accelerator architectures for ML or HPC such as GPUs, CPUs, FPGAs, or custom architectures
• Experience with GPU kernel optimization and GPGPU computing such as CUDA, NKI, Triton, OpenCL, SYCL, or ROCm
• Demonstrated experience with NVIDIA PTX and/or AMD GPU ISA
• Proficiency in low-level performance optimization for GPUs
• Knowledge of ML frameworks (PyTorch, TensorFlow) and their GPU backends
• Experience with parallel programming and optimization techniques
• Understanding of GPU memory hierarchies and optimization strategies