Software Development Manager, Neuron Tools, Annapurna Labs

About the role

AWS Neuron is the complete software stack for the AWS Inferentia and Trainium cloud-scale machine learning accelerators and the Trn1 and Inf1 servers that use them. As the Software Development Manager for the Tools Team, you will lead a team of engineers to develop and maintain high-performance monitoring and profiling tools for machine learning applications and AI accelerators.

Responsibilities

• Oversee the design, development, and deployment of the Neuron Profiler and other Neuron Tools.
• Provide deep insights into performance bottlenecks and system behavior to internal and external customers.
• Ensure scalability, reliability, and usability of the Neuron Profiler/Tools toolchain.
• Collaborate with cross-functional teams to ensure that the C++ compiler and runtime generate key information for customers to understand and optimize the performance of our custom hardware.
• Drive innovations that allow the profiler to support multiple frameworks like PyTorch, TensorFlow, and XLA.

Requirements

• 3+ years of engineering team management experience.
• 7+ years of working directly within engineering teams.
• 3+ years of designing or architecting new and existing systems.
• Experience partnering with product or program management teams.
• Experience in C++, Go, and Python.

Qualifications

• 2+ years experience leading teams that in Machine Learning development including building and training large models, working with Pytorch and/or Tensorflow using large distributed fleets of GPU or other accelerated systems.
• Experience with Linux distributions such as Ubuntu or CentOS, kernel development, and tooling such as perf and gdb.
• Experience with performance profiling, tracing, and analysis of AI training/inference applications.
• Experience with large scale, distributed AI training/inference applications, including libfabric, MPI, slurm, and EKS.
• Experience with fleet monitoring, debugging, and reliability.
• Knowledge of AI-powered optimization suggestions for profiling would be an advantage for this position.