Senior Software Development Engineer, AI/ML, AWS Neuron, Model Inference

About the role

The Inference Enablement and Acceleration team is at the forefront of running a wide range of models and supporting novel architecture alongside maximizing their performance for AWS's custom ML accelerators. This role offers a unique opportunity to work at the intersection of machine learning, high-performance computing, and distributed architectures, where you'll help shape the future of AI acceleration technology.

Responsibilities

• Architect and implement business critical features, and mentor a brilliant team of experienced engineers.

• Design, develop, and optimize machine learning models and frameworks for deployment on custom ML hardware accelerators.

• Participate in all stages of the ML system development lifecycle including distributed computing based architecture design, implementation, performance profiling, hardware-specific optimizations, testing and production deployment.

• Build infrastructure to systematically analyze and onboard multiple models with diverse architecture.

• Design and implement high-performance kernels and features for ML operations, leveraging the Neuron architecture and programming models.

• Analyze and optimize system-level performance across multiple generations of Neuron hardware.

• Conduct detailed performance analysis using profiling tools to identify and resolve bottlenecks.

• Implement optimizations such as fusion, sharding, tiling, and scheduling.

• Conduct comprehensive testing, including unit and end-to-end model testing with continuous deployment and releases through pipelines.

• Work directly with customers to enable and optimize their ML models on AWS accelerators.

• Collaborate across teams to develop innovative optimization techniques.

Qualifications

• Bachelor's degree in computer science or equivalent

• 5+ years of non-internship professional software development experience

• 5+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience

• Fundamentals of Machine learning and LLMs, their architecture, training and inference lifecycles along with work experience on some optimizations for improving the model execution.

• Software development experience in C++, Python (experience in at least one language is required).

• Strong understanding of system performance, memory management, and parallel computing principles.

• Proficiency in debugging, profiling, and implementing best software engineering practices in large-scale systems.