AI Accelerator Software Principal Engineer- Framework Integration

About the role

Ampere is a semiconductor design company for a new era, leading the future of computing with an innovative approach to CPU design focused on high-performance, energy efficient AI compute. As a pioneer in the new frontier of energy efficient high-performance computing, Ampere is part of the Softbank Group of companies driving sustainable computing for AI, Cloud, and edge applications. Join us at Ampere and work alongside a passionate and growing team - we’d love to have you apply!

Responsibilities

• Framework integration for accelerator backends
• Integrate and optimize deep learning frameworks—such as PyTorch, ONNX, and llama.cpp—for the Ampere deep learning accelerator backend, enabling efficient and correct execution across a wide set of model types.
• End-to-end deep learning performance acceleration
• Go deep into the full software/hardware execution stack, including: inference serving and orchestration framework integration layers, compiler and graph/runtime support, runtime libraries and user-mode execution paths, compute kernel development, profiling, benchmarking, and performance tuning.
• Model enablement with quality and speed
• Improve both performance and accuracy for models using popular frameworks, and ensure compatibility with serving ecosystems such as vLLM and SGLang—helping deliver production-ready inference behavior.
• Hardware/software co-design and optimization
• Partner with hardware and platform teams to co-optimize AI execution for better outcomes: increased throughput, reduced latency, improved scalability, better resource utilization (compute/memory/IO), higher sustained performance under realistic workloads.
• Build state-of-the-art AI software components
• Contribute to the development of software and hardware AI co-processors/accelerators, delivering reusable libraries, optimized execution paths, and robust integration with existing tooling.
• Cross-functional collaboration
• Work closely with cross-functional teams (compiler/runtime, kernels, platform, and product engineering) to integrate AI capabilities into Ampere’s cloud-native processor platforms and accelerators.

Requirements

• Education & experience: BS Computer Science, Computer Engineering, Electrical Engineering, or Software Engineering or related technical field & 8 years of related experience; or MS degree & 6 years; or PhD & 3 years
• Core framework experience: Strong experience building with or integrating AI frameworks such as PyTorch, llama.cpp, and ONNX.
• Linux + accelerator/runtime expertise (preferred): Experience with developing user-mode drivers, runtime libraries, or low-level integration for GPUs or deep learning accelerators in Linux is a plus.
• Strong systems programming & performance skills: Expert in Python and C/C++
• Strong background in performance profiling and tuning (latency/throughput, memory behavior, kernel efficiency)
• Deep ML understanding: Solid understanding of AI/ML concepts including neural networks and data processing frameworks. Experience with modern deep model architectures such as Transformers and Diffusion models is preferred.
• Modern AI tooling fluency (preferred): Fluent with modern AI programming tools such as Codex or Claude Code, and comfortable accelerating development workflows.

Qualifications

• BS Computer Science, Computer Engineering, Electrical Engineering, or Software Engineering or related technical field & 8 years of related experience; or MS degree & 6 years; or PhD & 3 years

Skills

• Strong experience building with or integrating AI frameworks such as PyTorch, llama.cpp, and ONNX.
• Experience with developing user-mode drivers, runtime libraries, or low-level integration for GPUs or deep learning accelerators in Linux is a plus.
• Expert in Python and C/C++
• Strong background in performance profiling and tuning (latency/throughput, memory behavior, kernel efficiency)
• Solid understanding of AI/ML concepts including neural networks and data processing frameworks. Experience with modern deep model architectures such as Transformers and Diffusion models is preferred.
• Fluent with modern AI programming tools such as Codex or Claude Code, and comfortable accelerating development workflows.

Benefits

• Premium medical insurance
• Dental insurance
• Vision insurance
• Income protection
• 401K retirement plan