AI Accelerator Software Principal Engineer – Runtime Library
About The Role
Build and evolve an AI Runtime Library for Ampere accelerators that supports execution, scheduling, and lifecycle management of deep learning workloads across multiple model types and popular frameworks.
Own end-to-end acceleration paths, going deep into the full SW/HW stack—including:
- Inference serving and integration layers
- Compiler/runtime interfaces and graph/IR execution flows
- Runtime library architecture (APIs, memory management, operators, execution engines)
- Communication mechanisms and device/host orchestration
What You’ll Achieve
Drive HW/SW co-design and optimization to improve:
- Throughput (tokens/requests per second)
- Lateny (kernel execution and scheduling efficiency)
- Memory efficiency (buffering, paging, reuse, caching)
- Overall compute utilization and scaling behavior
- BS Computer Science, Computer Engineering, Electrical Engineering, or Software Engineering or related technical field & 8 years of related experience; or MS degree & 6 years; or PhD & 3 years
- Proven experience developing user-mode drivers and/or runtime libraries for GPUs or deep learning accelerators in Linux or RTOS environments.
- Strong expertise in C/C++ and systems-level programming (memory, threading, synchronization, performance profiling).
- Demonstrated background in AI framework enablement, with hands-on experience in one or more of:
- PyTorch (operator/runtime integration, graph execution, correctness/performance work)
- llama.cpp (inference/runtime execution patterns)
- ONNX (graph handling, interoperability, execution engines)
- The full base pay range for this role is between $182,000 and $273,000, except in the San Francisco Bay Area where the range is between $195,000 and $292,000.
- Our benefits include health, wellness, and financial programs that support employees through every stage of life.
- Benefit Highlights Include Premium medical insurance, dental insurance, vision insurance, as well as income protection and a 401K retirement plan, so that you can feel secure in your health and financial future.
- Unlimited Flextime and 10+ paid holidays so that you can embrace a healthy work-life balance.
- A variety of healthy snacks, energizing espresso, and refreshing drinks to keep you fueled and focused throughout the day.
Contribute to AI co-processor/accelerator software enablement
Partner closely with hardware and systems teams to ensure runtime and kernel strategies match accelerator capabilities and constraints.
Collaborate cross-functionally
To integrate runtime components into Ampere platform stacks, ensuring robust deployment on target environments and consistent performance in production-like workloads.
About You
Strong performance engineering skills, including profiling/diagnostics and optimization of execution pipelines, data movement, and compute kernels.
Ability to operate effectively in a collaborative environment—owning complex components while partnering with compilers, hardware, and platform teams.