Senior Software Engineer, SRE and Production Engineering - DGX Cloud
Thomas To · Santa Clara, CA · 6 days ago
EngineeringFull-time
Responsibilities
- Build automation for bare-metal provisioning, hardware validation, firmware and software upgrades, repair, and cluster lifecycle management.
- Develop tools that interact with BMC and Redfish interfaces to monitor hardware health, manage server state, and assist recovery workflows.
- Handle and advance NVIDIA NVL72 systems and BlueField-3 or later DPUs throughout cloud partner and on-premises environments.
- Diagnose failures across servers, DPUs, GPU systems, CPU systems, networking, Linux, and Kubernetes; turn recurring issues into automated detection and repair.
- Define validation and handoff criteria so new capacity enters production safely and consistently.
- Take part in on-call duties, incident response, root-cause analysis, and follow-up to implement permanent solutions.
- Work with hardware, networking, platform, data center operations, and partner teams to resolve issues across ownership boundaries.
Requirements
- 8+ years building software for or operating production infrastructure, including substantial hands-on bare-metal experience.
- Strong Go or Python skills, with a record of delivering production automation and services.
- Direct experience with BMC and Redfish in server provisioning, health inspection, power management, or fault diagnosis.
- Practical experience working directly with NVIDIA GPU hardware, such as NVL72 systems, and BlueField-3 or newer DPUs.
- Experience with Linux, firmware and driver management, network boot, and the server lifecycle from initial provisioning through repair.
- Experience managing production reliability through on-call duties, incident response, observability, and durable solutions.
- Ability to debug failures across hardware, host operating systems, networking, and distributed services.
- Clear communication and demonstrated ownership of problems that span multiple teams.
- BS/MS in Computer Science or equivalent experience in a practical setting.
Preferred Qualifications
- Experience operating BlueField DPUs in DPU mode, including host-to-DPU connectivity and lifecycle debugging, with DPU or equivalent experience considered.
- Background with NVLink, InfiniBand, Spectrum-X, or GPU cluster performance validation.
- Experience building safe, repeatable workflows for rack-scale bringup, firmware upgrades, hardware replacement, and customer handoff.
- Experience with Kubernetes, GitOps, Argo CD, SLOs, and fleet-wide automation.
Pay
The base salary range is 184,000 USD - 287,500 USD.
Benefits
You will also be eligible for equity and benefits.