Jobs · Engineering · California

Senior Site Reliability Engineer, AIOPs

NVIDIA · Santa Clara, CA · 1 wk ago
EngineeringFull-time

About the role

Join our team of innovative engineers who are building an AI Data Center AIOps platform that turns raw, high-volume telemetry into reliable, job-centric insights and automation for GPU fleets. We’re hiring a DevOps Engineer to operate the platform itself (not the compute cluster): uptime, performance, data integrity, and safe change management.

Responsibilities

  • Continuously monitor platform health via dashboards/logs/metrics, automate recurring checks, and keep reliability + resource efficiency on track.
  • Own Kubernetes deployments end-to-end (runbooks, canary checks, post-deploy validation), and lead rollbacks/remediations when needed.
  • Lead first-level incident triage: collect diagnostics, identify likely root causes, and hand off clear, actionable findings to engineering.
  • Build and maintain runbooks/SOPs/checklists, pushing continuous improvement through automation.
  • Manage deployment infrastructure and packaging (Helm + Terraform/IaC) to keep environments scalable, consistent, and reproducible.
  • Contribute in adjacent functional areas to grow and help your team members!

Requirements

  • BS/MS in CS/CE (or equivalent experience) and 5+ years operating production distributed systems as SRE/DevOps/Platform Ops.
  • Proven ownership of reliability for an observability/AIOps platform: SLOs/SLIs, on-call, addressing incidents, and follow-up evaluations that drive measurable improvements.
  • Deep Kubernetes + containers experience (deploying, debugging, scaling) for telemetry-heavy microservices—ingestion, processing, storage, APIs, and UI.
  • Automation-first approach: solid scripting (Python/Bash), CI/CD, and infrastructure-as-code (Terraform + Helm) to deliver safe rollouts (canaries/rollbacks), reproducible environments, and minimal toil.
  • Clear communicator who writes excellent runbooks/docs and can translate ambiguous requirements into concrete operational practices and dependable customer-facing reliability.

Qualifications

  • Strong Linux + networking fundamentals, distributed systems instincts, and hands-on ops for Kubernetes/services/streaming stacks are ideal; bonus for experience with observability platforms at scale.
  • Experience building safe automation that operators trust: canary releases, automated rollback criteria, “monitoring for the monitoring” (lag/drop/error budgets), and replay/backfill pipelines with correctness checks.
  • Proven programming experience building automation tools or services — ideally in Python, or similar languages — to simplify operations and scale recurring processes.
  • Proven experience running large-scale production deployments and multiple Kubernetes environments or clusters across teams or customers, coordinating changes and rollouts with minimal disruption with hands-on experience with observability tools — you know your way around dashboards, metrics, logs, and traces using platforms like Prometheus, Grafana, or similar.

Skills

  • Strong communication skills to write excellent runbooks/docs and translate ambiguous requirements into concrete operational practices and dependable customer-facing reliability.

Benefits

  • Competitive salaries and a generous benefits package.
  • Opportunity to work with NVIDIA's cutting-edge technology and innovative team.
  • Exclusive engineering teams rapidly growing due to unprecedented growth.

Pay

Base salary range: $148,000 - $235,750 for Level 3, and $176,000 - $276,000 for Level 4.

Schedule

NVIDIA offers flexible scheduling options to support a healthy work-life balance.

Similar jobs