Jobs · Engineering · California

Contract Site Reliability Engineer — AI Accelerator Infrastructure

d-Matrix · Santa Clara, CA · 2 days ago
HybridEngineeringFull-time

About the role

The role is a 6-month contract with potential conversion to full-time. It is a hands-on, high-ownership role where you will build, operate, and automate real infrastructure.

Responsibilities

  • Own reliability and availability of assigned infrastructure domains: colo server fleets, on-premises lab clusters, cloud environments (AWS, Azure, GCP), and customer-facing platform services.
  • Perform hands-on infrastructure work: server provisioning, OS configuration, network setup, storage management, and hardware troubleshooting from bare metal up.
  • Support and operate high-speed interconnect environments — InfiniBand, RoCE, or high-speed Ethernet — in lab and colo settings.
  • Conduct capacity planning and hardware lifecycle management for assigned infrastructure domains.
  • Own IaC and configuration management (Terraform, Ansible) for your infrastructure domains — all provisioning and changes through code, not manual steps.
  • Build automation to eliminate toil: host lifecycle management, fleet health checks, auto-remediation workflows, and self-service tooling for engineering teams.
  • Develop networking automations for cluster interconnects, VLAN management, and lab network configurations.
  • Contribute to shared IaC modules and automation libraries used across the SRE team.
  • Design and maintain monitoring dashboards, alerting, and SLIs (Prometheus/Grafana, DataDog) for your infrastructure domains—ensuring signal quality and actionable alerts.
  • Participate in on-call rotation; triage and resolve incidents from bare metal to application layer, distinguishing infrastructure faults from software or hardware product issues.
  • Produce high-quality RCA reports for P0/P1 incidents with root cause analysis and tracked action items.
  • Detect performance issues, recommend solutions, and implement fixes that permanently improve system reliability.
  • Support and operate platform services used by external customers for hardware and software deployment.
  • Ensure QoS and uptime commitments for customer-facing environments; escalate reliability risks proactively.
  • Document platform configurations, access procedures, and operational runbooks for customer environments.
  • Maintain high-quality runbooks, architecture diagrams, and troubleshooting guides—documentation is part of the job, not an afterthought.
  • Partner with the DevOps team to ensure infrastructure reliability supports CI/CD pipeline performance and developer experience.
  • Serve as a technical resource for engineering teams—sharing operational knowledge and raising infrastructure risks early.

    Requirements

    • Bachelor's or Master's in Computer Science, Electrical Engineering, or a related field (or equivalent experience); 5+ years in SRE, infrastructure engineering, or systems administration.
    • Strong Linux systems knowledge: networking, storage, systemd, package management, kernel parameters, and performance diagnostics.
    • Hands-on experience with colocation or on-premises server infrastructure — physical hardware, rack networking, and bare-metal provisioning.
    • IaC experience with Terraform and/or Ansible — writing and maintaining production configurations, not just running existing playbooks.
    • Kubernetes operational experience: cluster troubleshooting, workload management, storage, and networking.
    • Prometheus + Grafana or DataDog: building dashboards, writing alert rules, and understanding signal quality.
    • Python and/or Bash scripting: production-quality automation, not just one-off scripts.
    • Incident response experience: structured triage, RCA production, and follow-through on action items.
    • Comfort operating in fast-moving startups: you own your systems, document what you build, and iterate without waiting for perfect requirements.

    Preferred

    • Experience operating customer-facing infrastructure or platform services with external reliability expectations.
    • HPC job scheduler experience: Slurm, LSF, or equivalent — operations and troubleshooting.
    • Knowledge of high-speed interconnect fabrics: InfiniBand, RoCE, or NVLink — configuration and troubleshooting.
    • Go programming for SRE tooling — health-check services, exporters, or auto-remediation agents.
    • Experience with large-scale infrastructure automation: host lifecycle management, fleet auto-healing, or AIOps-driven operations.

    Benefits

    We offer a competitive compensation package, including a market-competitive salary, benefits, and a supportive work environment.

    Pay

    Competitive compensation package, including a market-competitive salary.

    Schedule

    6-month contract with potential conversion to full-time.

Similar jobs