Senior Reliability Engineer - AV Labs
About the Role
We are looking for a hardware-focused Senior Reliability Engineer to own sensor and hardware system reliability, including observability, alerting, and automation that ensures Uber's in-vehicle sensor data collection systems operate reliably at scale. This role centers on maximizing sensor uptime, data yield, and supply hours across a large, geographically distributed fleet. You will design systems to detect and react to issues impacting data recording—whether caused by failing sensors, degraded onboard computers, software regressions, or systemic environmental factors. As the technical owner for sensor reliability and observability, you will build infrastructure that converts low-level signals into actionable intelligence and automated responses. This senior role requires strong software engineering fundamentals, deep systems thinking, and the ability to drive cross-team technical direction without direct authority.
Responsibilities
- Architect Observability Systems: Design and scale an observability platform capable of ingesting and analyzing real-time health telemetry from thousands of distributed vehicle nodes.
- Build for Edge Constraints: Develop systems that remain performant despite hardware diversity, intermittent connectivity, and rapid fleet scaling.
- Define Criticality Models: Establish alerting strategies that distinguish transient anomalies from systemic issues impacting sensor uptime and data yield.
- Detect Complex Failure Modes: Design detection logic for "silent" failures, such as sensor degradation, compute saturation, or recording pipeline stalls.
- Scale Through Automation: Design automated detection, triage, and mitigation mechanisms to eliminate manual intervention as the fleet grows.
- Partner on Mitigation: Collaborate with Operations and Engineering to build safe, automated responses to recurring hardware and software failure scenarios.
- Drive Operational Efficiency: Build technical interfaces to help Operations surface issues and Engineering diagnose and deploy mitigations rapidly (TTD/TTM).
- Lead Technical Strategy: Drive reliability-focused design reviews and translate operational pain points into concrete technical requirements and roadmaps.
- Uncover Proactive Insights: Apply advanced data analytics to identify latent patterns in fleet telemetry, enabling proactive detection of systemic regressions and hardware degradation before they impact operations.
Requirements
- 5+ years of relevant industry experience in software engineering, site reliability, or systems engineering.
- Distributed Systems: Experience with modern observability platforms (e.g., Prometheus, Grafana, ELK) in edge, IoT, or hardware-integrated environments.
- Language Proficiency: Coding skills in one or more of Go, Python, or C++, with experience building and operating production systems.
- Systems Expertise: Proficiency in Linux internals and shell scripting for triaging and debugging edge devices or hardware-adjacent systems.
- Engineering Fundamentals: Ability to debug across services, containers (Docker), and networking stacks.
- Reliability Experience: Proven track record owning reliability, infrastructure, or platform systems for large-scale production workloads.
- Observability Tooling: Experience designing and operating observability systems (metrics, logging, alerting, and dashboards).
- Metrics-Driven: Experience defining and implementing SLIs and SLOs for system availability or data yield.
- Networking Knowledge: Deep understanding of networking protocols (TCP/IP, gRPC, or MQTT) and data handling in bandwidth-constrained environments.
- Leadership: Experience driving complex technical projects and architectural reviews across multiple teams from design through production.
Preferred Qualifications
- Experience with modern observability platforms (e.g., Prometheus, Grafana, ELK) in edge, IoT, or hardware-integrated environments.
- Knowledge of sensor data protocols (e.g., Camera, LiDAR, Radar) or hardware-to-cloud data ingestion pipelines.
- Experience with "Grey Failure" detection and management in complex, distributed systems.
- Proven track record in 'Fleet Health' for large-scale hardware deployments (e.g., cloud infrastructure, global server fleets, or industrial IoT) where automation was used to replace manual intervention.
Pay
For Sunnyvale, CA-based roles: The base salary range is USD $180,000 per year - USD $200,000 per year. You will be eligible to participate in Uber's bonus program and may be offered an equity award and other types of compensation.
Benefits
- 401(k) plan participation for full-time employees.
- Eligibility for various benefits.
Schedule
Offices remain key to collaboration and Uber's culture. Unless approved for full remote work, employees must spend at least 50% of their time in-office. Some roles, like those at greenlight hubs, require full-time in-office presence.