Senior Data Scientist, Systems Performance

What You’ll Be Doing

• Lead the development of evaluation frameworks for the autonomous system, connecting technical problems to rigorous, data-driven approaches for measuring and validating performance.
• Collaborate closely with Functional Safety and Systems Engineering teams to ensure evaluation metrics map effectively to automotive safety standards (e.g., SOTIF, ISO 21448) and launch readiness decisions.
• Ensure evaluation metrics are reliable enough to inform safety cases and launch readiness decisions.
• Monitor the reliability of evaluation metrics and incoming performance data over time, including detecting drift, inconsistencies, and degradation in metric definitions, to ensure the evaluation ecosystem remains accurate and trustworthy.
• Drive our approach to performance analysis using data-backed statistical methods for simulation and on-road data.
• Develop new statistical analysis methods to analyze AV performance data and lead by example in applying them to real problems.
• Partner with triage operators and simulation engineers to turn raw disengagements and identified edge cases into procedural or generative scenarios, feeding them back into the simulation catalog to strengthen test coverage.
• Identify edge cases in an automated manner and coverage gaps, and partner with engineering to feed novel scenarios back into the simulation catalog and strengthen test coverage.
• Build confidence in the evaluation framework through data-driven insights and clear communication of findings to technical leaders and stakeholders.
• Establish correlation between on-road and simulation data to improve how we interpret and act on evaluation results.
• Make sense of large datasets to drive insights, solve ambiguous performance questions, and communicate results effectively across teams and upward to leadership.
• Develop new metrics, interpret trends, and investigate anomalies in simulation and on-road data.
• Collaborate with developers to drive action based on these results.
• Serve as an advisor and influence collaborators across multiple teams, promote data-aware decision making, and establish best practices around the use of data.
• Mentor and collaborate with fellow engineers and foster a positive, collaborative work environment.
• Introduce the use of ML methods for performance evaluation where they add rigor and scale.

What You Bring

• 5+ years of industry experience solving complex problems with large datasets, with a track record of framing ambiguous questions into rigorous, data-driven analyses.
• Bachelor's or higher degree in Computer Science, Computer Engineering, Data Science, Robotics, Physics, Mathematics, or a related quantitative field. Master's or PhD preferred.
• Strong problem-solving skills: ability to break down complex performance and evaluation challenges, think logically, and remove bias from how problems are defined and assessed.
• Strong Python and SQL skills, with demonstrated experience using data analysis libraries to work with large, complex datasets.
• Experience applying advanced statistical and ML methods to drive insights from large and complex data sets.
• Demonstrated experience with statistical analysis, hypothesis testing, causal analysis and data analysis.
• Demonstrated ability to work independently with minimal guidance and drive projects from problem definition through to actionable results.
• Proven communication and interpersonal skills, with the ability to explain technical findings clearly to engineering partners and leadership.
• Eager to learn new statistical and ML techniques and demonstrated willingness to teach.

Bonus Points

• Experience with adversarial scenario generation and closed-loop simulation environments.
• Experience in autonomous driving or robotics, specifically evaluating sub-systems like Perception, Prediction, or Motion Planning.
• Familiarity with data pipelines and distributed compute (e.g., AWS) to seamlessly collaborate with our Data Engineering and MLOps partners.
• Expertise in Machine Learning and Deep Learning.
• Expertise in modern sequence modeling (e.g., Transformers applied to time-series or trajectory data) and probabilistic ML / uncertainty quantification for distinguishing rare-but-safe behavior from out-of-distribution failures.
• Familiarity with automotive safety standards like ISO 26262 or ISO 21448 (SOTIF).