Senior Software Engineer, Simulator Evaluation

Job Summary

The Simulator Evaluation team at Waymo is seeking a Senior Software Engineer to architect the metrics and systems that grade the hybrid virtual environments used by the Waymo Driver. This role involves developing novel methodologies to evaluate the simulator across the stack, designing and implementing high-throughput pipelines, and partnering with AI research and other simulation teams.

Key Responsibilities

• Architect the Eval Rubrik: Develop novel methodologies to evaluate the simulator across the stack, distinguishing between true driving challenges and realism artifacts.
• Build at Scale: Design and implement high-throughput pipelines (C++/Python) capable of processing massive datasets of simulation logs, turning raw, noisy data into clear, actionable signals.
• "Critic" for the System: Partner closely with AI research and other simulation teams, ensuring that the eval workflows you build drive rapid innovation and research roadmaps.
• Strategic Leadership: Navigate ambiguity to determine what matters most for realism, leading the strategy for specific domains and ensuring our evaluation evolves as fast as our simulation technology.

Qualifications

• 5+ years of software development experience
• Proficiency in Python or C++, with experience building scalable data processing systems or evaluation frameworks
• Strong software design principles: Write clean, testable code that is built to last
• Data Intuition & Quantitative Rigor: A "Data Detective" mindset, able to look at a distribution of outcomes and intuitively spot anomalies, selection bias, or system errors
• Experience designing and implementing evaluation frameworks for complex systems or machine learning models
• System & Model Fluency: Comfort working with complex, hybrid systems, understanding how to evaluate different types of "black boxes"
• Background in fields that blend code, math, and simulation: Autonomous Vehicles, Algorithmic Trading, AdTech/Search Ranking, Machine Learning, or Robotics
• Experience with SQL and the Python data stack (Pandas, NumPy, SciPy)
• Familiarity with evaluating Generative AI / LLMs or experience with agent-based modeling and behavioral logic
• Experience taking a metric from "research concept" to "production pipeline."