Staff Machine Learning Engineer – VLM/LLM Evaluation

Job Summary

The Waymo AI Foundations team is seeking a software engineer to lead the development of end-to-end evaluation systems and benchmarks for Waymo Foundation models. This role involves working with a creative team to build state-of-the-art Foundation Models used throughout Waymo’s systems, both onboard autonomous vehicles and offboard in simulation.

Responsibilities

• Lead the development of end-to-end evaluation systems and benchmarks for Waymo Foundation models, encompassing the entire lifecycle from pretraining and supervised fine-tuning (SFT) to reinforcement learning (RL).
• Partner within and across organizations to land disruptive and innovative tech in production.
• Implement and extend large-scale data and evaluation pipelines.

Requirements

• Master’s degree or PhD degree in Computer Science, similar technical field of study, or equivalent practical experience.
• 5+ years of experience in ML engineering and applied Deep Learning, with a strong portfolio of shipped products or publication record.
• Experience with large-scale distributed systems.
• Proficient programming skills (Python, C/C++).
• Strong analytical and debugging skills.

Preferred

• ML infrastructure experience: training, evaluating, and deploying ML models at scale.
• Deep learning experience, especially with generative models, e.g., LLMs/VLMs, and/or reinforcement learning.
• Proficiency and in-depth knowledge of the inner workings of an ML framework (Pytorch, JAX, Tensorflow).

Benefits

Benefits for this role include:

• Health, dental, vision, life, disability insurance
• Roth 401(k) with company match
• Paid Time Off: 20 days per year, accruing at a rate of 6.15 hours per pay period for the first five years of employment
• Sick Time: 40 hours/year (statutory, where applicable); 5 days/event (discretionary)
• Maternity Leave (Short-Term Disability + Baby Bonding): 28-30 weeks
• Baby Bonding Leave: 18 weeks
• Holidays: 13 paid days per year