Machine Learning Engineer Perception LLM/VLM (PhD, New Grad)

Job Summary

The Perception team at Waymo is seeking a highly skilled engineer to design, implement, and optimize large-scale continual pre-training pipelines for cutting-edge Vision-Language Models (VLMs). This role involves conducting research and development on novel pre-training techniques, developing and evaluating metrics for performance and transferability of continually pre-trained foundation models, and staying current with advancements in large language models, vision-language models, and continual learning.

About the Role

Join Waymo, where you'll work on the cutting edge of autonomous driving technology, contributing to the development of the Waymo Driver—the world's most experienced driver. As part of the Perception team, you'll play a crucial role in enhancing the Waymo Driver's ability to perceive and interact with the world around it through large-scale real-world data.

Responsibilities

• Design, implement, and optimize large-scale continual pre-training pipelines for cutting-edge VLM foundation models.
• Conduct research and development on novel pre-training techniques, focusing on efficiently integrating new, diverse, and multimodal data streams into existing models.
• Develop and rigorously evaluate metrics and methodologies for measuring the performance and transferability of continually pre-trained foundation models in the context of autonomous driving.
• Stay current with the latest advancements in large language models, vision-language models, and continual learning, and translate relevant research into production-ready systems.

Requirements

• 5+ years of experience in Machine Learning, with a focus on large-scale model development (LLM, VLM, or similar foundation models).
• Proven expertise in LLM/VLM pre-training, continual learning with large scale datasets.
• Strong coding proficiency in Python and deep learning frameworks (e.g., Jax, TensorFlow, PyTorch).
• Hands-on experience with model training, evaluation, and deployment in a production environment.
• Master's degree in Computer Science, Electrical Engineering, or a related field, or equivalent practical experience.

Qualifications

• Experience in fine-tuning foundation models for autonomous driving or robotics applications.
• Familiarity with large-scale data curation and quality assurance processes for multimodal datasets.
• Background in autonomous vehicle perception, motion planning, or decision-making systems.
• Publications in top-tier machine learning or computer vision conferences (e.g., NeurIPS, ICML, CVPR, ICCV, ECCV).
• PhD in a relevant field.

Benefits

We offer competitive compensation, a comprehensive benefits package, and opportunities for professional growth within a dynamic and innovative organization. Waymo employees are also eligible to participate in our discretionary annual bonus program, equity incentive plan, and generous Company benefits program, subject to eligibility requirements.

Pay

The expected base salary range for this full-time position across US locations is listed below. Actual starting pay will be based on job-related factors, including exact work location, experience, relevant training and education, and skill level.

Contact

Your recruiter can share more about the specific salary range for the role location or, if the role can be performed remote, the specific salary range for your preferred location, during the hiring process.