Jobs · Engineering · Washington

Senior Context Fusion AI Engineer - Autonomous Vehicles

NVIDIA · Redmond, WA · 1 mo ago
EngineeringFull-time
We need to output HTML fragment with sections: About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule. Only include sections that have real specific content. We need to extract content. The posting includes: - Intro paragraph: "We are looking for a strong engineer to join the DRIVE Road Structure / Online Mapping / Context Fusion team. In this role, you will help craft and guide the future of our L3/L4 autonomous-driving solution by building a complete, learned 3D/4D world model that fuses navigation, ego-motion, perception, and sensor signals. You will work closely with perception, prediction, planning, and simulation teams to deliver a world representation that is complete, temporally consistent, uncertainty-aware, and robust enough to drive through the most challenging roads and intersections in L3/L4 autonomy level. This role is central to our vision for AV: developing a shared multimodal scene representation that can jointly support road understanding, 3D object and occupancy perception, motion prediction, and route-conditioned planning. Are you interested in inventing human-level AI for navigation in the unconstrained world under any conditions? If so, join us!" That's about the role. Then "What You'll Be Doing" list: bullet points. Then "What We Need To See": list of requirements. Then "Ways To Stand Out From The Crowd": list of nice-to-have. Then "We believe that building self-driving vehicles..." etc: maybe not needed. Then "Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 184,000 USD - 287,500 USD for Level 4, and 224,000 USD - 356,500 USD for Level 5. You will also be eligible for equity and benefits." Thus we have Pay info. Benefits: mention equity and benefits, but not specifics. Could we include Benefits section? The posting says "You will also be eligible for equity and benefits." That's vague but still a statement about benefits. However rule: never create Benefits section unless posting actually states them. It does state eligibility for equity and benefits, but not specifics. Might be considered real content? It says "eligible for equity and benefits." That's a statement. I'd include a Benefits section with that sentence. Schedule: none. Thus sections: About the role (intro), Responsibilities (What You'll Be Doing), Requirements (What We Need To See), Qualifications? Could combine Requirements and Qualifications? The posting has "What We Need To See" which includes qualifications. Could treat as Requirements. Also "Ways To Stand Out From The Crowd" could be Skills or Qualifications? Might be "Preferred Qualifications" or "Skills". We could make a Skills section for the standout items. We need to avoid repeating job title as heading. No Job ID. We need to use

for section headings. We need
  • for any list of duties/requirements/benefits. We need

    for prose. We must preserve all substantive information, not invent or summarize away detail. We must remove boilerplate: cookie/navigation/legal/EEO/equal-opportunity/scam-warning boilerplate, duplicated lines, and application instructions. The posting includes EEO statement, AI tools, etc. We must remove those. Also remove duplicated lines: there is duplicate of "Experience with Transformer, VLM, or multimodal foundation-model architectures, including pre-training, fine-tuning, distillation, quantization, or efficient inference." appears both in Requirements and Ways To Stand Out. We need to preserve all substantive information; but if duplicated, we should not repeat? The instruction: "remove duplicated lines". So we should deduplicate. We need to preserve all substantive information, but we can deduplicate identical lines. Let's extract content. Intro paragraph (About the role): We'll keep as is. Responsibilities: "What You'll Be Doing" bullet list: - Design and develop learning-based, multimodal sensor-fusion systems that transform synchronized sensor history, ego-motion, navigation context, and driving context into a unified spatiotemporal world representation. - Build architectures that jointly reason over camera, LiDAR, radar, and vehicle-state inputs, with appropriate handling of calibration, synchronization, coordinate transforms, sensor latency, and uncertainty. - Develop end-to-end and multi-task models that produce driving-relevant outputs from a shared scene representation, including; road graph elements such as lanes, boundaries, crosswalks, and traffic controls; semantic scene understanding; occupancy and free-space representations, including uncertain and occluded regions; - Develop scalable multimodal fusion architectures, including Transformer-based early, late, and hierarchical fusion; BEV, point/voxel, and image-based representations; temporal context aggregation; and cross-modal attention. - Create training, fine-tuning, and evaluation pipelines for large-scale multimodal datasets. Define multi-task objectives and metrics that balance perception quality, geometric consistency, prediction accuracy, latency, and safety-critical behavior. - Investigate foundation-model approaches for autonomous driving, including vision-language models, multimodal pre-training, representation learning, and efficient deployment of learned world models. - Work closely with perception, mapping, prediction, planning, simulation, data, and embedded-software teams to convert research advances into robust, production-quality AV systems. - Develop systematic analysis and debugging tools for model failures, cross-sensor disagreement, long-tail scenarios, distribution shift, and regressions in closed-loop simulation and on-road evaluation. Note there are some stray semicolons and punctuation; we keep as is. Requirements: "What We Need To See" bullet list: - BS, MS, or PhD in Computer Science, Robotics, Electrical Engineering, Machine Learning, or a related technical field, or equivalent experience. - 8+ years of experience, with at least 2+ years in the AV or robotics industry and 2+ years of leadership experience in a technically area (typo: "technically area" maybe "technical area"). - Strong experience developing production-quality sensor-fusion, perception, state-estimation, or autonomous-driving systems. - Demonstrated experience with learning-based multimodal perception or fusion involving two or more cameras, LiDAR, radar, map, navigation, and ego-motion signals. - Solid understanding of 3D geometry, coordinate frames, calibration, temporal synchronization, ego-motion compensation, tracking, uncertainty estimation, and sensor failure modes. - Experience with deep-learning methods for 3D perception, lego-context scene representation, occupancy/occlusion prediction, semantic segmentation, object detection/tracking, motion prediction, or planning. - Strong C++ and Python programming skills, with hands-on experience developing, training, and optimizing deep-learning models in PyTorch. Experience with CUDA, distributed training, mixed-precision techniques, and efficient GPU inference using NVIDIA software and hardware is highly valued. - Experience with Transformer, VLM, or multimodal foundation-model architectures, including pre-training, fine-tuning, distillation, quantization, or efficient inference. - Experience training and evaluating models at scale, including distributed training, dataset curation, offline evaluation, simulation-based validation, and production monitoring. - Ability to work across research and engineering boundaries: turn an ambiguous AV problem into measurable technical objectives, build the solution, and drive it to deployment. Note: there is a typo "technically area". We'll keep as is. Ways To Stand Out From The Crowd (Preferred Qualifications/Skills): - Experience building multi-task driving models that jointly predict perception, road structure, occupancy, motion, and/or trajectories from shared multimodal features. - Experience with Transformer, VLM, or multimodal foundation-model architectures, including pre-training, fine-tuning, distillation, quantization, or efficient inference. (duplicate) - Experience with BEV, point-cloud/voxel, neural scene representation, 3D reconstruction, occupancy-flow, or spatiotemporal world-model methods. - Publications or open-source contributions in computer vision, robotics, machine learning, 3D perception, multimodal learning, or autonomous driving. - Experience optimizing models for automotive-grade real-time deployment using NVIDIA GPUs, TensorRT, CUDA, or edge inference toolchains. We need to deduplicate the duplicate line about Transformer etc. Keep only one occurrence. Pay: "Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 184,000 USD - 287,500 USD for Level 4, and 224,000 USD - 356,500 USD for Level 5. You will also be eligible for equity and benefits." We could split into Pay and Benefits. Pay: salary range. Benefits: equity and benefits eligibility. Thus: Pay section:

    Pay

    Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 184,000 USD - 287,500 USD for Level 4, and 224,000 USD - 356,500 USD for Level 5.

    Benefits section:

    Benefits

    You will also be eligible for equity and benefits.

    Schedule: none. We must not include sections with filler like "commensurate with experience". Not present. Now produce HTML fragment. We need to ensure we don't include any markdown fences. Let's craft:

    About the role

    We are looking for a strong engineer to join the DRIVE Road Structure / Online Mapping / Context Fusion team. In this role, you will help craft and guide the future of our L3/L4 autonomous-driving solution by building a complete, learned 3D/4D world model that fuses navigation, ego-motion, perception, and sensor signals. You will work closely with perception, prediction, planning, and simulation teams to deliver a world representation that is complete, temporally consistent, uncertainty-aware, and robust enough to drive through the most challenging roads and intersections in L3/L4 autonomy level. This role is central to our vision for AV: developing a shared multimodal scene representation that can jointly support road understanding, 3D object and occupancy perception, motion prediction, and route-conditioned planning. Are you interested in inventing human-level AI for navigation in the unconstrained world under any conditions? If so, join us!

    Responsibilities

    • Design and develop learning-based, multimodal sensor-fusion systems that transform synchronized sensor history, ego-motion, navigation context, and driving context into a unified spatiotemporal world representation.
    • Build architectures that jointly reason over camera, LiDAR, radar, and vehicle-state inputs, with appropriate handling of calibration, synchronization, coordinate transforms, sensor latency, and uncertainty.
    • Develop end-to-end and multi-task models that produce driving-relevant outputs from a shared scene representation, including; road graph elements such as lanes, boundaries, crosswalks, and traffic controls; semantic scene understanding; occupancy and free-space representations, including uncertain and occluded regions;
    • Develop scalable multimodal fusion architectures, including Transformer-based early, late, and hierarchical fusion; BEV, point/voxel, and image-based representations; temporal context aggregation; and cross-modal attention.
    • Create training, fine-tuning, and evaluation pipelines for large-scale multimodal datasets. Define multi-task objectives and metrics that balance perception quality, geometric consistency, prediction accuracy, latency, and safety-critical behavior.
    • Investigate foundation-model approaches for autonomous driving, including vision-language models, multimodal pre-training, representation learning, and efficient deployment of learned world models.
    • Work closely with perception, mapping, prediction, planning, simulation, data, and embedded-software teams to convert research advances into robust, production-quality AV systems.
    • Develop systematic analysis and debugging tools for model failures, cross-sensor disagreement, long-tail scenarios, distribution shift, and regressions in

Similar jobs