AI Research Scientist

About the Role

Human‑centered physical intelligence starts with models that learn from how people actually do things in the real world. Our systems run in homes and workplaces with people doing real tasks, and every deployment returns hours of continuous, time‑synchronized multimodal data. Nothing comparable exists publicly, and the models and benchmarks for this kind of data have yet to be built. We’re looking for an experienced researcher to shape what these models should be capable of, build the evaluations that measure it, pre‑train and post‑train multimodal foundation models on our data, and ship them into systems operating in the real world, where telemetry returns within days and shapes what we collect next.

What You’ll Do

• Design and run the eval suite: create a repeatable, versioned system that measures capabilities over real‑world input and is used by the whole team.
• Build novel evals for tasks without established benchmarks, tying metrics to real‑world usefulness.
• Extend evals to online evaluation of model behavior in deployment, where static metrics stop being predictive.
• Analyze failure modes, red‑team behavior, and turn results into concrete priorities for data and training.
• Train, deploy, and study multimodal foundation models (SFT, RL, preference optimization) and design experiments that isolate performance improvements.
• Take models from research into production: export, optimize, and run inference across edge and cloud targets under real‑time constraints, and build telemetry that feeds failures and edge cases back into the data engine.
• Prototype agentic systems that perceive, reason, and act over continuous multimodal input, both in real time and offline over recorded data.
• Run scaling‑law studies to understand how capability moves with data volume, diversity, model size, and compute, and use insights to shape the research and data roadmap.
• Scale the data engine: work across the full data lifecycle for large volumes of multimodal, time‑synchronized data captured from real hardware on real‑world tasks.
• Grow the engine along raw volume and domain coverage so models generalize across environments, tasks, embodiments, and people.
• Build tooling that tightens the loop between deployment, data, and training, turning real‑world signal into supervision that existing datasets don’t offer.
• Collaborate cross‑functionally to translate model needs into human data‑collection priorities, partnering with engineering on pipeline infrastructure for storage, versioning, and serving.

What We’re Looking For

• PhD in machine learning, computer vision, robotics, or a related field, with a record of training models that reached production.
• Deep experience training and post‑training multimodal models (vision, video, audio, language) – SFT, RLHF, DPO or GRPO – and building RL environments.
• Track record of designing evals and benchmarks, especially for tasks without established benchmarks, and strong opinions about eval trustworthiness.
• Hands‑on experience building data pipelines for model training at scale: ownership of curation, annotation, and quality on real datasets, and ability to identify hidden data problems.
• Experience taking models into production on‑device or real‑time inference on compute‑constrained hardware, including quantization and latency trade‑offs.
• Strong engineering fundamentals: clean, reproducible code and comfort building infrastructure when needed.
• Practical fluency with AI coding tools and agents in daily workflow, with good judgment about when to rely on them.
• Startup DNA: high ownership, comfort with ambiguity, and pragmatic decision‑making on a small, flat, collaborative team.

Stack & Skills

• Fluent in PyTorch and the modern training stack – distributed training, mixed precision, experiment tracking, reproducible pipelines.
• Multimodal and vision‑language models (VLMs) – architectures, post‑training recipes (SFT, RLHF, DPO/GRPO), and RL environments.
• Eval methodology – benchmark design, held‑out set construction, statistical rigor, and human preference data.
• Python, with ability to move into systems code when pipelines need it.
• Video understanding – temporal segmentation, action recognition, long‑horizon reasoning.
• Data engineering for ML – large‑scale multimodal data processing, dataset versioning, annotation tooling, quality/coverage measurement.
• Speech and audio models – streaming ASR and diarization in real‑world conditions.
• Retrieval – multimodal embeddings and vector search.
• Model deployment and optimization – ONNX, TensorRT, vLLM, or similar inference stacks; quantization, distillation, profiling on edge and real‑time targets.
• Cloud infrastructure (AWS or GCP), containers, and orchestration for training and data jobs.
• Streaming and real‑time inference over continuous sensor input.

Bonus Experience

• Building physical AI data engines – pipelines, annotation systems, and quality loops behind large real‑world datasets.
• Running human data‑collection programs, including consent and privacy handling.
• Published work at top venues (NeurIPS, CVPR, ICCV, ICML, CoRL, RSS, ICRA) grounded in real‑world deployment.
• Experience with active learning, data selection, or scaling‑law studies.
• Research in embodied perception, human‑robot or human‑agent interaction, or robot learning.
• Cross‑embodiment learning, world models, or sim‑to‑real transfer.
• Learning from demonstration or policy learning over multimodal sensor streams – vision‑language‑action models, diffusion policies, behavior cloning from teleoperation data.
• Simulation environments – Isaac, MuJoCo, or similar – for training, evaluation, or data generation.
• Robot perception stacks – SLAM, 3D reconstruction, depth and point‑cloud processing, grasp or affordance prediction.
• Real‑world robot evaluation on physical hardware, or contributions to open robot learning ecosystems (Open X‑Embodiment, LeRobot, RoboCasa).

Who You’ll Work With

You’ll join a tightly knit team that has pioneered frontier AI research, shipped tens of millions of devices, scaled infrastructure used by millions every day, and defined how people and machines interact in the physical world. Team members have experience at Meta, Amazon, Microsoft, and Valve.

Compensation & Benefits

• Competitive salary
• Meaningful early‑stage equity
• Benefits package