Nuance Research Fellow
About Nuance Labs
Nuance Labs is building photorealistic, real-time AI avatars with emotional intelligence: a full-duplex audiovisual system that can listen, speak, react, interrupt, and respond like a real person.
The team is small, the work is real, and the problems are unsolved.
How Nuance Differentiates
Most conversational AI avatars today are hacks — a face slapped on a speech-to-speech pipeline, stuck in the uncanny valley: emotionless, mechanical, one-turn-at-a-time. Current systems take 2–5 seconds to respond; natural conversation requires sub-500ms. That's a 10x improvement, and it demands rethinking the entire stack.
About the Role
The Nuance Research Fellowship is a 3-month engagement for early-career researchers who want to work at the frontier of Multimodal LLMs, generative modeling, and real-time audiovisual AI. The program is open to current PhD students (on internship, leave, or in their final stretch) and recent graduates from BS, MS, or PhD programs.
What You’ll Own
- Own a concrete research problem from framing through experiments, analysis, and integration into the Nuance stack
- Work on frontier Multimodal LLM systems spanning audio, video, language, and real-time interaction
- Explore and adapt modern generative modeling techniques, including flow matching, diffusion, autoregressive modeling, and hybrid approaches where they fit
- Read papers, reproduce key results, and turn promising ideas into production-grade experiments
- Design, instrument, debug, and interpret training and evaluation runs with scientific rigor
- Build evaluation harnesses, benchmarks, and analysis tooling for real-time conversational agents
- Take research-grade prototypes and turn them into systems that ship
- Work closely with senior researchers and engineers across the team; ramp on the stack fast
What We’re Looking For
- Strong working knowledge of PyTorch and deep learning — you can train a model, debug a training run, and reason about what’s happening at the loss level
- At least one first-author paper at a tier 1 venue (main conference proceedings) — NeurIPS, ICML, ICLR, CVPR, ICCV, ECCV, ACL, EMNLP, NAACL, ICASSP, Interspeech, MLSys, SIGGRAPH, or equivalent — or equivalent evidence of unusually strong research taste and execution
- Genuine interest in joining Nuance full-time after the fellowship. We are looking for long-term partners on this journey
Beyond the hard bar
- Currently enrolled in or recently completed a BS, MS, or PhD in CS, ML, math, physics, EE, or a related field
- Strong programming ability and software engineering instincts
- A bias toward shipping over polishing, with the judgment to know when each matters
Bonus Points
- Hands-on experience with Multimodal LLMs, omni models, audio-language models, video-language models, speech generation, or real-time interactive agents
- Research or implementation experience with flow matching, diffusion models, rectified flows, autoregressive generation, neural codecs, or related generative modeling methods
- Multiple tier 1 publications, or a paper that received significant attention (best paper award, broad adoption, high citation impact for its age)
- Olympiad medals or finalist-level results in IMO, IPhO, IOI, IChO, IBO, IMC, or equivalent
- Open-source contributions to major ML frameworks or research codebases
- A track record of independent projects that made something noticeably faster, smaller, or better
Compensation
$200,000 – $250,000 annualized base salary during the 3-month fellowship (paid as a prorated stipend). Fellows who convert to a full-time Member of Technical Staff role step into a base salary of $250,000 – $350,000 plus meaningful equity.
Logistics
- Location: In-person in Seattle, five days a week — we believe in the compounding value of working shoulder-to-shoulder.
- Visa sponsorship: We sponsor visas (O-1, H-1B, green card) from day one.
- AI-native tooling: Do your best work with the best tools, including unlimited tokens.
Benefits
- Health: HSA plan with ~$2,000 in annual company contributions — roughly 2x what most big tech companies put in.
- Time off: 15 days of PTO plus public holidays, and we close the office for a full week at year-end.
- Food: Lunch, drinks, and snacks on us every workday — the small thing that quietly makes the day better.
- Commuter benefits: We help cover the cost of getting to the office.
- 401(k): In the works.