Jobs · Education · California

Principal Applied Scientist, Real-Time Conversational AI , AGI

TheDataJob · Sunnyvale, CA · 2 mo ago
EducationFull-time

About the role

We are looking for a Principal Applied Scientist to drive the research and development of real-time multimodal conversational AI. You will operate across two focus areas: advancing foundation models for speech and audio, and building the post-training systems (reward modeling, reinforcement learning) that shape natural, human-like conversational behavior. You will work at the frontier of conversational AI, with the compute, data, and resources to tackle problems few teams in the world can address.

As a Principal Scientist, you will set the technical direction for your research area, influence the broader roadmap, and collaborate closely with inference engineers to ensure models are designed for real-time production deployment from inception.

Responsibilities

  • Build and train large-scale multimodal foundation models for real-time speech and audio generation, from architecture design through production-scale training.
  • Advance the scaling and efficiency of conversational modes, including the relationship between data, model size, and real-time performance.
  • Design model architectures informed by hardware constraints and inference requirements, working with inference engineers to ensure models are servable from inception.
  • Develop training methodologies for multimodal models that jointly process and generate speech, language, and audio in real-time streaming contexts.
  • Design and build reward models and reward functions for speech systems—capturing naturalness, fluency, conversational quality, and real-time responsiveness.
  • Develop and apply reinforcement learning methods to shape conversational behavior, teaching models natural timing, responsiveness, and fluid interaction.
  • Build the post-training pipeline from SFT through RL alignment, optimized for real-time multimodal outputs rather than text-only generation.
  • Design evaluation frameworks that capture the quality dimensions unique to real-time conversation.
  • Advance the team's capabilities in real-time perception and work at the intersection of model architecture and production constraints to ensure multimodal capabilities function within hard real-time latency budgets.
  • Contribute across the full model lifecycle—from pre-training and architecture design through post-training alignment and real-time deployment.

Similar jobs