Applied Research - Evals & Data

Role Impact

This is a customer facing role at the intersection of cutting-edge RL/post-training methods, applied data, and agent systems. You’ll have a direct impact on shaping how advanced models are aligned, evaluated, deployed, and used in the real world by:

• Advancing Agent Capabilities: Designing and iterating on next-generation AI agents that tackle real workloads—workflow automation, reasoning-intensive tasks, and decision-making at scale.
• Working with applied data from real deployments to continuously refine policies, improve reasoning, and enhance reliability and safety.
• Bridging the gap between customers and research: Translating customer needs and insights from applied data into clear technical requirements that guide product and research priorities.
• Collaborating closely with RL and eval teams to ensure real-world signals inform model alignment and reward shaping.
• Prototyping in the field: Rapidly designing and deploying agents, evals, and harnesses alongside customers to validate solutions.
• Using applied evaluation data to iterate on model performance and discover new capabilities.
• Customer-facing engineering work: Side-by-side with customers to deeply understand workflows, data sources, and bottlenecks.
• Translating customer insights and evaluation results into roadmap and research direction.

Requirements

• Strong background in machine learning engineering, with experience in post-training, RL, or large-scale model alignment.
• Experience with applied data workflows and evaluation frameworks for large models or agents (e.g., SWE-Bench, HELM, EvalFlow, internal eval pipelines).
• Deep expertise in distributed training/inference frameworks (e.g., vLLM, sglang, Ray, Accelerate).
• Experience deploying containerized systems at scale (Docker, Kubernetes, Terraform).
• Track record of research contributions (publications, open-source contributions, benchmarks) in ML/RL.
• Passion for advancing the state-of-the-art in reasoning, measurement, and building practical, agentic AI systems.