Staff Software Engineer - Agent Quality

About the role

The Databricks AI Research organization is pushing the frontier of next-generation enterprise AI. This role involves designing and scaling the infrastructure, tooling, and developer workflows that enable rigorous evaluation of Databricks' AI Agents. The goal is to build a flywheel where evaluation results drive continuous improvement across the full lifecycle of agents, from development and training to production.

Responsibilities

• Stand up the foundational evaluation infrastructure for Genie Agents, enabling rigorous benchmarking, regression detection, and quality measurement across research and product teams.
• Build the flywheel that connects evaluation results back into agent improvement, closing the loop between production signals, training, and iterative development.
• Shape the long-term technical direction for agent quality infrastructure, influencing how Databricks measures and improves its first-party agents and agent development platform.

Requirements

• 6+ years industry experience building software systems.
• Strong Python programming skills, with experience building production or research infrastructure.
• Experience building or operating distributed systems, data pipelines, or large-scale infrastructure with a focus on reliability, correctness, and operational maturity.
• Ability to design pragmatic but rigorous systems that produce trustworthy, reproducible signals for complex applications.
• Comfort working across ambiguous research and product boundaries, and partnering with both researchers and engineers to turn ideas into robust internal platforms.
• A high bar for technical quality, strong ownership, and the ability to influence roadmap and execution across multiple teams.

Qualifications

• Nice to have: Experience with devtools, CI/CD platforms, testing frameworks, observability tooling, or benchmarking infrastructure.
• Familiarity with how LLM or agent quality is measured — whether through evals, experimentation platforms, or production monitoring.

Skills

• Strong technical skills in Python and related technologies.
• Experience with distributed systems and data pipelines.
• Ability to design and implement rigorous evaluation systems.
• Comfort working across research and product boundaries.

Benefits

Details on benefits are available upon request.

Pay

The pay range for this role is $190,000—$270,000 USD. Actual compensation packages are based on several factors including job-related skills, depth of experience, relevant certifications and training, and specific work location.

Schedule

Details on the schedule are available upon request.