Verifier Engineer

The role A reward is only as good as the check behind it. You’ll write the verifiers that decide whether an agent did the work: executable tests, state diffs, rubric graders, and the adversarial cases that catch an agent taking a shortcut. What you’ll own Executable checks for coding, data and operational tasks: tests, invariants, state comparisons, output validators.Rubric-based graders for tasks without a single right answer, calibrated against expert review.Shortcut hunting: ways to get the reward without doing the task, closed before a customer trains on them.Verifier precision and recall against held-out human judgements, reported honestly.A library of reusable verification primitives across environments. What you bring Strong software testing instincts and a habit of thinking adversarially about your own code.Fluent Python, and comfort reading unfamiliar codebases in other languages.Statistical literacy: you can tell when a grader’s agreement rate is good enough and when it isn’t.Patience for edge cases. The tenth one matters as much as the first. Helpful, not required Property-based testing, fuzzing or formal checks.Model-as-judge pipelines and their failure modes.