Reward Integrity Analyst
Reinforced Learning · Buffalo-Niagara Falls Area · 3 days ago
Human ResourcesFull-time
The role Models find shortcuts. If a verifier can be gamed, it will be. You’ll audit verifiers, rubrics and environments for ways to earn reward without doing the task, and work with engineers to close them. What you’ll own Attempts to exploit verifiers and environments the way a model would.Rollout review for reward hacking, with each pattern documented.A catalogue of exploit classes and the checks for them.Measurement of how often rewards disagree with expert judgement.Sign-off on reward integrity before a program ships. What you bring An adversarial mindset from security testing, QA, competitive programming or similar.Comfort reading code, logs and trajectories.Python for scripting your own attacks.Clear, precise reporting. Helpful, not required RL or language model training.CTFs, bug bounty or red teaming.