Program Manager, Human Evaluations (Recommendations)

For our client, we are seeking a Program Manager, Human Evaluations (Recommendations) to join the team of a leader in the Hospitality & Recreation space. This role will lead the execution of human evaluation programs that help shape recommendation quality and improve decision-making across a large-scale consumer experience. You will partner closely with strategy, operations, and vendor teams to turn evaluation goals into clear designs, reliable scoring frameworks, and actionable outcomes. The position requires strong program management discipline, a focus on process improvement, and the ability to keep complex evaluation work moving efficiently from planning through delivery. Location: Remote - US based candidates only, no visa sponsorship available Compensation: $230,000 – $360,000 annually Responsibilities Lead execution of human evaluation initiatives from scoping to delivery Translate recommendation strategy into evaluation designs and metrics Manage human annotation vendors, ensuring quality and performance Drive operational efficiency across evaluation programs Develop scalable guidelines and scoring rubrics for rater judgment Oversee rater onboarding and calibration processes Anticipate and resolve project bottlenecks and blockers Qualifications 4+ years in human evaluations or data operations in ML environments Proven track record in process improvements for data collection Experience managing human annotation vendors and teams Strong understanding of evaluation design and scoring protocols Fluency in film and TV content, including genres and formats Excellent communication skills for synthesizing feedback Strong organizational skills with a focus on execution Benefits Comprehensive health plans and mental health support 401(k) retirement plan with employer match Stock option program Flexible spending accounts and health savings accounts Family-forming benefits Paid leave of absence programs 35 days of paid time off for full-time hourly employees Immediate flexible time off for full-time salaried employees Our client is an equal opportunity employer. We encourage you to apply even if you don’t meet every qualification—your background could be exactly what this team needs.