Linguist - AI Rater Guidelines Expert

About the role

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

This is a remote contract position with compensation of $45–$65/hour.

Responsibilities

• Guide research and program teams to close the gap between ambiguous program requirements and rater-ready instructions across various domains.
• Design clear, non-contradictory rating guidelines and rubrics for consistent application, including under edge cases.
• Evaluate and revise draft guidelines for ambiguity, contradiction, and coverage gaps to ensure raters can apply them without escalation.
• Translate program specifications from domains such as finance, retail, insurance, legal, and sports into precise, discipline-specific rater instructions.
• Collaborate with subject matter experts and program leads to ensure consistency and accuracy across guideline sets.

Requirements

• 3+ years of professional experience in linguistics, instructional design, technical writing, or a related field.
• Direct experience writing or refining guidelines/rubrics for human raters in a GenAI/RLHF context.
• Ability to work across multiple domains (e.g., finance, retail, insurance, legal, sports) and translate domain-specific nuances.
• Strong track record of resolving ambiguity and contradiction in written specifications.
• Ability to engage reliably for at least 35 hours/week during weekdays.
• Strong written communication skills to explain complex guidance clearly.