Senior Computational AI Architect

Responsibilities

• Instruction design: develop annotation guidelines, create reviewer handbooks, design evaluator training programs, and establish escalation frameworks.
• Rubric development: design evaluation rubrics, build reasoning-assessment frameworks, create multilingual quality standards, and define quality acceptance criteria.
• Linguistic consulting: review project instructions, identify ambiguity risks, improve linguistic consistency, and guide localization efforts.
• Evaluation oversight: assess evaluator agreement, review pilot outputs, identify systematic quality issues, and recommend rubric improvements.

Requirements

• Master's or PhD in Computational Linguistics, NLP, Linguistics, or Cognitive Science.
• 8+ years of experience.
• Expertise across NLP, LLM evaluation, human annotation systems, localization, multilingual AI, and quality frameworks.
• Strong written English and the ability to communicate complex ideas clearly.
• Comfortable working independently in a remote environment.

Based in the United States.