Jobs · OTHR

JSON-fluent & Rubrics Expert

Weekday AI (YC W21) · United States · 1 mo ago
RemoteRemoteOTHR$50–$175/hrPart-time

Key Responsibilities

  • Review and understand written task descriptions and evaluation objectives
  • Interpret detailed grading rubrics and apply them consistently across assignments
  • Read, parse, and evaluate JSON-formatted AI model outputs against predefined quality criteria
  • Assign accurate, evidence-based scores while maintaining consistency across evaluations
  • Identify discrepancies, missing information, or rubric violations during the review process
  • Maintain high standards of quality, accuracy, and attention to detail while meeting project turnaround expectations
  • Complete all evaluations independently without using AI-assisted tools

Core Qualifications

  • Strong ability to read and interpret JSON files and structured data
  • Experience applying detailed rubrics, quality standards, or evaluation frameworks with consistency
  • Excellent analytical thinking and exceptional attention to detail
  • Comfortable reviewing technical or structured documentation and following written guidelines precisely
  • Previous experience in data annotation, quality assurance, content evaluation, AI model assessment, or similar analytical work is highly preferred
  • Strong written communication skills and the ability to justify evaluation decisions clearly
  • Ability to work independently while maintaining accuracy under tight deadlines

Engagement Details

  • Immediate onboarding with the opportunity to begin work right away
  • Expected commitment of 20-40 hours per week
  • Ability to complete assigned work within 12-24 hours of receiving tasks
  • Requires access to a desktop or laptop computer
  • Project duration may be extended based on business needs and individual performance
  • Weekly payments are processed through supported payment platforms

Contract & Payment Terms

  • Independent contractor engagement
  • Full-time remote with flexible working hours
  • Project duration may be extended, shortened, or concluded based on project requirements and performance
  • Weekly payments are processed through supported payment platforms

Similar jobs