JSON-fluent & Rubrics Expert
Weekday AI (YC W21) · United States · 1 mo ago
RemoteRemoteOTHR$50–$175/hrPart-time
Key Responsibilities
- Review and understand written task descriptions and evaluation objectives
- Interpret detailed grading rubrics and apply them consistently across assignments
- Read, parse, and evaluate JSON-formatted AI model outputs against predefined quality criteria
- Assign accurate, evidence-based scores while maintaining consistency across evaluations
- Identify discrepancies, missing information, or rubric violations during the review process
- Maintain high standards of quality, accuracy, and attention to detail while meeting project turnaround expectations
- Complete all evaluations independently without using AI-assisted tools
Core Qualifications
- Strong ability to read and interpret JSON files and structured data
- Experience applying detailed rubrics, quality standards, or evaluation frameworks with consistency
- Excellent analytical thinking and exceptional attention to detail
- Comfortable reviewing technical or structured documentation and following written guidelines precisely
- Previous experience in data annotation, quality assurance, content evaluation, AI model assessment, or similar analytical work is highly preferred
- Strong written communication skills and the ability to justify evaluation decisions clearly
- Ability to work independently while maintaining accuracy under tight deadlines
Engagement Details
- Immediate onboarding with the opportunity to begin work right away
- Expected commitment of 20-40 hours per week
- Ability to complete assigned work within 12-24 hours of receiving tasks
- Requires access to a desktop or laptop computer
- Project duration may be extended based on business needs and individual performance
- Weekly payments are processed through supported payment platforms
Contract & Payment Terms
- Independent contractor engagement
- Full-time remote with flexible working hours
- Project duration may be extended, shortened, or concluded based on project requirements and performance
- Weekly payments are processed through supported payment platforms