Language QA Analyst | $20/hr Remote

Short-term contract (4 months) – Remote – Flexible 20–40 hrs/week with a minimum 4-hour overlap with PST – $20 per hour

Responsibilities

• Create and evaluate multi-turn prompts based on personal context and experiences.
• Assess AI responses for personalization, accuracy, relevance, and helpfulness.
• Review responses for unsupported assumptions, incorrect personalization, and other quality issues.
• Compare model responses and provide structured evaluations.
• Write clear rationales explaining evaluation decisions and observations.
• Review how relevant information is incorporated into responses.
• Follow project guidelines and maintain appropriate data-handling practices.

Requirements

• Strong Japanese proficiency, including reading and writing.
• Excellent analytical thinking and attention to detail.
• Ability to evaluate nuanced AI-generated responses.
• Strong written communication skills and ability to provide clear feedback.
• Ability to create and evaluate multi-turn prompts.
• Experience in AI evaluation, data annotation, content moderation, or related work is preferred.