LLM Analyst | $20/hr Remote

About the role

Short-term contract (4 months) for an LLM AI Quality Analyst (Personalization) focused on Japanese language content. This is a remote position with flexible hours (20–40 hrs/week) and requires a minimum 4-hour overlap with PST. Compensation is $20 per hour.

Responsibilities

• Create and evaluate multi-turn prompts based on personal context and experiences.
• Assess AI responses for personalization, accuracy, relevance, and helpfulness.
• Review responses for unsupported assumptions, incorrect personalization, and other quality issues.
• Compare model responses and provide structured evaluations.
• Write clear rationales explaining evaluation decisions and observations.
• Review how relevant information is incorporated into responses.
• Follow project guidelines and maintain appropriate data-handling practices.

Requirements

• Strong Japanese proficiency, including reading and writing.
• Excellent analytical thinking and attention to detail.
• Ability to evaluate nuanced AI-generated responses.
• Strong written communication skills and ability to provide clear feedback.
• Ability to create and evaluate multi-turn prompts.
• Experience in AI evaluation, data annotation, content moderation, or related work is preferred.