AI Quality Analyst (Chinese) | $18/hr Remote

Role Responsibilities

• Design and execute creative multi-turn conversational prompts based on personal context and experiences.
• Evaluate AI responses for personalization quality, relevance, helpfulness, and alignment with the user's intent.
• Analyze responses for Grounding, identifying unsupported claims, incorrect personalization, flawed inferences, and hallucinations.
• Assess Integration quality to ensure personal information is incorporated naturally without unnecessary overnarration.

Requirements

• 1+ year of relevant experience in data annotation, AI quality evaluation, content moderation, or a related analytical role.
• Chinese proficiency, with the ability to read and write Chinese at a high level, as Chinese is the focus language for the project.
• Exceptional analytical thinking and ability to evaluate nuanced and ambiguous AI responses.
• Experience or demonstrated ability in creative prompt engineering and multi-turn conversational testing.
• Strong understanding of personalization concepts, including identifying incorrect personalization, poor inferences, and forced connections.