Quality Analyst - Portuguese | $15/hr Remote

Role Responsibilities

• Design and execute conversational prompts to evaluate personalized AI interactions
• Assess how effectively AI models utilize personal context to generate relevant and helpful responses
• Evaluate model responses for grounding, personalization accuracy, integration quality, and overall helpfulness
• Compare and rank AI-generated responses side-by-side using predefined evaluation criteria
• Identify hallucinations, flawed inferences, incorrect personalization, and unnatural response behavior
• Write clear, structured rationales supporting evaluation decisions
• Review model debug information and verify proper utilization of available data sources
• Maintain data quality standards and follow project guidelines throughout the evaluation process

Requirements

• Professional proficiency in reading and writing Portuguese
• Experience in data annotation, AI quality evaluation, content moderation, content review, or related fields
• Strong analytical thinking and attention to detail
• Excellent written communication skills
• Ability to evaluate nuanced and ambiguous AI responses objectively
• Comfortable working independently in a remote environment
• Desktop or laptop with a stable internet connection
• Full-time availability

Preferred Qualifications

• Bachelor's degree or equivalent experience in Linguistics, Journalism, Policy, Law, Ethics, Computer Science, or related fields
• Experience with AI evaluation, annotation, content review, or localization projects
• Understanding of personalization concepts and AI-generated content evaluation
• Experience writing structured feedback and detailed annotations
• Familiarity with prompt creation and conversational AI systems