Portuguese Content Analyst | $15.3/hr Remote

Role Responsibilities

• Design and execute conversational prompts to evaluate personalized AI interactions.
• Assess how effectively AI models utilize personal context to generate relevant and helpful responses.
• Evaluate model responses for grounding, personalization accuracy, integration quality, and overall helpfulness.
• Compare and rank AI-generated responses side-by-side using predefined evaluation criteria.
• Identify hallucinations, flawed inferences, incorrect personalization, and unnatural response behavior.
• Write clear, structured rationales supporting evaluation decisions.
• Review model debug information and verify proper utilization of available data sources.
• Maintain data quality standards and follow project guidelines throughout the evaluation process.

Requirements

• Professional proficiency in reading and writing Portuguese.
• Experience in data annotation, AI quality evaluation, content moderation, content review, or related fields.
• Strong analytical thinking and attention to detail.
• Excellent written communication skills.
• Ability to evaluate nuanced and ambiguous AI responses objectively.
• Comfortable working independently in a remote environment.
• Desktop or laptop with a stable internet connection.
• Full-time availability.

Preferred Qualifications

• Bachelor’s degree or equivalent experience in Linguistics, Journalism, Policy, Law, Ethics, Computer Science, or related fields.
• Experience with AI evaluation, annotation, content review, or localization projects.
• Understanding of personalization concepts and AI-generated content evaluation.
• Experience writing structured feedback and detailed annotations.
• Familiarity with prompt creation and conversational AI systems.