Data Analyst | $15/hr Remote

Role Responsibilities

• Evaluate a personalization feature for GeminiDesign
• Execute multi-turn conversational prompts requiring the AI to utilize personal information and experiences
• Assess how effectively the model uses past conversations and activity to generate relevant and helpful responses
• Evaluate model responses based on intent and appropriate personalization
• Analyze responses for grounding issues, including flawed inferences or hallucinations
• Assess integration quality to ensure personal data is incorporated naturally into responses
• Perform side-by-side evaluations and stack-rank model responses based on helpfulness and naturalness
• Write clear rationales referencing specific conversation turns
• Extract and verify debug information to confirm correct use of summaries and data sources

Requirements

• Experience in data annotation, AI quality evaluation, content moderation, or related roles
• Strong Japanese proficiency (reading and writing)
• Willingness to use a primary personal Google account and enable personal data sources for assessment
• Strong analytical thinking and attention to detail
• Experience with creative prompt engineering and personalization concepts
• Ability to provide structured feedback and clear written explanations
• Ability to work independently in a remote environment
• Desktop or laptop with a stable internet connection