Generative AI Data Analyst - California, USA (Remote)

Key Responsibilities

• Creatively writing prompts and responses to a variety of diverse topics
• Perform LLM annotation and evaluation tasks (ranking, scoring, labeling, tagging)
• Evaluate model outputs for accuracy, relevance, and instruction-following
• Identify and document issues such as hallucinations and inconsistencies
• Participating in and/or supporting labeling workflows, including hands-on annotation and collaboration with internal or external teams
• Training teams on best practices for creating Large Language Models/Data sets

Requirements

• Hands-on experience performing data annotation or evaluation tasks (e.g., labeling, ranking, scoring, or tagging LLM outputs)
• Native or near-native English with excellent writing skills
• Strong attention to detail and ability to follow guidelines consistently
• Self-driven, motivated and enthusiastic to work on state-of-art machine learning tools
• 4 year Accredited College degree or equivalent experience