AI Safety Expert (remote)

Evaluating AI-generated Conversations

• Evaluate quality, safety, and usefulness of AI-generated content
• Rate model outputs using structured evaluation rubrics and project guidelines

Annotating Conversational Data

• Support AI training and benchmarking by annotating conversational data

Identifying Risks and Improvements

• Identify emerging risks, behavioral patterns, and areas for model improvement
• Provide feedback to help improve model performance

Confidentiality and Collaboration

• Maintain strict confidentiality while working with proprietary AI systems and sensitive content
• Participate in calibration sessions and quality reviews to ensure consistent evaluations

Qualities We're Looking For

• Curious, analytical, thoughtful communicators
• Enjoy solving complex problems and exercising sound judgment
• Comfortable evaluating nuanced situations, following detailed guidelines, and contributing to trustworthy AI development

Basic Qualifications

• Professional experience or subject matter expertise in psychology, behavioral science, social work, trust & safety, research, or a related discipline
• Strong written communication skills with excellent attention to detail
• Comfortable learning structured evaluation frameworks and applying them consistently
• Strong critical thinking and problem-solving skills
• High ethical standards and sound judgment when working with sensitive or ambiguous content
• Comfortable using AI tools, Google Workspace, Slack, and other web-based collaboration platforms
• Willingness to sign NDAs and work on confidential projects