Data Science Expert – AI Evaluation Project

Key Responsibilities

• Design detailed evaluation criteria for real-world data science deliverables, including:
• Data analysis reports
• Predictive models
• Dashboards and visualizations
• Experiment results and A/B testing analyses
• Business recommendations and technical documentation
• Evaluate AI-generated and human-produced work using structured scoring frameworks
• Provide detailed written justifications for evaluation decisions
• Apply consistent, evidence-based judgment to ensure reliable and reproducible assessments
• Review analytical approaches, statistical methodologies, and business insights for quality and accuracy
• Incorporate reviewer feedback and continuously refine evaluation standards
• Collaborate with project teams to improve AI performance in data science tasks

Required Qualifications

• 5+ years of professional experience in Data Science or Analytics
• Experience in product, business operations, growth analytics, or related functions at leading technology companies
• Strong expertise in:
• SQL and Python
• Experiment design and A/B testing
• Statistical analysis and hypothesis testing
• Metric definition and KPI development
• Data visualization and dashboard design
• Ability to communicate technical findings effectively to business and executive stakeholders
• Excellent analytical thinking and written communication skills
• High attention to detail and consistency in evaluation

Preferred Qualifications

• Experience evaluating technical work or mentoring data science teams
• Familiarity with AI model evaluation, reinforcement learning, or human-feedback projects
• Experience working with large-scale datasets and business intelligence tools
• Knowledge of machine learning workflows and model performance evaluation
• Passion for advancing AI through expert human feedback