Remote | AI/ML Technical Evaluation Consultant -- $60-$90/hour

Key Responsibilities

• Design challenging agentic tasks for AI/ML, data science, data engineering, software, and STEM workflows.
• Write accurate, well-documented solutions as ground truth for evaluation.
• Evaluate AI agent outputs for correctness, efficiency, reasoning quality, and technical rigor.
• Provide detailed written feedback on technical issues and improvement areas.
• Develop and refine evaluation frameworks and rubrics for agentic behavior.
• Collaborate with subject-matter experts for consistency and accuracy.

Ideal Profile

• 3+ years of experience in ML, data science, software engineering, statistics, mathematics, physics, chemistry, biology, materials science, or another STEM field.
• Expertise in programming, data analysis, ML modeling, statistical methods, or computational methods.
• Ability to design/evaluate complex technical tasks with strong judgment.
• Strong written communication skills.
• Prior experience with data annotation, evaluation, or human feedback collection (plus).
• Experience with LLMs, AI systems, agentic workflows (plus).

Nice to Have

• Familiarity with Python, R, SQL, data pipelines, ML workflows, software development, model evaluation.
• Experience developing benchmark tasks, evaluation frameworks, rubrics.
• Familiarity with AI agent behavior, tool use, multi-step reasoning, agentic task execution.
• Comfort working across multiple technical domains.