Medical AI Evaluator (Remote, Hourly Contractor)

Position Summary

In this remote, hourly contractor role, you will evaluate AI-generated medical content and develop cases that test clinical reasoning accuracy. Your work directly improves how leading AI models handle healthcare information, making them more accurate, reliable, and safe.

• Evaluating AI-generated medical responses for clinical accuracy, reasoning quality, and patient safety implications
• Identifying errors in clinical methodology, unsafe assumptions, missing contraindications, and misinterpretation of diagnostic data
• Writing clear, precise feedback explaining corrections and reasoning gaps
• Developing prompts and test cases that probe AI accuracy across clinical scenarios
• Rating and comparing AI responses based on correctness, internal consistency, and contextual appropriateness
• Fact-checking medical content against reliable sources using consistent reasoning

Profile Requirements

• Bachelor's degree or higher in Medicine (MD/DO), Nursing, Public Health, Health Sciences, or Allied Health
• 5+ years of professional experience in a relevant healthcare discipline
• Strong clinical reasoning skills: differential diagnosis, risk stratification, red-flag recognition
• Solid grounding in disease processes, patient care, public health principles, and medical terminology
• Full professional English proficiency
• Exceptional attention to detail and ability to explain corrections clearly in writing
• Reliable and self-directed, with consistent output quality in a remote, asynchronous workflow

Preferred Experience

• Experience in clinical documentation review, utilization review, or healthcare editorial QA
• Prior experience with AI data training or annotation

About CNTXT AI

CNTXT AI builds artificial intelligence products and data solutions with a focus on making AI accurate, safe, and globally relevant for impact. Our work spans data services, custom AI solutions, and proprietary AI products, with deep expertise in Arabic-native and secure, sovereign solutions.