Medicine QA Lead - Remote

Key Responsibilities

• Spot-check medical items, identify quality issues, provide ongoing feedback through DMs, and escalate recurring or critical issues.
• Evaluate AI-generated medical explanations, clinical reasoning, patient-facing responses, case analyses, diagnostic discussions, treatment summaries, medication-related content, and health-education materials for accuracy, clarity, and appropriate caution.
• Update trainers and QAs on Discord about new item guidelines, project changes, workflow updates, quality expectations, and medical-review-specific standards.
• Respond to trainer/QA questions clearly and promptly, especially around clinical reasoning, terminology, patient-safety risks, medical claims, guideline interpretation, evidence quality, and rubric application.
• DM contributors who are inactive or not working, encourage activation, track follow-ups, and flag availability issues when needed.
• Create and maintain medical project documentation, including style guides, trackers, FAQs, quality notes, examples, honeypots, calibration tasks, and onboarding materials.
• Schedule and run onboarding/training calls with trainers and QAs to explain project expectations, workflows, rubrics, quality standards, and medicine-specific review requirements.
• Ensure all trainers and QAs apply medical-review guidelines consistently and understand updates as projects evolve.
• Flag unsafe, misleading, overconfident, or clinically inappropriate medical outputs, especially where the content could be interpreted as personalized diagnosis, treatment, emergency guidance, medication instructions, or professional medical advice.
• Identify recurring quality gaps, propose workflow improvements, and help build scalable QA processes for medical AI training projects.

Requirements

• Medical degree such as MD, DO, MBBS, MBChB, or equivalent; advanced clinical, biomedical, nursing, pharmacy, or healthcare-related degrees may be considered depending on project requirements.
• Strong grasp of the English language to follow project guidelines, communicate with teams, and provide clear medical-review feedback in English.
• 3+ years of professional experience in medicine, clinical practice, medical research, medical education, clinical documentation, healthcare QA, medical writing, guideline review, or related workflows.
• Strong understanding of core medical topics such as clinical reasoning, differential diagnosis, pathophysiology, pharmacology, diagnostics, treatment principles, patient safety, evidence-based medicine, medical terminology, and healthcare communication.
• Ability to evaluate medical content against detailed rubrics and identify issues such as unsafe recommendations, hallucinated facts, missing caveats, incorrect clinical reasoning, overconfident diagnosis/treatment claims, inappropriate patient advice, or incomplete explanations.
• Familiarity with medical workflows or references such as clinical guidelines, diagnostic pathways, medication safety, chart review, case summaries, patient education materials, medical literature, and evidence-based review is preferred.
• Experience leading or supporting remote teams of trainers, annotators, reviewers, clinicians, medical writers, researchers, educators, or QAs is strongly preferred.
• Comfortable working in fast-moving remote environments using tools such as Discord, Google Sheets, Google Docs, trackers, dashboards, and project management systems.
• Highly detail-oriented and organized, with the ability to maintain style guides, FAQs, trackers, onboarding materials, honeypots, calibration tasks, and other quality documentation.
• Experience with AI training, data annotation, large language models, prompt/response evaluation, medical content QA, or rubric-based LLM evaluation is a strong plus.