Applied Health & Medicine Benchmark Specialist

About The Role

We are seeking medical and health science professionals to create and review rigorous academic assessment content for an AI research initiative. You will develop and evaluate challenging multiple-choice questions across health and medicine, assess solution quality, and contribute to high-quality benchmarks used to advance AI capabilities. Depending on your assignment, you will work on question authoring or question verification.

Responsibilities

• Question Authoring
  • Create original, challenging questions that assess deep conceptual understanding rather than surface-level recall.
  • Ensure each question is unambiguous, self-contained, precisely defined, and includes all information necessary to solve it.
  • Rate question difficulty as:
    • Medium: Introductory undergraduate
    • Hard: Advanced undergraduate
    • Expert: Postgraduate level and above
  • Provide 1 correct answer and 9 plausible but subtly incorrect alternatives designed to challenge advanced solvers.
  • Develop clear, concise step-by-step reasoning in markdown format.
  • Provide 1–5 reputable academic references per question, including peer-reviewed research and clinical guidelines.
• Question Verification
  • Review existing questions for medical accuracy, clarity, completeness, precision, and solvability.
  • Identify potential issues or ambiguities.
  • Make necessary edits to improve question quality.
  • Rate difficulty using the defined assessment levels.
  • Document and justify changes made.

Health & Medicine Domains

Assignments may cover:

• Clinical Medicine & Surgery
• Medical Imaging & Diagnostics
• Pharmacovigilance
• Healthcare Management & Economics
• Rehabilitation and Allied Health

Qualifications

• MD, DO, PhD, or doctoral candidate in Medicine, Biomedical Sciences, Public Health, or a closely related field.
• Master's degree may be considered for candidates with exceptional expertise in a specific subdomain.
• Strong graduate-level knowledge of medicine and health sciences.
• Strong clinical reasoning and biomedical research methodology skills.
• Board certification, clinical experience, or health-related research publications are advantageous.
• Excellent written English with the ability to communicate complex concepts clearly and concisely.

Work Arrangement

• 10+ hours per week expected
• Fully remote
• Asynchronous schedule
• Independent contractor engagement

Contract & Payment Terms

• Projects may be extended, shortened, or concluded early based on project needs and performance.
• Work will not involve access to confidential or proprietary information from any employer, client, or institution.
• Payments are made weekly through Stripe or Wise based on services rendered.
• H1-B and STEM OPT candidates cannot be supported at this time.