Biomedical Data Annotator

About the role

Mind Moves is seeking 20+ independent Subject Matter Experts (SMEs) to develop a benchmark evaluation dataset for AI systems operating over the biomedical literature. SMEs will independently create authentic biomedical information needs and reference responses grounded exclusively in evidence from PubMed and PubMed Central (PMC), following standardized annotation guidelines. The resulting benchmark will provide a transparent, reproducible resource for evaluating AI systems’ ability to retrieve, synthesize, and accurately communicate biomedical evidence. This work supports the National Library of Medicine (NLM) at the National Institutes of Health (NIH) and contributes to building trustworthy AI systems.

Responsibilities

• Develop biomedical question‑answer pairs for AI benchmark datasets.
• Formulate authentic information needs and craft reference answers using only PubMed/PMC literature.
• Follow standardized annotation guidelines to ensure consistency and reproducibility.
• Conduct literature searches using decomposed conceptual queries (gene aliases, drug targets, mechanistic keyword pairs) rather than verbatim question strings.
• Certify that all deliverables are entirely human authored, without use of any large language model.
• Participate in a short hiring exercise and a 20‑30 minute interview as part of the selection process.

Qualifications

• Profile 1: PhD candidates (post‑candidacy, actively engaged in dissertation research) or early‑stage postdoctoral researchers with deep command of their literature in sub‑domains such as molecular genetics, structural biology, immunology, oncology, multi‑omics, computational biology, or related fields.
• Profile 2: Medical, pharmacy, nursing, or allied health students; early‑career clinicians and healthcare professionals who regularly interpret primary biomedical literature for research or evidence‑based practice, with specialties including cardiology, oncology, neurology, infectious disease, pharmacology, subgroup‑specific therapeutics, etc.
• Ability to work remotely and commit approximately 10‑20 hours per week during the September–November 2026 project period.
• Familiarity with PubMed/PMC and reference managers (EndNote, Zotero) is expected.

Skills

• Expertise in searching biomedical literature using conceptual queries and keyword strategies.
• Ability to synthesize complex scientific information into clear, evidence‑based question‑answer formats.
• Strong attention to detail and adherence to annotation guidelines.
• Capacity to certify human origination of work and avoid any LLM assistance.
• Effective communication and collaboration in a remote, flexible‑hour environment.

Pay

$30 per hour.

Schedule

Project duration: September–November 2026; expected commitment: approximately 10‑20 hours per week with a flexible remote schedule.