Data Annotator

About Mind Moves

Mind Moves is a women-owned Washington, D.C.-based firm that helps government and business partners navigate digital transformation using "human-in-the-loop" AI. Our expert team has delivered responsibly developed AI products that drive millions in impact across agencies like the National Institutes of Health (NIH).

About the Role

Mind Moves is seeking 20+ independent Subject Matter Experts (SMEs) to develop a benchmark evaluation dataset for AI systems operating over the biomedical literature. As these systems become integrated into research and clinical workflows, rigorous assessment of their safety, factual accuracy, and scientific reliability requires expert-authored ground truth. SMEs will independently create authentic biomedical information needs and reference responses grounded exclusively in evidence from scientific literature resources PubMed and PubMed Central (PMC), following standardized annotation guidelines. The resulting benchmark will provide a transparent, reproducible resource for evaluating AI systems’ ability to retrieve, synthesize, and accurately communicate biomedical evidence.

Please note that accepting work on this contract makes you ineligible to participate in the Challenges competition.

Contract Engagement

This is a project-based, hourly contract engagement supporting a defined research project in AI-powered health. This is a short-term contract assignment and is not intended to lead to full-time regular employment.

Schedule

Project Duration: September–November 2026

Expected Commitment: Approximately 10-20 hours per week, with a flexible remote schedule.

Pay

Compensation: $60 per hour.

Expert Profiles Sought

We are contracting against two distinct expert profiles:

• Profile 1—Academic and Bench Researchers: PhD candidates (post-candidacy, actively engaged in dissertation research) and early-stage postdoctoral researchers with current, deep command of their literature. Sub-domains may include, but are not limited to, molecular genetics, structural biology, immunology, oncology, multi-omics, and computational biology.
• Profile 2—Research-Savvy Clinicians and Healthcare Professionals: Medical, pharmacy, nursing, and allied health students, along with early-career clinicians and healthcare professionals, who regularly interpret primary biomedical literature for research or evidence-based clinical practice. Specialties sought include, but are not limited to, cardiology, oncology, neurology, infectious disease, pharmacology, and subgroup-specific therapeutics.

Responsibilities

• Develop authentic biomedical information needs and reference responses grounded exclusively in evidence from scientific literature resources PubMed and PubMed Central (PMC).
• Follow standardized annotation guidelines to create a transparent, reproducible benchmark for evaluating AI systems.
• Ensure all deliverables are entirely human-authored, with no use of large language models (e.g., ChatGPT, Claude, Gemini, etc.).
• Run literature searches as decomposed conceptual queries (e.g., gene aliases, drug targets, mechanistic keyword pairs) rather than submitting a topic's full question string verbatim.
• Submit a short certification of human origination with each deliverable.

Requirements

• Must be based in the U.S. and work remotely.
• Must adhere to the human origination requirement; no use of AI tools for drafting, rephrasing, summarizing, or brainstorming.
• Use of search engines, corpus keywords, retrieval filters, reference managers (e.g., EndNote, Zotero), and consultation with medical librarians or colleagues is permitted.