Expert Contributor - Literature (Task-Based)

About the role

Snorkel AI is a data development company spun out of the Stanford AI Lab, focused on improving the truthfulness and reasoning abilities of advanced AI systems. We are recruiting literary scholars and social scientists to serve as Expert Contributors on a project involving the development and evaluation of advanced AI systems. Contributors apply their disciplinary expertise to help improve how AI models engage with complex literary, cultural, linguistic, and social-scientific material. We are particularly excited to expand participation from scholars in literature, sociology, and anthropology, whose perspectives are often underrepresented in AI development. This is a rare opportunity to apply your expertise directly to AI systems—no prior AI experience required.

This is project/task-based work completed at your own pace. Tasks are designed to reflect genuine disciplinary complexity and require real scholarly expertise to execute well.

Responsibilities

• Craft graduate-level, exam-style question-and-answer sets that will serve as LLM benchmarks. For example: constructing a prompt that presents passages from two novels of different national literary traditions (e.g., a French naturalist text and a Latin American magical realist text) alongside competing critical frameworks, asks the model to identify the formal and thematic conventions specific to each tradition, analyze how each text engages with or subverts those conventions, situate the comparison within broader debates in comparative literature, and produce a defensible synthetic argument—complete with a model answer and a clear rubric for evaluating responses.
• Develop scenario-driven prompts that reflect authentic disciplinary reasoning. For example: providing an unmarked literary passage from a German, Italian, or Iberian author alongside ethnographic field notes or sociological case data addressing a related cultural phenomenon, then asking the model to perform a close reading of the literary text, evaluate how sociological or anthropological frameworks illuminate or complicate the literary interpretation, and construct an interdisciplinary argument that could withstand peer review.
• Create a structured checklist of specific criteria to evaluate a model's response—covering literary-historical accuracy, sensitivity to linguistic and cultural context (e.g., recognizing tradition-specific narrative conventions in French, Germanic, Iberian, or Latin American literatures), quality of close reading, engagement with relevant sociological or anthropological theory, logical coherence, and completeness of argument.
• Write grading rubrics and scoring guides for the questions and model answers you have authored, specifying what a full-credit, partial-credit, and failing response looks like at each stage of reasoning.

Requirements

• Active or practicing scholars, researchers, or educators with deep expertise in one or more of the following: French Literature, German or Germanic Language Literatures, Italian Literature, Latin American Literature, Iberian Literature, Sociology, or Anthropology.
• Ability to construct intellectually rigorous, multi-step questions with defensible model answers and objectively gradable rubrics.
• Familiarity with scholarly argumentation, close reading and textual analysis, primary source interpretation, and the critical and interpretive conventions of your field.
• Comfortable working independently with loosely defined deliverables and self-managing task completion.

Pay

Snorkel pays you per accepted task rather than an hourly rate because we believe our experts should be in control of their earnings. Accepted task payments for this modality can range from $50 to $300 with an average handling time of 60–180 minutes. Rather than locking experts into an hourly rate, our pay philosophy seeks to reward experts for their expertise: higher productivity and better quality can earn more money, rather than making the same amount on an hourly basis for someone less productive or less qualified than you.

Schedule

Work independently from anywhere, with no hours commitment. This is a 1099 contractor role (must be eligible) with project-based engagements on an as-needed basis. Start date is ASAP.

Applicants must successfully complete an assessment as part of the selection process. Note: Cannot reside in California.