Mathematics QA Lead - Remote

About the role

This is an hourly, remote contractor role as a Mathematics Quality Assurance Lead. The role involves overseeing quality, consistency, and trainer performance across mathematics AI training projects. Responsibilities include reviewing AI-generated math content and trainer/QA work, evaluating output quality against project guidelines, providing precise written feedback, and ensuring contributors adhere to quality standards.

Responsibilities

• Spot-check mathematics items, identify quality issues, provide ongoing feedback through DMs, and escalate recurring or critical issues.
• Evaluate AI-generated math explanations, proofs, derivations, calculations, word-problem solutions, diagrams/descriptions, and step-by-step reasoning for correctness and clarity.
• Update trainers and QAs on Discord about new item guidelines, project changes, workflow updates, quality expectations, and math-specific review standards.
• Respond to trainer/QA questions clearly and promptly, especially around reasoning validity, notation, assumptions, solution methods, proof structure, formatting, and rubric interpretation.
• DM contributors who are inactive or not working, encourage activation, track follow-ups, and flag availability issues when needed.
• Create and maintain mathematics project documentation, including style guides, trackers, FAQs, quality notes, examples, honeypots, calibration tasks, and onboarding materials.
• Schedule and run onboarding/training calls with trainers and QAs to explain project expectations, workflows, rubrics, quality standards, and mathematics-specific review requirements.
• Ensure all trainers and QAs apply mathematics guidelines consistently and understand updates as projects evolve.
• Identify recurring issues such as skipped reasoning steps, invalid simplifications, wrong formulas, notation inconsistencies, arithmetic mistakes, or answers that are correct but poorly justified.
• Propose workflow improvements and help build scalable QA processes for mathematics AI training projects.

Requirements

• Bachelor’s, Master’s, or PhD degree in Mathematics, Applied Mathematics, Statistics, Physics, Engineering, Computer Science, Mathematics Education, or a closely related quantitative field.
• Strong grasp of the English language to follow project guidelines, communicate with teams, and provide clear mathematical feedback in English.
• 3+ years of professional experience in mathematics, teaching, tutoring, research, quantitative analysis, technical writing, curriculum development, problem creation, assessment design, or math-content review.
• Strong understanding of core mathematics topics such as algebra, geometry, trigonometry, calculus, linear algebra, discrete mathematics, probability, statistics, number theory, combinatorics, differential equations, and mathematical proofs.
• Ability to evaluate math content against detailed rubrics and identify issues such as incorrect assumptions, flawed reasoning, invalid proofs, calculation errors, notation problems, missing steps, hallucinated facts, or incomplete explanations.
• Familiarity with mathematical tools or workflows such as LaTeX, Python, MATLAB, R, WolframAlpha/Mathematica, GeoGebra, Desmos, spreadsheet modeling, or symbolic computation tools is preferred.
• Experience leading or supporting remote teams of trainers, annotators, reviewers, educators, technical writers, or QAs is strongly preferred.
• Comfortable working in fast-moving remote environments using tools such as Discord, Google Sheets, Google Docs, trackers, dashboards, and project management systems.
• Highly detail-oriented and organized, with the ability to maintain style guides, FAQs, trackers, onboarding materials, honeypots, calibration tasks, and other quality documentation.
• Experience with AI training, data annotation, large language models, prompt/response evaluation, mathematical content QA, or rubric-based LLM evaluation is a strong plus.