Sociology QA Lead - Remote

Key Responsibilities

• Quality monitoring: Spot-check sociology items, identify quality issues, provide ongoing feedback through DMs, and escalate recurring or critical issues.
• Sociology review: Evaluate AI-generated sociology explanations, theory applications, research summaries, social policy discussions, demographic interpretations, and reasoning for accuracy and nuance.
• Trainer and QA communication: Update trainers and QAs on Discord about new item guidelines, project changes, workflow updates, quality expectations, and sociology-specific review standards.
• Question handling: Respond to trainer/QA questions clearly and promptly, especially around sociological concepts, social theory, research methods, inequality, culture, ethics, bias, and rubric interpretation.
• Trainer/QA activation management: DM contributors who are inactive or not working, encourage activation, track follow-ups, and flag availability issues when needed.
• Documentation: Create and maintain sociology project documentation, including style guides, trackers, FAQs, quality notes, examples, honeypots, calibration tasks, and onboarding materials.
• Onboarding and training: Schedule and run onboarding/training calls with trainers and QAs to explain project expectations, workflows, rubrics, quality standards, and sociology-specific review requirements.
• Quality alignment: Ensure all trainers and QAs apply sociology review guidelines consistently and understand updates as projects evolve.
• Bias and ethics review: Flag stereotyping, stigmatizing language, unsupported claims about social groups, weak causal reasoning, or ethically problematic social analysis.
• Process improvement: Identify recurring quality gaps, propose workflow improvements, and help build scalable QA processes for sociology AI training projects.

Qualifications

• Bachelor’s, Master’s, or PhD degree in Sociology, Social Sciences, Social Research, Demography, Criminology, Gender Studies, Ethnic Studies, Public Policy, Anthropology, or a closely related field.
• Strong grasp of the English language to follow project guidelines, communicate with teams, and provide clear written feedback.
• 3+ years of experience in sociology research, teaching, social policy analysis, qualitative/quantitative research, academic writing, social science review, community research, or related workflows.
• Strong understanding of sociological theory, social institutions, inequality, stratification, race/ethnicity, gender, class, culture, deviance, organizations, social change, demography, and research methods.
• Ability to evaluate sociology content against detailed rubrics and identify issues such as unsupported generalizations, stereotyping, methodological flaws, weak causal claims, oversimplification, biased framing, or inaccurate use of sociological concepts.
• Familiarity with areas such as survey research, interviews, ethnography, statistical analysis, social policy, urban sociology, family sociology, education, health disparities, work/organizations, or criminology is preferred.
• Experience leading or supporting remote teams of researchers, educators, analysts, reviewers, annotators, or QAs is strongly preferred.
• Comfortable working in fast-moving remote environments using tools such as Discord, Google Sheets, Google Docs, trackers, dashboards, and project management systems.
• Highly detail-oriented and organized, with the ability to maintain style guides, FAQs, trackers, onboarding materials, calibration tasks, and documentation.
• Experience with AI training, data annotation, LLM evaluation, social science QA, academic review, or rubric-based review is a strong plus.