Red-Teaming QA Lead - Remote

Key Responsibilities

• Quality monitoring: Spot-check red-teaming items, identify quality issues, provide ongoing feedback through DMs, and escalate recurring or critical issues.
• Safety and red-team review: Evaluate adversarial prompts, model responses, risk classifications, safety analyses, policy explanations, and vulnerability reports for accuracy, realism, and usefulness.
• Trainer and QA communication: Update trainers and QAs on Discord about new item guidelines, project changes, workflow updates, quality expectations, and red-teaming-specific review standards.
• Question handling: Respond to trainer/QA questions clearly and promptly, especially around risk categories, adversarial strategy, policy boundaries, edge cases, severity, and rubric interpretation.
• Trainer/QA activation management: DM contributors who are inactive or not working, encourage activation, track follow-ups, and flag availability issues when needed.
• Documentation: Create and maintain red-teaming project documentation, including style guides, trackers, FAQs, quality notes, examples, honeypots, calibration tasks, and onboarding materials.
• Onboarding and training: Schedule and run onboarding/training calls with trainers and QAs to explain project expectations, workflows, rubrics, quality standards, and red-teaming-specific review requirements.
• Quality alignment: Ensure all trainers and QAs apply red-teaming and safety-review guidelines consistently and understand updates as projects evolve.
• Risk review: Flag unsafe, low-quality, unrealistic, policy-inconsistent, or insufficiently documented red-team items.
• Process improvement: Identify recurring quality gaps, propose workflow improvements, and help build scalable QA processes for AI red-teaming projects.

About the role

This hourly, remote contractor role involves overseeing quality, consistency, and trainer performance across AI red-teaming and safety-evaluation projects. You will review AI-generated safety evaluations, adversarial prompts, risk analyses, and trainer/QA work; evaluate output quality against project guidelines; provide precise written feedback; and ensure that all contributors follow the expected quality standards.