Python QA Lead - Remote

About the role

This is a remote, hourly contractor position overseeing quality, consistency, and trainer performance across Python AI training projects. The role involves reviewing AI-generated Python code and trainer/QA work, evaluating output quality against project guidelines, providing precise written feedback, and ensuring contributors follow expected quality standards.

Responsibilities

• Spot-check Python items, identify quality issues, provide feedback through DMs, and escalate recurring or critical issues.
• Review AI-generated Python code, debugging responses, algorithms, backend snippets, automation scripts, tests, and explanations.
• Update trainers/QAs on Discord about guidelines, workflow updates, and Python-specific quality expectations.
• Respond to questions around Python syntax, runtime behavior, exceptions, package usage, testing, typing, security, performance, and rubric interpretation.
• DM inactive contributors, encourage activation, track follow-ups, and flag availability issues.
• Create and maintain Python documentation, style guides, trackers, FAQs, examples, honeypots, and onboarding materials.
• Run onboarding/training calls for Python contributors.
• Flag misleading, non-executable, insecure, inefficient, or non-production-ready Python recommendations.
• Identify recurring quality gaps and improve Python QA workflows.

Requirements

• Bachelor’s or Master’s degree in Computer Science, Software Engineering, Data Science, Information Technology, or equivalent professional software engineering experience.
• Strong grasp of the English language to follow guidelines, communicate with teams, and provide clear technical feedback.
• 3+ years of professional experience in Python development, backend engineering, automation, scripting, data workflows, code review, QA, teaching, or technical mentoring.
• Strong understanding of Python fundamentals such as data structures, functions, classes, modules, exceptions, comprehensions, iterators, generators, decorators, context managers, virtual environments, packaging, and testing.
• Ability to evaluate Python content against rubrics and identify issues such as incorrect logic, non-executable code, poor exception handling, inefficient algorithms, unsafe file/network operations, hallucinated APIs, or incomplete explanations.
• Familiarity with pytest, unittest, typing, mypy, pip, poetry, virtualenv, FastAPI, Flask, Django, requests, asyncio, pandas, SQLAlchemy, GitHub, Docker, and CI/CD is preferred.
• Experience leading or supporting remote teams of trainers, engineers, reviewers, annotators, educators, or QAs is strongly preferred.
• Comfortable with Discord, Google Sheets, Google Docs, trackers, dashboards, GitHub, and project management systems.
• Highly organized and able to maintain style guides, FAQs, trackers, onboarding materials, honeypots, calibration tasks, and documentation.
• Experience with AI training, data annotation, LLM evaluation, code QA, or rubric-based code review is a strong plus.

Qualifications

• Master’s or PhD in Computer Science, Software Engineering, Data Science, Information Technology, or related field.
• Expert-level proficiency in Python, including advanced topics like concurrency, distributed systems, and machine learning frameworks.
• Proven track record of improving code quality and reducing bugs in large-scale Python applications.
• Experience with automated testing tools and continuous integration pipelines.
• Excellent communication skills, both written and verbal, to effectively collaborate with cross-functional teams.
• Strong problem-solving skills and ability to work independently and as part of a team.