Senior Data Annotator Jobs in Phoenix

Senior Data Annotator (Remote, Full-Time) — Phoenix (US) Senior data annotator jobs in Phoenix focus on producing and auditing high-quality labeled datasets for AI/ML training workflows, including RLHF, LLM evaluation, prompt evaluation, and QA evaluation. You will collaborate with engineers and AI teams to resolve ambiguity, document edge-case decisions, and improve training data quality and downstream model performance. What You Will Do Perform senior-level data labeling across NLP, LLM evaluation, and computer vision annotation tasksExecute RLHF workflows such as preference ranking, rubric-based scoring, and pairwise comparisonsConduct QA evaluation: spot checks, gold-set validation, and inter-annotator agreement reviewsApply annotation guidelines compliance and document edge-case decisions for repeatabilityRun error analysis to identify systematic labeling issues impacting training data qualityProvide feedback to improve taxonomy, rubrics, and evaluation criteriaPartner with engineering/data ops on task instructions, sampling, and dataset versioningSupport content safety labeling aligned to policy and risk categories Required Qualifications Mid-Senior experience in data annotation, data labeling, or AI evaluation with strong accuracy and consistencyStrong reading comprehension and attention to detail for rubric-based scoring and policy applicationFamiliarity with RLHF task formats, LLM evaluation concepts, and training data quality practicesAbility to communicate edge cases clearly and improve annotation guidelines complianceComfort working remotely with distributed teams on a full-time schedule Preferred Qualifications Named entity recognition, intent classification, and prompt evaluation for chat-based systemsComputer vision annotation (bounding boxes, polygons, segmentation, attribute tagging)Content safety labeling, policy taxonomy design, and red-teaming style evaluationsQA metrics familiarity (e.g., inter-annotator agreement) and calibration sessionsDataset lifecycle practices: versioning, audit trails, release notes Compensation Hourly base pay range: $30–$50 per hour (USD), depending on evaluation complexity, domain expertise, and QA leadership. How To Apply Apply through Rex.zone and include a brief summary of relevant experience in RLHF, LLM evaluation, data labeling, QA evaluation, and examples of training data quality improvements.