Senior Data Labeling Specialist (Denver)

About the role

Rex.zone is hiring a senior data labeling specialist to lead and execute high-quality training data workflows for AI/ML systems. You will support RLHF, LLM evaluation, QA evaluation, and annotation operations across NLP, computer vision, and content safety labeling to drive measurable model performance improvement.

Responsibilities

• Perform expert data labeling across text, image, and multimodal tasks aligned to production requirements
• Execute RLHF-style preference labeling, rubric-based scoring, and LLM/prompt evaluation
• Apply annotation guidelines compliance checks; resolve ambiguity, edge cases, and failure modes
• Run QA evaluation workflows (spot checks, double-pass review, inter-annotator agreement, calibration)
• Deliver issue reports and propose guideline/rubric updates to reduce label noise and improve consistency
• Support content safety labeling and policy adherence evaluation
• Collaborate with engineering and data operations on sampling, task design, and measurement of quality impact

Requirements

• Mid-senior experience in data labeling/data annotation or LLM evaluation (production or vendor settings)
• Strong understanding of training data quality, taxonomy design, and annotation guidelines compliance
• Experience with QA evaluation methods (audit plans, disagreement analysis, calibration sessions)
• Familiarity with RLHF concepts, preference ranking, and rubric-driven evaluation
• Ability to document decisions clearly and communicate edge cases precisely
• Comfort working remotely in a full-time schedule with measurable output targets

Preferred Qualifications

• Named entity recognition, semantic similarity, summarization evaluation, or retrieval relevance grading
• Computer vision annotation (polygons, bounding boxes, instance segmentation, keypoints) and/or multimodal evaluation
• Content safety labeling (policy mapping, severity ratings, refusals/allowlists)
• Experience supporting AI labs, tech startups, BPOs, or annotation vendors