Remote Data Annotation Specialist (San Diego)

Role Overview

Rex.zone is hiring a Remote Data Annotation Specialist to create and evaluate labeled datasets that train and validate modern AI/ML systems. You will work across NLP labeling, computer vision annotation, RLHF preference ranking, and prompt-response evaluation while following versioned guidelines and meeting clear quality targets.

What You Will Do

• Execute data labeling tasks across NLP and computer vision annotation projects (text span labeling, classification, bounding boxes, segmentation, attribute tagging).
• Perform RLHF and large language model evaluation by ranking outputs, scoring prompt-response pairs, and identifying instruction-following failures.
• Run QA evaluation workflows including self-checks, peer reviews, and gold-standard audits to improve training data quality and consistency.
• Apply content safety labeling taxonomies and escalate uncertain cases using defined escalation paths.
• Write clear rationales, document edge cases, and propose guideline updates when recurring ambiguities appear.
• Track throughput and quality metrics (accuracy, agreement rates, defect density) to support model performance improvement.

Required Qualifications

• Mid-Senior experience in data annotation, data labeling, QA evaluation, or LLM evaluation in production or vendor environments.
• Strong reading comprehension and attention to detail for rubric-based judgment and policy-based labeling.
• Comfort with annotation tools, task queues, and strict annotation guidelines compliance.
• Ability to explain decisions with concise written rationales.
• Familiarity with NLP concepts such as named entity recognition and taxonomy-driven labeling (preferred).

Work Arrangement

This is a Remote, FULL_TIME role aligned to US-based projects. Collaboration is primarily asynchronous with periodic calibration and QA review sessions. Location reference: San Diego, CA, US.