Senior Data Annotator Jobs in Austin

About the role

Senior Data Annotator role at Rex.zone supporting AI/ML training data quality across NLP, computer vision, and content safety workflows. This is a US-based remote, full-time role focused on rigorous labeling, RLHF-style evaluation, and QA evaluation to improve downstream model performance.

Responsibilities

• Execute expert-level data labeling across text, image, and multimodal tasks
• Perform RLHF comparisons and preference judgments using rubrics and safety-oriented criteria
• Run QA evaluation and audits to detect systematic errors and guideline drift
• Conduct prompt evaluation for factuality, coherence, instruction following, and safety
• Contribute to named entity recognition and taxonomy/entity mapping
• Support computer vision annotation quality (bounding boxes, segmentation, keypoints)
• Apply content safety labeling policies for sensitive categories
• Document edge cases, propose guideline updates, and maintain gold sets
• Participate in calibration sessions and improve inter-annotator agreement

Requirements

• Experience in data annotation/data labeling with strong accuracy and consistency
• Experience with QA evaluation, auditing, or reviewer responsibilities
• Familiarity with RLHF or large language model evaluation concepts
• Strong written communication for rationales and edge-case documentation
• Comfort working in a remote, metrics-driven production environment
• Ability to handle sensitive content as part of content safety labeling

Work Environment

Remote, US-based. You will collaborate asynchronously with distributed teams supporting AI labs, tech startups, BPOs, and annotation vendors through Rex.zone.

Pay

Competitive hourly rate: $30–$50/hr (base pay).