Senior Data Annotation Specialist (Dallas)

About The Role

Rex.zone is hiring a Senior Data Annotation Specialist to lead high-quality labeling and evaluation workflows that improve training data quality and large language model evaluation. This is a full-time Remote role aligned to Dallas-area recruiting needs and US-based projects.

What You Will Do

• Execute and review data labeling tasks for NLP, named entity recognition, and document understanding.
• Perform RLHF evaluation and comparative ranking to improve alignment, helpfulness, and instruction-following.
• Conduct prompt evaluation using calibrated rubrics for correctness, policy compliance, and style.
• Run QA evaluation to ensure consistency, low disagreement, and strong training data quality.
• Create and refine annotation guidelines, edge-case policies, and decision trees to maintain compliance.
• Support computer vision annotation (boxes, polygons, keypoints) when needed.
• Label and review content safety data across policy categories with high precision.
• Document dataset issues (noise, ambiguity, distribution shift) and propose fixes to improve model performance.
• Collaborate with AI/ML and data ops teams to improve quality metrics in LLM training pipelines.

Required Qualifications

• Mid-Senior experience in data annotation, data labeling, or AI/ML evaluation workflows.
• Proven ability to follow and enforce detailed annotation guidelines in ambiguous edge cases.
• Experience with training data quality processes (sampling, adjudication, calibration, error taxonomy).
• Exposure to LLM evaluation, RLHF-style ranking, or rubric-based prompt evaluation.
• Strong written reasoning to justify labels, escalations, and QA decisions.
• Comfort working full-time in a Remote environment with throughput and quality targets.

Compensation

Base salary range: $30–$50 per hour.

Location

Remote (US). Recruiting aligned with Dallas, TX.