Senior Data Labeling Specialist (Detroit)

Rex.zone is hiring a mid-senior contributor to lead high-impact data labeling and evaluation workflows that improve training data quality for modern AI systems. This role supports LLM training pipelines, RLHF-style preference/ranking tasks, and prompt evaluation while operating remotely with Detroit-aligned search intent.

Responsibilities

• Lead complex data labeling workflows for LLM evaluation, including prompt evaluation and RLHF evaluation
• Create, maintain, and iterate annotation guidelines, taxonomies, and edge-case rules
• Run QA evaluation loops using sampling plans, inter-annotator agreement checks, and targeted audits
• Triage ambiguity and coordinate adjudication to maintain consistent ground truth
• Monitor training data quality signals and provide feedback tied to model performance improvement
• Support dataset curation, coverage analysis, and reproducible documentation for compliance

Requirements

• Mid-senior experience in data labeling / annotation operations including review and adjudication
• Hands-on knowledge of QA methods (sampling, error taxonomies, agreement metrics)
• Ability to write clear, enforceable annotation guidelines with strong reasoning
• Experience in at least one domain: NLP labeling, NER, computer vision annotation, or content safety labeling
• Strong written communication for guideline clarity and audit trails

Preferred Qualifications

• Exposure to RLHF, preference ranking, and/or prompt evaluation
• Dataset auditing for training data quality, bias/error analysis
• Experience scaling review operations with vendors/BPO workflows

Work Arrangement

Remote, full-time role open to qualified candidates in the US.