Remote Data Annotation Specialist (Los Angeles)

What You Will Do

• Perform data labeling for NLP, computer vision annotation, and multimodal (text-image) tasks
• Execute RLHF-style ranking and preference judgments to improve model alignment
• Run LLM evaluation tasks (rubric-based scoring, prompt-response grading, error categorization)
• Apply content safety labeling policies (toxicity, self-harm, harassment, sensitive attributes)
• Conduct QA evaluation and audits; track issue/disagreement rates and improve consistency
• Follow detailed annotation guidelines, document edge cases, and escalate ambiguous cases

Workflow and Tools

• Work in web-based annotation platforms using task instructions, gold-standard examples, and calibration sets
• Maintain review notes and support continuous improvement of training data quality

Required Qualifications

• Mid-Senior experience in data annotation, data labeling, QA evaluation, or model evaluation
• Strong attention to detail and consistent guideline-based decision-making
• Comfort working with ambiguity; strong written communication for documenting edge cases
• Reliable remote work habits and time management

Preferred Qualifications

• RLHF / preference ranking experience
• LLM evaluation and prompt evaluation experience
• Named entity recognition (NER) or taxonomy-based labeling
• Computer vision annotation (bounding boxes, segmentation, keypoints)
• Quality programs: auditing, calibration sessions, disagreement analysis

Why Rex.zone

Contribute to measurable improvements in training data quality and model evaluation outcomes across domains including NLP, computer vision, LLM training, and content safety in a remote-first environment.

How To Apply

Apply via Rex.zone with a resume highlighting annotation experience, QA evaluation work, and any RLHF, LLM evaluation, NER, computer vision annotation, or content safety labeling projects.