Remote Data Labeling Specialist (United States)

Role Overview

As a Remote Data Labeling Specialist, you will label and evaluate multi-modal AI training data (text, images, and conversations) to improve model performance in production systems. You will follow detailed rubrics, maintain strong annotation guidelines compliance, and contribute to measurable training data quality improvements.

What You Will Do

• Execute data labeling and data annotation for LLM training pipelines (classification, extraction, ranking)
• Perform RLHF workflows including preference comparisons, rationale tagging, and rubric-based scoring
• Conduct LLM evaluation and prompt evaluation for helpfulness, factuality, grounding, relevance, and instruction-following
• Apply named entity recognition and span labeling with strict rubric adherence
• Support computer vision annotation (bounding boxes, polygons, keypoints, image categorization) when needed
• Run QA evaluation audits, inter-annotator agreement checks, and targeted reviews to ensure training data quality
• Perform content safety labeling and policy-based moderation tagging
• Document edge cases, propose taxonomy updates, and escalate ambiguous items with evidence

Required Qualifications

• Experience with production data labeling or related QA evaluation workflows
• Comfort working with detailed taxonomies, rubrics, and policy language
• Ability to maintain throughput while protecting training data quality
• Clear asynchronous communication in a fully remote environment

Remote Work Details

This is a full-time Remote role within the United States, with calibration sessions and QA checkpoints handled online. You must follow confidentiality and secure dataset handling requirements.