Remote Data Annotator Jobs in New York

About The Role

As a Remote Data Annotator supporting New York-based programs, you will label and evaluate datasets used to train and validate machine learning models. Your work improves training data quality, supports RLHF feedback loops, and drives model performance improvement across NLP, computer vision, and LLM workflows.

Core Workflows

• Produce high-accuracy labels for text, images, and multimodal inputs
• Perform RLHF-style preference ranking and prompt evaluation for LLMs
• Complete QA evaluation checks, audits, and consistency reviews
• Conduct named entity recognition and taxonomy tagging
• Conduct content safety labeling for policy and harm categories
• Document edge cases and guideline gaps to improve labeling instructions
• Collaborate asynchronously with reviewers, QA, and data operations teams

Required Qualifications

• Professional experience in data annotation or data labeling
• Strong written English comprehension and attention to detail
• Able to follow detailed guidelines with high precision and consistency

Preferred Qualifications

• Experience with RLHF, LLM training pipelines, prompt evaluation, or QA evaluation
• Familiarity with NER, content safety labeling, and CV annotation tools

Remote Work Notes

These roles are Remote and can be performed from New York. Some projects may require availability aligned with US business hours or structured QA review cycles.

How To Apply

Apply via Rex.zone with a resume highlighting annotation accuracy, guideline-driven decision making, QA evaluation, and any RLHF or prompt evaluation experience.