Evaluations Operations Analyst (Contract)

About the role

We are a research lab looking for an Evaluations Operations Analyst to support our evaluation operations as we scale our audio model development. You will work alongside our Evaluations Manager within the Product and Research Operations team, helping run the day-to-day operations for our globally distributed workforce.

Responsibilities

• Serve evaluation tasks to our annotator workforce and manage assignment distribution across projects and locales.
• Track and verify completion of assignments, flagging delays or issues before they become blockers.
• Act as a first line of support for evaluators, fielding operational questions about project setup, platform access, and task instructions.
• Help maintain project trackers, assignment records, and operational documentation.
• Support calibration workflows as needed to ensure evaluation data meets our quality bar.

Requirements

• Experience managing or coordinating annotator workforces, including task assignment, scheduling, and follow-up.
• Hands-on experience with data annotation platforms.
• Strong written communication skills, particularly in coordinating with globally distributed contractor workforces in different time zones and language groups.
• Ability to work independently and manage your own time effectively to meet tight project deadlines.
• Comfort working in spreadsheets, task management tools (e.g., Asana), and messaging platforms (e.g., Slack).

Qualifications

• Experience with linguistics or in language-related fields is a plus.
• Proficiency in languages beyond English to assist with multilingual evaluation workflows is a plus.
• Familiarity with human evaluation methodologies (e.g., scale-based ratings) is a plus.
• Proficiency in using LLM assistants to move quickly without sacrificing quality is a plus.

Skills

• Excellent organizational and time management skills.
• Strong attention to detail.
• Ability to communicate effectively with diverse teams.

Benefits

Details Rate: $25-35/hr
Hours: 20-40 hrs a week
Location: US-based (remote)
Duration: Initial 3 months (with potential for extension)

Details

• Please Note: This is an independent contractor role and is not eligible for Cartesia’s full-time employee benefits. Contractors are engaged under a separate contractor agreement and do not receive company-sponsored health insurance, retirement plan participation, paid time off, or other benefits made available only to eligible full-time employees, except as expressly required by applicable law or stated in the contractor agreement.
• Cartesia participates in E-Verify and will provide the federal government with Form I-9 information to confirm employment eligibility after hire.

In-office policy

• We’re an in-person team based out of offices in 🇺🇸 San Francisco, 🇬🇧 London and 🇮🇳 Bangalore. We love being in the office, hanging out together, and learning from each other every day.

Visa sponsorship

• We provide visa sponsorship support and assess each circumstance on a case-by-case basis. However, visa sponsorship is dependent on many factors, including the role you are applying for, and the location you are going to be based, and so we can't always guarantee success.

Your Recruiter

• Your Recruiter will work with you to understand your visa sponsorship needs from the first call.

Shipping fast

• All of our work is novel and cutting edge, and execution speed is paramount. We have a high bar, and we don’t sacrifice quality or design along the way.

Supporting each other

• We have an open & inclusive culture that’s focused on giving everyone the resources they need to succeed.

Our Benefits (US Employees Only)

• Compensation: Competitive base salary alongside attractive equity package.
• Health Insurance: Fully covered medical insurance along with dental and vision for you and your family.
• Parental Leave: 9 weeks paternity & 12 weeks maternity leave.
• 401(k):
• Commuter Allowance: A monthly stipend to help you get to and from the office.
• Flexible PTO: Take as much time as you need to recharge your batteries.
• Meals & Snacks: Lunch, dinner and plenty of snacks, provided daily.
• Your own personal Yoshi: Our Commitment to Equal Opportunity

Cartesia is an equal opportunity employer.

We consider qualified applicants without regard to race, color, religion, sex, national origin, age, disability, veteran status, genetic information, or any other legally protected status.