Verification & Validation Data Scientist

Responsibilities

• Design and run the independent validation methodology: benchmark synthetic-audience and automated output against real panel holdout and other ground-truth data.
• Own the evaluation metrics, test harness design, and reported accuracy bounds; quantify where each system is and is not reliable.
• Certify fitness for use, translate results into clear “validated for this use / not validated for that use” determinations.
• Operate independently of the AI/Synthetic Engineer: verify the models and pipelines AI/Synthetic Engineer builds without having built them.
• Maintain the audit trail and evidence base that supports the function’s credibility with stakeholders and any external auditor.
• Monitor for model and data drift over time; re-validate as models, prompts, and source data change.

Requirements

• Education & Experience
• Education: Bachelor’s degree in computer science, mathematics, or engineering or relevant equivalent experience in lieu of degree. Master’s degree preferred.
• Experience: Seven (7) or more years in data science, model validation, ML evaluation, or an independent V&V / QA function. Demonstrated experience independently validating models or systems against ground-truth data, evaluation design, benchmarking, and error analysis.
• Certifications: NA
• Knowledge, Skills & Abilities
• Strong applied statistics and metrics design; fluency in accuracy, bias, calibration, and uncertainty quantification.
• Working understanding of LLMs and synthetic-respondent systems and their failure modes — enough to test them rigorously.
• Independence and rigor: the temperament to challenge results and withhold sign-off, including from colleagues’ work.
• Clear communication: able to tell a non-technical executive, with evidence, why a result can or cannot be trusted.
• Ability to effectively leverage artificial intelligence (AI) tools and technologies to streamline workflows, enhance productivity, and improve overall work quality.
• Familiarity with survey/behavioral data as a validation target preferred.

Physical Requirements

• Prolonged periods of sitting at a desk and working on a computer.
• Frequent use of hands and fingers for typing, handling documents, and using office equipment.
• Occasional standing, walking, bending, and reaching.
• Ability to lift and carry up to 30 pounds as needed.

Work Environment

• Hybrid Schedule (3 Days In-Office/2 Days Remote)
• This position follows a hybrid work schedule, with Tuesday through Thursday in office and Monday and Friday remote.
• Employees must be available during standard business hours, with core hours beginning between 8:00–9:00 a.m. and concluding between 5:00–6:00 p.m. local time.
• Travel: Occasional 0 – 10%

Pay

The hiring range for this position is $110,000 to $135,000 per year.

Schedule

This position follows a hybrid work schedule, with Tuesday through Thursday in office and Monday and Friday remote.