Senior Data Scientist - GenAI

About the role

The Senior Data Scientist for Generative AI within our Central AI Hub will be the foundational scientific owner of our LLM strategy. This role involves evaluating model performance, fine-tuning open-source or proprietary models for complex business logic, and ensuring our AI products behave predictably at scale.

Responsibilities

• Design and implement robust frameworks to quantitatively evaluate LLM performance (accuracy, repeatability, hallucination rates) before and after deployment.
• Lead the fine-tuning and optimization of our Core Policy Models.
• Act as the central GenAI expert for product engineering teams. Guide them on improving JSON payload structures, advanced prompt engineering techniques, RAG (Retrieval-Augmented Generation) architectures, and evaluation pipelines.
• Establish best practices, playbooks, and standardized pipelines for all AI/LLM integrations across Cover Genius.
• Stay at the cutting edge of Generative AI research, actively identifying new models, tools, and methodologies that can improve our operational efficiency and product offerings.

Requirements

• Masters or PhD in Physics, Statistics, Mathematics, Computer Science, or other Quantitative fields.
• 5+ years of practical, hands-on ML/Data Science experience, with a heavy emphasis on NLP and at least 1-2 years dedicated to deep Generative AI/LLM applications.
• Hands-on experience with LLM orchestration frameworks, RAG architectures, and model fine-tuning techniques.
• Deep understanding of how to measure generative text performance (e.g., LLM-as-a-judge, traditional NLP metrics like BLEU/ROUGE, custom programmatic evaluators).
• Strong foundation in validating AI-driven features, ensuring thresholds are statistically sound to prevent surfacing noise, and integrating human-in-the-loop feedback mechanisms.
• Advanced proficiency in Python and SQL. Experience working with major LLM APIs as well as open-weights models.
• Ability to closely collaborate with software engineers to optimize data structures (like JSON schemas) for better LLM parsing and repeatability.

Qualifications

• Exceptional problem-solving skills with a high tolerance for ambiguity in the fast-paced GenAI landscape.
• Outstanding communication skills, capable of explaining the nuances of LLM behavior to both technical and non-technical stakeholders.
• A proactive mindset in identifying areas where "random LLM implementations" can be standardized into a robust, centralized service.