Senior Specialist, AI Model Validation

About the role

Your opportunity At Schwab, innovative thinking meets practical problem solving. As part of Model Risk Oversight, you will work at the intersection of AI /ML engineering, quantitative analysis, and responsible model governance. You will help evaluate AI and machine learning systems before and after they reach production, with a particular focus on generative AI and applications that use large language models. The AI Model Validation Team designed this role for an early-career engineer or quantitative researcher who wants hands-on exposure to real AI systems, rigorous experimentation, and high-impact technical communication. You will write code, design tests, analyze model behavior, investigate failure modes, and present evidence-based conclusions to technical teams and senior leaders.

Responsibilities

• Validate AI and machine learning models by independently assessing intended use, design choices, data, implementation, controls, performance, and limitations.
• Build reproducible testing frameworks, analyses, and prototypes to evaluate model quality, robustness, safety, and reliability.
• Evaluate generative AI and retrieval-augmented generation applications, including prompt behavior, retrieval quality, groundedness, hallucination risk, response consistency, and guardrail effectiveness.
• Apply statistical and machine learning methods to benchmark models, analyze errors, compare alternatives, and assess results.
• Review source code, notebooks, technical documentation, model inventories, performance monitoring, and production-control evidence.
• Partner with model developers, product teams, data scientists, engineers, risk partners, and external consultants to understand systems and challenge assumptions constructively.
• Document findings in clear validation reports, technical appendices, issue statements, and executive-ready presentations.
• Evaluate ongoing performance monitoring and participate in periodic reviews of models after implementation.
• Contribute to reusable validation frameworks, testing utilities, and effective-challenge methods as AI capabilities and industry practices evolve.

Requirements

• Master's degree, or bachelor's degree with strong relevant project or research experience, in computer science, data science, statistics, applied mathematics, engineering, economics, quantitative finance, or a related field.
• 0-2 years of professional experience. Relevant internships, research, capstone projects, open-source contributions, or substantial independent projects are welcome.
• Proficiency in Python and experience using common data science or machine learning libraries.
• Working knowledge of core machine learning concepts, experimental design, model evaluation, and statistical reasoning.
• Ability to read technical documentation and code, investigate unfamiliar systems, and communicate conclusions clearly.
• Curiosity, sound judgment, attention to detail, and a willingness to challenge assumptions with evidence.
• Applicants must be currently authorized to work in the United States on a full-time basis without employer sponsorship.

Qualifications

• Hands-on experience with large language models, generative AI, natural language processing, embeddings, vector search, or retrieval-augmented generation.
• Experience with evaluation frameworks, prompt testing, red teaming, model monitoring, or responsible AI techniques.
• Familiarity with SQL, Git, containers, cloud platforms, or modern ML development workflows.
• Understanding software testing, data quality, reproducibility, or model deployment concepts.
• Experience presenting technical work through research