AI/ML QA Tester

Location: Rosemead, CA | Duration: 6 months

About the role

Ensure AI products behave accurately, safely, fairly, and reliably. Validate AI products across real-world scenarios and use cases. Partner with product teams, engineering teams, and subject matter experts (SMEs) to validate model behavior, data quality, and trust risks throughout the AI lifecycle.

Responsibilities

• Test AI outputs for accuracy, consistency, edge cases, and failure modes.
• Validate training and inference data quality.
• Identify potential bias, data leakage, and data quality issues.
• Define and execute model evaluation and regression tests.
• Reuse and expand the Golden Dataset for regression testing across builds, prompt changes, model changes, and product releases.
• Test AI behavior within end-to-end user workflows and integrations.
• Support Responsible AI testing, including fairness, explainability, and compliance.
• Monitor model performance after release, identifying and tracking model drift, user feedback, regressions, and performance degradation.
• Evaluate and improve test automation coverage.
• Communicate testing risks and findings clearly to stakeholders.
• Track and report quality metrics, including accuracy, precision, and recall where applicable.
• Clearly identify missing content, incorrect content, and hallucinated content.

Requirements

• 3-7+ years of experience in QA, software testing, or quality assurance/quality roles.
• Experience working with AI- or data-driven products preferred.
• ISTQB certification is a plus.
• AI/Data-related training or certifications are a plus.
• Responsible AI training is a plus.

Skills

• Strong experience in QA test planning, exploratory testing, and User Acceptance Testing (UAT).
• Understanding of AI/ML fundamentals and evaluation metrics.
• Strong data analysis skills and ability to analyze and reason about model behavior.
• Ability to test Generative AI prompts and outputs preferred.
• Strong domain knowledge and user-centric mindset.
• Excellent documentation and communication skills.

Key Skills / Keywords

• AI QA
• AI Testing
• GenAI Testing
• Machine Learning Testing
• Model Evaluation
• Regression Testing
• Exploratory Testing
• UAT
• Data Quality
• Golden Dataset
• Responsible AI
• AI Bias Testing
• Explainability
• Model Drift
• Prompt Testing
• AI Hallucination Testing
• Test Automation
• Accuracy / Precision / Recall
• Data Analysis