Lead Software Test Automation Architect

About the Role

The client is seeking a Lead Software Test Automation Architect to define and own an enterprise-grade test automation strategy across UI, API, data, and platform layers while evolving quality engineering capabilities to support Agentic AI and LLM-based systems.

Responsibilities

• Define and own the test automation architecture and multi-layer strategy (UI, API, contract, integration, and end-to-end), aligned to product risk and release cadence.
• Design and implement testing approaches for non-deterministic AI outputs, including confidence-based assertions, semantic similarity checks, hallucination detection, and regression detection across model and prompt versions.
• Build and scale high-throughput automation programs that are reliable, fast, and easy to extend, with a focus on reducing flaky tests and optimizing CI/CD pipelines.
• Integrate automation into CI/CD pipelines with quality gates including PR checks, nightly suites, and release readiness criteria.
• Establish automation standards, patterns, and reference implementations covering framework structure, test data strategy, and engineering-friendly quality dashboards.
• Partner with ML/AI, Product, and Engineering teams to define acceptance criteria and measurable quality gates for AI capabilities.
• Serve as the quality engineering technical authority, leading design reviews, coaching engineers on automation best practices, and influencing delivery roadmaps.

Minimum Requirements

• 8-12 years in Quality Engineering and Test Automation with architect-level ownership of frameworks and strategy.
• Hands-on expertise with Playwright and TypeScript as primary automation tooling.
• 3-5 years of experience defining and owning test automation architecture at an enterprise or platform level.
• Proven experience designing and building agentic AI testing capabilities, including working with or testing AI agents, evaluation harnesses, and benchmark datasets.
• Strong programming skills in at least one modern language, such as TypeScript, JavaScript, Java, Python, or C#.
• Experience integrating automation into CI/CD platforms such as GitHub Actions, Jenkins, or Azure DevOps.

Preferred Qualifications

• Experience testing LLM or Agentic AI features including semantic scoring, prompt regression strategies, and structured evaluation reports.
• Familiarity with modern quality approaches including test pyramid, contract testing, consumer-driven contracts, risk-based testing, and shift-left practices.
• Knowledge of security and performance testing integration.
• Experience building reusable automation libraries shared across multiple teams or products.
• Proficiency in Python as a secondary scripting or automation language.

Pay

$55-66.00/hr

Benefits

Eligible for Health, Dental, Vision, and 401K

Location-Type: Hybrid - Houston / Coppell, TX 2 days onsite
Duration: Contract-to-Hire, 6 months
Start Date: ASAP
Visa Sponsorship: Not eligible for visa sponsorship