Sr AI Quality & Reliability Engineer

About the role

The Sr AI Quality & Reliability Engineer will drive the hands‑on design, development, and execution of AI Quality Engineering initiatives supporting enterprise AI transformation efforts. This role designs and implements AI Quality Engineering practices, AI validation processes, AI‑assisted testing approaches, runtime quality controls, and scalable testing frameworks to enable responsible deployment of AI‑powered business solutions. It combines hands‑on quality engineering, AI‑enabled testing modernization, healthcare workflow validation, technical mentorship, and cross‑functional collaboration.

Location: Chicago/Irving (Hybrid – 3 days onsite at either location).

Responsibilities

• Design and execute AI Quality Engineering activities supporting AI‑powered applications, LLM‑enabled workflows, intelligent automation solutions, agentic systems, and enterprise AI platforms.
• Build and implement AI Quality Engineering practices, including AI‑native testing approaches, validation processes, runtime quality controls, reusable testing accelerators, and scalable testing workflows.
• Drive modernization of traditional Quality Engineering practices to address AI‑enabled workflows, intelligent orchestration, and evolving healthcare operational workflows.
• Support implementation of AI quality standards, validation approaches, release readiness processes, and governance‑aligned testing practices for enterprise AI adoption.
• Coordinate AI validation activities, defect management, test execution, release support, and quality improvement initiatives across AI‑enabled systems.
• Support AI validation activities such as functional validation, prompt testing, workflow testing, regression testing, runtime quality assurance, and production reliability support.
• Partner with AI Engineering, AIOps, LLMOps, Security, Governance, Clinical, and Data teams to scale AI Quality Engineering processes across enterprise AI delivery initiatives.
• Support runtime quality and reliability practices, including telemetry alignment, distributed tracing, observability, monitoring coordination, incident support, release validation, and runtime reliability improvement.
• Develop and maintain AI evaluation frameworks, validation datasets, quality scoring approaches, and automated testing workflows.
• Implement drift detection, runtime monitoring, and operational quality‑assurance initiatives for AI‑enabled systems.
• Facilitate human‑in‑the‑loop validation processes and operational review workflows to ensure responsible AI deployment.
• Drive adoption of AI‑assisted testing approaches, intelligent automation, reusable testing accelerators, and scalable test automation practices.
• Advance observability initiatives to improve reliability, traceability, and confidence across AI‑enabled systems.
• Collaborate with Clinical, Operational, and Engineering stakeholders to validate healthcare workflows, payer operations, and AI‑enabled business processes.
• Contribute to delivery coordination across AI Quality Engineering initiatives, including sprint execution, testing coordination, issue tracking, risk identification, and release support.
• Evaluate testing readiness, implementation dependencies, quality risks, and operational support considerations with stakeholders.
• Support tooling evaluations, automation initiatives, and modernization efforts to mature AI Quality Engineering capabilities.
• Establish scalable testing processes, reusable quality engineering assets, and operational support models for AI delivery teams.
• Promote adoption of modern AI Quality Engineering practices across engineering and delivery organizations.
• Mentor quality engineers and analysts, providing technical guidance while fostering a collaborative, continuously learning, engineering‑focused culture.
• Communicate testing risks, implementation issues, delivery trade‑offs, and operational recommendations to technical and business stakeholders.
• Promote engineering discipline, continuous improvement, responsible AI adoption, and operational accountability across AI Quality Engineering initiatives.
• Research and evaluate emerging AI quality engineering, testing, observability, automation, and runtime assurance technologies to support continuous improvement.

Qualifications

• Engineering degree (BE, ME, BTech, MTech, BSc, or MSc).
• Technical certification in multiple technologies is desirable.