Senior AI Quality & Reliability Engineer

About the role

In this role, you will drive the hands-on design, development, and execution of AI Quality Engineering initiatives supporting Vizient's enterprise AI transformation efforts. You will design and implement AI Quality Engineering practices, AI validation processes, AI-assisted testing approaches, runtime quality controls, and scalable testing frameworks that support the responsible deployment of AI-powered business solutions. This role combines hands-on quality engineering, AI-enabled testing modernization, healthcare workflow validation, technical mentorship, and cross-functional collaboration. You will help advance Vizient's Quality Engineering capabilities beyond traditional software testing toward AI-native validation, AI-assisted testing, runtime observability, reliability engineering, and modern AI Quality Engineering practices.

Responsibilities

• Design, develop, and execute AI Quality Engineering strategies supporting AI-powered applications, large language model (LLM) solutions, intelligent automation, agentic systems, and enterprise AI platforms.
• Build and implement scalable AI Quality Engineering practices, including AI-native testing approaches, validation processes, runtime quality controls, reusable testing accelerators, and automated testing frameworks.
• Lead AI validation activities including functional testing, prompt validation, workflow testing, regression testing, release validation, runtime quality assurance, and production reliability support.
• Partner with AI Engineering, AIOps, LLMOps, Security, Governance, Clinical, Data, and Product teams to deliver scalable AI Quality Engineering processes across enterprise AI initiatives.
• Support runtime reliability through observability, telemetry, distributed tracing, monitoring, drift detection, incident response, and operational quality assurance for AI-enabled systems.
• Develop and maintain AI evaluation frameworks, validation datasets, quality scoring methodologies, and automated testing workflows that improve the reliability and scalability of AI solutions.
• Collaborate with Clinical, Operational, and Engineering stakeholders to validate healthcare workflows, payer operations, and AI-enabled business processes while supporting responsible AI deployment through human-in-the-loop validation practices.
• Coordinate testing activities across Agile delivery teams, including sprint planning, test execution, defect management, issue tracking, release readiness, risk identification, and production support.
• Mentor Quality Engineers and provide technical guidance that promotes engineering excellence, AI-enabled testing modernization, continuous improvement, and adoption of modern Quality Engineering practices.
• Research, evaluate, and recommend emerging AI Quality Engineering, testing automation, observability, and runtime assurance technologies to continuously improve enterprise AI quality capabilities.

Requirements

• Relevant degree preferred. Advanced degree in Computer Science, Engineering, or a related field highly preferred.
• Bachelor's degree in Computer Science or equivalent.
• 5 or more years of experience in Quality Engineering, Quality Assurance, software testing, enterprise application delivery, technology operations, or related technology functions required.
• 2 or more years of experience providing technical leadership for testing initiatives, automation programs, or enterprise technology delivery projects required.
• Experience supporting Quality Engineering or Quality Assurance across enterprise platforms, APIs, healthcare applications, operational workflows, or integrated business systems required.
• Strong knowledge of software development life cycle (SDLC), Agile methodologies, test automation frameworks, defect management, release validation, and production support processes required.
• Experience validating AI-powered applications, intelligent automation, machine learning, large language model (LLM), or AI-enabled business workflows preferred.
• Experience with AI Quality Engineering practices, AI-assisted testing, runtime observability, monitoring, telemetry, or reliability engineering preferred.
• Strong analytical, problem-solving, organizational, communication, collaboration, and leadership skills required.
• Demonstrated ability to manage multiple priorities and deliver results within fast-paced, highly collaborative enterprise environments required.
• Experience in healthcare technology, payer operations, clinical workflows, or other regulated industries supporting AI governance and responsible AI deployment preferred.

Pay

The current salary range for this role is $102,400.00 to $179,000.00. This position is also incentive eligible.

Benefits

Vizient offers a comprehensive benefits plan. View our benefits here.