DirectJobHub
Jobs · Quality Assurance · Georgia

AI Quality Test Automation - Lead Software Engineer

NiCE · Atlanta, GA · 1 wk ago
Quality AssuranceFull-time

Platform & Automation Leadership

Own the automation platform: define and build scalable frameworks, standards, and governance (services, APIs, UI) that enable teams to produce robust, consistent coverage.

Set direction for test automation tooling and AI-assisted techniques that accelerate test design, authoring, maintenance, and triage.

Lead by example — write automation, set the quality bar, mentor engineers, and drive adoption of the practices you establish.

Own CI automation health: build and maintain test environments, pipelines, and tooling (including containers/CI), reduce flakiness, and shorten feedback loops.

Be a highly visible advocate for quality — lead cross-team discussions, drive alignment and decisions, and escalate risks and blockers immediately.

AI-Native Test Engineering

Lead the adoption of LLM-powered test generation: from natural language requirement ingestion to executable, maintainable test output.

Build and maintain a self-healing test infrastructure layer — leveraging AI to detect broken selectors, drifted APIs, or changed behaviors and propose or apply fixes autonomously.

Define prompt engineering standards, context injection patterns, and RAG architectures that ground test generation in real codebase context.

Implement guardrails to ensure AI-generated test output is verified, traceable, and safe to ship — including versioning, ownership attribution, and confidence scoring.

Own automated coverage and risk reporting (unit/integration/e2e) and use it to drive targeted gap closure and release readiness.

Quality Gates & CI/CD

Lead risk-based test strategy with product and engineering — define acceptance criteria and quality gates that support high delivery velocity without sacrificing customer-impacting quality.

Design adaptive quality gates for AI-accelerated CI/CD pipelines — gates that reason about risk, not just pass/fail thresholds.

Build risk-scoring models that adjust gate strictness based on change scope, code origin (human vs. AI-generated), historical failure patterns, and deployment context.

Architect the observability layer for automated pipelines: surface signals that indicate poor quality decisions in real time.

Establish rollback and circuit-breaker patterns for autonomous deployments triggered by quality signal degradation.

AI Model & Agent Validation

Build behavioral testing frameworks for validating AI agents and LLM-powered features in production — testing non-deterministic outputs with statistical rigor.

Design evaluation benchmarks for internal AI tooling: measuring task completion accuracy, hallucination rates, and decision quality over time.

Define adversarial and edge-case testing methodologies for AI features: prompt injection resistance, boundary condition handling, and graceful degradation.

Partner with ML platform and data science teams to establish quality acceptance criteria for every model and agent promoted to production.

Desired Skills & Experience

  • Modern test frameworks (e.g., pytest, JUnit, NUnit, Playwright, Cypress) and API testing (REST, gRPC)
  • Containerization and environments: Docker; Kubernetes a plus
  • Relational databases and SQL; ability to validate data pipelines and analytics outputs
  • Experience with Elasticsearch or similar text-retrieval data stores
  • Performance/load testing (e.g., JMeter, k6, Locust) and profiling/observability
  • Cloud experience (AWS/Azure/GCP) and Infrastructure-as-Code (e.g., Terraform)
  • Experience building AI-enabled automation workflows (e.g., Claude/OpenAI APIs, prompt patterns for test generation, repo-aware RAG, scripts/services that turn AI output into runnable tests)
  • Familiarity with agent frameworks (LangChain, LlamaIndex, AutoGen, or equivalents) and their tradeoffs in production quality pipelines
  • Experience designing evaluation harnesses for non-deterministic AI systems — statistical confidence, behavioral consistency, and regression detection
  • Agile software development experience (Scrum / XP)

About NiCE

NICE Ltd. (NASDAQ: NICE) software products are used by 25,000+ global businesses, including 85 of the Fortune 100 corporations, to deliver extraordinary customer experiences, fight financial crime and ensure public safety. Every day, NiCE software manages more than 120 million customer interactions and monitors 3+ billion financial transactions.

Known as an innovation powerhouse that excels in AI, cloud and digital, NiCE is consistently recognized as the market leader in its domains, with over 8,500 employees across 30+ countries.

NiCE is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, age, sex, marital status, ancestry, neurotype, physical or mental disability, veteran status, gender identity, sexual orientation or any other category protected by law.

Similar jobs