Sr. QA Test Engineer

About the role

This role is part of a platform team responsible for testing APIs for high performance. The primary focus is on test automation.

Responsibilities

• Performance & Reliability Testing (AI-Aware)
• Load/performance testing for inference endpoints (latency, throughput, concurrency)
• Cost-aware testing (token usage, rate limits, fallbacks)
• Resilience tests: retries, circuit breakers, model timeouts, degraded-mode behavior

Requirements

• 3+ years in QA automation or SDET-type work (adjust by level)
• 1+ year exposure to AI/LLM or ML-driven features is a plus
• Strong test automation in Python and/or Java/TypeScript
• Hands-on with frameworks/tools such as: UI: Playwright / Cypress / Selenium and API: pytest + requests, Postman/Newman, REST Assured
• Ci/Cd integration: Git, GitHub Actions/Jenkins/GitLab CI, test reporting, gating
• Test design: equivalence partitioning, boundary testing, risk-based testing, defect triage
• AI-Specific Testing Competencies (Key): LLM/application behavior testing, validating correctness when outputs are probabilistic, evaluation strategies, golden datasets, scoring rubrics, human-in-the-loop reviews, non-determinism handling, statistical assertions, repeated runs, variance thresholds, prompt and regression management, versioning prompts, detecting prompt drift, replay tests, RAG testing (if applicable), retrieval quality (recall/precision), grounding checks, citation validation, doc freshness, safety & quality checks, hallucination detection, toxicity/PII leakage checks, policy compliance tests, data & observability ability to create and maintain test datasets, familiarity with telemetry for AI systems, logging prompts/outputs safely, traceability, correlation IDs, tools like OpenTelemetry, ELK/Splunk, Datadog/Grafana (any equivalent), understanding of data privacy constraints (masking/redaction) and secure test data practices, API / Microservices / Cloud Comfortable testing distributed systems, microservices, async workflows, queues/events, basic cloud proficiency (AWS/Azure/GCP) and containerization (Docker, optional Kubernetes)

Qualifications

• Basic cloud proficiency (AWS/Azure/GCP) and containerization (Docker, optional Kubernetes)

Skills

• Strong communication and analytical skills
• Ownership mindset: building reusable test harnesses, improving quality metrics, preventing regressions

Benefits

N/A

Pay

N/A

Schedule

N/A