Developer-Full Stack Senior

About the role

Design and implement agentic testing patterns that can be adopted by multiple Underwriting teams (and later other domains). Create reference implementations demonstrating test generation assistance, test maintenance assistance, and failure analysis assistance. Establish a standard architecture for test code organization, tagging, data management, and execution across UI, API, and service layers.

Responsibilities

• Design and implement agentic testing patterns with reference implementations (sample repos/templates) demonstrating:
  • Test generation assistance (from requirements, APIs, contracts, schemas)
  • Test maintenance assistance (auto-updating selectors/contracts, flaky test triage)
  • Failure analysis assistance (root cause suggestions, log correlation, defect drafting)
• Establish standard architecture for test code organization, tagging, data management, and execution across UI + API + service layers
• Define and publish coverage standards including:
  • Test type mix (unit vs API vs UI vs contract vs integration)
  • Risk-based prioritization and traceability to requirements
• Provide templates usable across teams:
  • Test case/spec templates (Gherkin-style or equivalent)
  • Definition of Ready / Definition of Done quality checklists
• Create scalable tagging/metadata strategy (feature, service, risk, priority, data sensitivity) to support reporting and quality gates
• Build automated reporting that aggregates test + service data across multiple microservices:
  • Service health signals (logs/metrics/traces if available)
  • Defect signals (issue tracker metadata if available)
• Generate GenAI-driven summaries:
  • Failure clustering and trend analysis
  • "What changed?" insights (commit/PR correlation)
• Produce outputs consumable by engineering leadership and teams (dashboards, markdown summaries in PRs, artifacts in CI)
• Build automated review agents that evaluate user stories/requirements for minimum required clarity and data before development/testing starts:
  • Ambiguity detection and missing edge cases
  • Data/privacy considerations and environment needs
• Integrate gates into workflow (PR checks, issue templates, GitHub Actions) to reduce churn and rework

Required Technical Skills

• GenAI / LLM + agentic development:
  • Hands-on experience building LLM-powered agents (tool-using, multi-step reasoning, guardrails)
  • Experience with prompting patterns, structured outputs (JSON schemas), evaluation, and reducing hallucinations
  • Ability to design agent workflows for:
    • Test generation/augmentation
    • GitHub Actions (CI/CD pipelines, reusable workflows, composite actions)
• Test automation engineering (framework expertise):
  • Karate (API testing, contract-like checks, data-driven testing, mocks)
  • Happy path scenarios
• Automated requirements review agents:
  • Acceptance criteria completeness
  • Requirements review and completeness validation
  • Report generation and summarization
• GitHub platform + GHCP (Copilot) for engineering workflows:
  • Strong proficiency with GitHub Copilot in day-to-day development
  • Deep experience with GitHub platform capabilities: PR checks, branch protections, CODEOWNERS, templates
  • Automation via GitHub APIs/webhooks (as needed)
• Advanced experience designing and implementing automation with:
  • Playwright (UI automation, selectors strategy, parallelization, trace/video artifacts)
• Strong understanding of test design and coverage:
  • Negative/validation scenarios
  • Edge/boundary scenarios
  • Data setup/teardown strategies and test isolation
• Cross-service reporting and data aggregation:
  • Proven ability to aggregate and normalize results from multiple microservices and multiple pipelines
  • Experience producing actionable automated reports (trend analysis, failure clustering, service correlation)
• Experience implementing automated checks that validate:
  • Required test data and environment dependencies
  • Non-functional requirements (performance, security, observability) when applicable

Deliverables / What Success Looks Like

• A reusable agentic testing automation kit adopted by multiple teams
• Published coverage standards + templates and onboarding documentation
• A working GenAI-assisted reporting pipeline aggregating results across microservices
• Automated quality gates integrated into GitHub workflows that measurably reduce story churn