Lead QA Engineer

About the role

Titan builds AI software for banks: purpose-built small language models, a banking ontology, and AI bankers that financial institutions can trust. Our models outperform general-purpose LLMs by 30 to 80 percent on banking tasks. Customers include community banks, credit unions, and large regional and super-regional institutions. We are backed by leading fintech investors and operate under the compliance, audit, and model-risk standards that banking requires.

Why This Role Exists

This role exists because Titan is scaling from a handful of live banking customers to hundreds. Right now, there is no formal QA function. There is no evaluation framework, no regression baseline, no quality gate in CI/CD. A QA failure at a bank is not a user experience problem. It is an operational and regulatory risk. This role exists because that gap has to close before the customer count grows. This is a hands-on, individual-contributor role first. You are coming in to do the work: write the test cases, build the evaluation framework, set up CI/CD gates, and triage bugs alongside engineering. The function gets built because you build it yourself. Once the practice is stable and documented, you bring in QE engineers to scale it.

What You Own

• AI Evaluation: Write the assertions, define the behavioral contracts, and own regression baselines for model behavior. Build tooling that handles non-deterministic systems like LLMs.
• Test Coverage: Write and maintain the automated test suite, including end-to-end, integration, and regression coverage for backend APIs, document ingestion pipelines, AI inference workflows, and frontend surfaces. Set up and enforce quality gates in CI/CD pipelines.
• Compliance and Client Quality: Produce test artifacts, audit logs, and process documentation meeting SOC 2 Type II standards. Work directly with Forward Deployed Engineering on client-side validation and production issue reproduction. Ensure work is defensible on its own.

Who You Are

• Seven or more years in software QA engineering, with at least two years personally testing AI or ML systems. Fluent in Python and have built automated suites using pytest, Playwright, or Selenium. Hands-on experience with RAGAS, DeepEval, LangSmith, or comparable evaluation tooling. Experience integrating QA gates into CI/CD pipelines and owning the process end to end.
• Experience in fintech, banking, or another regulated environment is a strong advantage. Familiarity with document processing pipelines, multi-agent architectures, RAG validation, or observability tooling such as Arize or Langfuse puts you ahead.
• You are not here to manage. You are here to build and test.

What Success Looks Like

• In your first 90 days: a diagnostic of current test coverage shared with engineering leadership, an evaluation framework running against at least one AI-powered workflow that you built yourself, and quality gates live in CI/CD.
• In your first six months: regression baselines established for model behavior, SOC 2 test artifacts documented and audit-ready, and the test suite running on every release without manual intervention.
• At one year: the function is staffed, coverage scales with every product release, and quality is a first-class input to every deployment decision. The work you did personally is the foundation the team builds on.

Compensation and Structure

• Competitive base and meaningful equity. Remote (US). Occasional travel to client sites and team offsites.
• Compensation Range: $140K - $165K