Staff Engineer, Engineering Productivity & AI Quality

The role

Every great AI company ends up building the same invisible machine: the harnesses, tests, instructions, and review loops that let a small team ship with impossible leverage. At Harper that machine is existential. Our agents write code, serve customers, assemble submissions, and make decisions that move revenue — and AI-generated code volume has pulled the scaling problem forward. Even with a 20-person engineering team, our coding agents create the surface area, review burden, and architectural drift of a 100-person org. If the rails are strong, twenty engineers operate like a hundred; if they're weak, velocity turns into drag and the CTO becomes the rail — which doesn't scale.

This is the founding seat for that machine. You'll turn the CTO's taste into systems — PR preflight, integration tests, architecture rules, agent instructions, eval gates, the feedback loops every engineer feels daily — across three sub-disciplines: Harness Engineering (the meta-harness over our frontier coding agents, OpenClaw, Hermes, and internal agents), Developer Experience (CI/CD gates, build caching, merge queues, dev/staging/CI parity, the internal platform, eval infrastructure), and AI Quality (eval suite design, golden datasets, LLM-as-judge graders, production trajectory monitoring, drift detection, anti-slop guardrails).

What You'll Own

• CI/CD quality gates across Harper's most critical services — the minimum bar before code can merge.
• Integration test harnesses anchored to real failure modes — every repeated operational failure becomes a regression test, a validation, or an architecture rule.
• The agent harness substrate — sandbox lifecycle, tool routing, prompt/context layer, model-provider abstraction, multi-agent coordination.
• The information environment our coding agents read.
• Automated PR preflight — service-impact summary, tests run, missing tests, model/migration changes, critical-path warnings.
• The robot that reviews every PR before a human does.
• Architecture-rule enforcement — custom lints and structural tests that encode the CTO's taste mechanically.
• Evaluation framework infrastructure — pre-merge eval gating, experiments against curated datasets, production trajectory monitoring, all wired together.
• Engineering metrics that matter — rework rate, escaped defects, flaky-test count, deploy rollbacks, time-to-confident-ship, AI-generated PR quality.
• Anti-vanity, anti-LOC.

What We're Looking For

• 8+ years building software, including Senior+ scope at a high-growth company (8–12 years total, 3+ at Senior+).
• A track record of building developer-productivity, platform, CI/CD, build, test-infra, or internal tooling that other engineers actually adopted.
• Production AI/ML systems experience (agent harness, eval frameworks, LLM-as-judge graders, prompt/context engineering), even if it's not your primary stack.
• Strong opinions on maintainability, architecture, testability, and DX — backed by mechanical enforcement, not lectures.
• Excited by AI coding agents, skeptical enough to build the guardrails they need.
• Can describe a specific lint rule, integration test, or eval-harness pattern you built that kept a class of bugs out of production for good.
• Writes code with AI daily and routinely run 3+ parallel sessions, and you'd rather create leverage for other engineers than own one product surface.
• Strong written communication (RFCs, architecture-rule docs, lint-rule rationale, playbooks).

Bonus

• Evaluation framework infrastructure (OSS or internal).
• Developer platforms at an AI-native company.
• Custom lint/structural-test authoring at scale.
• Agent harnesses (sandboxing, isolation, execution environments).
• Encoding a CTO's architectural taste into mechanical rules.

The information environment our coding agents read

The information environment our coding agents read includes automated PR preflight, integration test harnesses, and architecture rules. These elements ensure that the code is ready for merging and that it adheres to the established guidelines and best practices.

Automated PR preflight

The automated PR preflight checks provide a summary of the service impact, tests run, missing tests, model/migration changes, and critical-path warnings. This ensures that the PR is thoroughly reviewed and validated before it reaches the human reviewers.

Architecture-rule enforcement

Architecture-rule enforcement involves custom lints and structural tests that encode the CTO's taste mechanically. Once a rule is written down, it never gets argued in PR comments again, ensuring consistency and adherence to the desired architectural standards.

Evaluation framework infrastructure

Evaluation framework infrastructure includes pre-merge eval gating, experiments against curated datasets, production trajectory monitoring, and all wired together. This ensures that the evaluation process is robust and effective, allowing for continuous improvement and monitoring of the system's performance.