Product Engineer - AI
Alma · San Francisco Bay Area · 2 wk ago
On-siteEngineering$15/hrFull-time
Responsibilities
- Work closely with the founding team and early customers to build a world-class AI-powered product.
- Own AI-powered features end-to-end — design the agentic pipelines and LLM integrations, build the backend services that orchestrate them, create the frontend that exposes them, and ship to production.
- Design, build, and iterate on RAG pipelines, agentic workflows, prompt chains, and evaluation frameworks that power our core product.
- Develop and maintain robust evaluation and observability systems — you’ll define how we measure whether our AI outputs are actually good, and build the tooling to catch regressions before customers do.
- Partner with our designer and product team to translate user needs into shipped AI-powered experiences — not just APIs or model outputs that “someone else will integrate later.”
- Operate as a force multiplier with AI tooling — use coding agents to move 5–10x faster than traditional engineers without sacrificing quality, and continuously evolve how our team builds, reviews, and tests software.
- Engage in the entire application lifecycle with a high bar for coding, debugging, testing, and deploying.
Role Requirements
- 2+ years of experience building production AI/LLM-powered features — not research prototypes, but systems that real users depend on.
- You’ve dealt with hallucination mitigation, latency constraints, cost optimization, and the messy reality of shipping AI to production.
- Strong fundamentals in RAG architectures, prompt engineering, agentic frameworks, and LLM orchestration (e.g., multi-step chains, tool use, structured output extraction).
- Experience designing evaluation and quality systems for AI outputs — you’ve built evals, human-in-the-loop review flows, or regression detection pipelines, and you have opinions about what “good enough” means for production AI.
- Proficient in Python, FastAPI, PostgreSQL, containerization and scaling microservices.
- Comfortable in (or excited to ramp on) TypeScript, React, and modern frontend tooling. You don’t need to be a CSS wizard on day one, but you need to be the kind of engineer who’ll learn React deeply rather than punt it to someone else.
- Daily, hands-on use of modern coding harnesses (Claude Code, Codex, Cursor, or equivalent) — not as autocomplete, but as a primary way of shipping production code.
- Comfortable building and consuming MCP servers to integrate internal tools, data, and workflows into your dev loop.
- Strong point of view on AI-assisted code review, automated testing, and how to keep quality high when shipping at agent speed.
- Enjoy working in a fast-paced environment & wear multiple hats — including ones outside your historical comfort zone.
- Ability to seek & find the right resources for solving open-ended problems.
- Located in the San Francisco Bay Area or willing to relocate.
- BS/MS in Computer Science, Engineering, or a related technical field.
Nice-to-haves
- Experience working in a small startup environment (Seed or Series A).
- Deep understanding of transformer architectures, attention mechanisms, and GPU inference — you can reason about why a model behaves the way it does, not just prompt around it.
- Experience with fine-tuning, RLHF, or distillation workflows.
- Experience with vector databases, embedding models, and retrieval optimization (hybrid search, reranking, chunking strategies).
- Familiarity with AI observability and tracing tools (LangSmith, Braintrust, or similar).
- Experience with AWS (Bedrock, SageMaker, or similar managed AI infra), Supabase, LaunchDarkly & Betterstack.
- Built your own MCP servers, custom agents, or internal AI dev tooling.
- Published, blogged, or shared opinions about modern AI-assisted development workflows.
- Genuine full-stack experience — comfortable across backend, frontend, and infra.