Jobs · Engineering · California

Backend Engineer, AI Systems

A1 · Palo Alto, CA · 2 wk ago
HybridEngineeringFull-time

Role

RoleAs a Backend Engineer, AI, you own the inference and orchestration layer that powers every AI interaction in the product. Your work sits between models and users, where latency, correctness, reliability, and cost directly impact real-world experience. Build and operate production systems that turn model capability into fast, stable, observable APIs used across mobile and desktop clients. FocusBuild and operate backend systems that serve AI-powered features in production.Design inference pipelines and orchestration layers that handle multi-step workflows, tool calls, and retriesManage the full lifecycle of AI requests: routing, caching, batching, streaming, and state managementOptimize latency, throughput, and cost across model inference and downstream systemsDesign systems that remain reliable despite non-deterministic model behavior and external dependenciesImplement observability for AI systems, including logging, tracing, and debugging of model outputs and failuresCollaborate with ML and product teams to translate model capabilities into stable, production-grade APIs

Ideal Experiences

  • Strong backend engineering fundamentals in production environments.
  • Experience running high-throughput, low-latency services.
  • Familiarity with AI inference patterns (LLMs, embeddings, multimodal).
  • Comfortable debugging distributed systems under load.
  • Bias toward shipping and learning from production behavior.

Outcomes

  • Backend systems run reliably at scale, handling production AI traffic with low latency and high throughput.
  • Multi-step AI workflows complete successfully across tools and services, with robust handling of failures and retries.
  • APIs are stable, clear, and support seamless integration with frontend and ML systems.
  • Production incidents are quickly detected, diagnosed, and resolved, minimizing user impact.
  • Iterative improvements based on real usage continuously increase system performance and reliability.
  • System design evolves to support increasing scale, complexity, and new AI capabilities without major rewrites.

How We Work

  • The best products today in the world were built by small, world class teams.
  • We are a high talent density and hands-on team.
  • We make decisions collectively, move at rapid speed, striking a balance between shipping high quality work and learning.
  • Joining our team requires the ability to bring structure, exercise judgment, and execute independently.
  • Our goal is to put in hands of our users a truly magical product.

Interview Process

  • If there appears to be a fit, we'll reach out to schedule 3, but no more than 4 interviews.
  • Applications are evaluated by our technical team members.
  • Interviews will be conducted via virtual meetings and/or onsite.
  • We value transparency and efficiency, so expect a prompt decision.
  • If you've demonstrated the exceptional skills and mindset we're looking for, we'll extend an offer to join us.

Similar jobs

Backend/AI Engineer

ReductoSan Francisco, CA· 3 wk ago
Information Technology$100/hrapply on jobs.ashbyhq.com

Backend Engineer - AI

GTMfundSan Francisco, CA· Yesterday
Engineering$125k–$200k/yrapply on jobs.ashbyhq.com

Backend Engineer - AI

AnythingSan Francisco, CA· 1 wk ago
Engineering$125k–$200k/yrapply on jobs.ashbyhq.com

AI Backend Engineer

KlaritySan Francisco, CA· 1 mo ago
Engineering$201k–$240k/yrapply on jobs.ashbyhq.com

AI Engineer - Backend

TraversalNew York, NY· 5 mo ago
Engineering$150k–$300k/yrapply on jobs.ashbyhq.com