LLM Platform Engineer

About the role

At AllSpice, we're building the automation engine that powers the next generation of circuit design, enabling breakthroughs in smart vehicles, robotics, rockets, IoT, medical devices, and more. We're creating the agile development environment for hardware engineers with a Git-friendly translation layer and automated CI/CD framework for native circuit designs. Think GitHub/GitLab + Copilot for electronics. Our mission is to bring the same modern workflows that revolutionized software—version control, automation, and collaboration—to the hardware world.

Responsibilities

• Partner with engineers and product managers to discover workflow bottlenecks, prioritize the highest-impact problems, and own solutions end-to-end—from requirements and architecture to rollout and iteration.
• Own the evaluation and observability layer for our AI products, including automated and human review workflows, regression detection, and comparisons across models, prompts, and pipelines; make quality, cost, latency, usage, and customer outcomes visible and actionable.
• Build usable internal applications and self-service tools that help engineers and product managers inspect AI results, investigate failures, and provide feedback; where useful, extend these capabilities into customer-facing experiences.
• Build and maintain scripts, reliable data pipelines, and integrations that connect internal systems, third-party services, LLMs, and Model Context Protocol (MCP) tools, with attention to data quality, schema evolution, and customer data privacy.
• Write clear documentation and communicate findings to both technical and non-technical audiences, including leadership and customers.
• Champion code quality, testing, and reliability across the tooling stack through reviews, mentorship, and thoughtful design.
• Example projects include building an internal evaluation workspace for DRCY, developing review workflows for AI-generated findings, creating data pipelines that aggregate evaluation results and telemetry, building scripting tools to streamline Gen AI team workflows, designing RAG-based information repositories, integrating with CI/CD systems to surface AI insights, and partnering with infrastructure to measure cache effectiveness.
• As an aspirational next step, contribute directly to core AllSpice product features—for example, integrating AI review findings and feedback into existing engineering workflows—in partnership with the product engineering team.

Requirements

• 4+ years of software engineering experience with a focus on tooling, data platforms, or full-stack web development, with demonstrated senior-level ownership of systems or products that other people depend on.
• Strong SQL skills and experience designing data pipelines and storage schemas, handling data quality and schema evolution, and using data/analytics tools (e.g., Jupyter, Metabase, Looker, or similar BI platforms).
• Proficiency in Python scripting and automation for data pipelines, integrations, and internal tools.
• Experience building web applications or customer-facing tools using modern frameworks (e.g., Vue, React, TypeScript).
• Practical experience shipping or maintaining LLM-enabled systems and building infrastructure around them, such as evaluation pipelines, observability, or integrations; understanding of non-determinism, quality and reliability challenges, and tradeoffs across cost, latency, and model performance.
• Excellent communication skills — able to translate technical findings into clear insights for engineers, product managers, and customers alike.
• A customer-first mindset: think about the end-user experience and proactively seek feedback to improve the tools you build.
• Project management skills with the ability to lead and collaborate with varied stakeholders to move projects forward.
• Comfort with ambiguity and autonomy: independently identify problems, propose solutions, make pragmatic build-vs.-buy and architectural decisions, and bring stakeholders along in a fast-moving environment.
• (Preferred) Availability to work out of our flex offices in San Francisco or Boston on a hybrid basis (2–3 times/week), but we are flexible for the right candidate.

Qualifications

• Bachelor's degree or higher in a technology-related field, or equivalent practical experience.

Skills

• Python, TypeScript, SQL
• Business analytics and data visualization tools
• Langchain, Pydantic AI, and LLM evaluation frameworks
• GitHub Actions development and CI/CD
• Common web architectures, APIs, and front-end frameworks (Vue, React)
• Monitoring and observability tools (e.g., Langfuse, DataDog, Grafana)
• Project management and stakeholder communication

Benefits

• Supportive and smart colleagues
• Flexible work
• Opportunity to make a big impact
• Competitive salary & equity
• Health, dental, vision
• Generous PTO
• Home office stipend

Pay

Compensation Range: $160K - $190K