Sr. Product Manager, AI

About the role

Pantomath is hiring a Senior Product Manager for AI with direct, hands-on experience building agentic systems to own the strategy and execution of our AI product surface. This role sits squarely at the intersection of product leadership and applied AI engineering.

Responsibilities

• Own the AI product strategy and roadmap, making informed, technical decisions about which workflows an agent should own end to end, which it should assist, and which it should stay out of.
• Leverage your applied AI background to evaluate feasibility, failure modes, and long-term maintainability of agentic features.
• Define and defend the agent boundary: what the agent is trusted to do autonomously, what requires human confirmation, and how that boundary widens as trust is earned.
• Translate customer needs, technical constraints, and model capabilities into clear, detailed PRDs that engineers can execute against without ambiguity.
• Partner closely with engineering to design, build, evaluate, and ship agents that meet strict standards for accuracy, latency, cost, observability, and security.
• Build and own the evaluation harness: define golden datasets, quality rubrics, and regression testing so that model and prompt changes ship on evidence rather than anecdote.
• Replicate real-world customer data environments and incident scenarios to validate agent behavior, and ensure production readiness.
• Define success metrics for AI features (resolution accuracy, acceptance rate, time to root cause, deflection, token cost per outcome) and actively monitor, diagnose, and iterate based on data and feedback.
• Work with Go-to-Market teams to support launch readiness, technical enablement, and customer-facing documentation, including how to explain what the agent knows and why it is right.
• Stay current on trends and best practices across foundation models, agent frameworks, MCP and tool-calling standards, retrieval, and evaluation to inform roadmap decisions.
• Advocate for a best-in-class experience for data engineers and analysts, ensuring AI capabilities are transparent, correctable, and trustworthy at scale.

Requirements

• Hands-on experience building with LLMs in a technical role (e.g., shipping production AI features, building agent workflows, prompt and context engineering, running model evaluations).
• 3+ years of product management experience working on technically complex products such as AI/ML platforms, data platforms, developer tools, or APIs.
• Strong, working knowledge of: Foundation model capabilities, limitations, and cost/latency trade-offs Agent architecture patterns: tool calling, context assembly, retrieval, orchestration, and guardrails Evaluation methodology for non-deterministic systems Demonstrated judgment about when AI is the right answer and when it is not.
• Proven ability to partner deeply with engineering teams and make informed trade-offs.
• Able to clearly communicate complex technical concepts to both technical and non-technical stakeholders.
• Bachelor's degree in Computer Science, Data Science, Information Systems, Engineering, or a related technical field (or equivalent practical experience).

Preferred Qualifications

• Experience with MCP, agent frameworks, or embedding AI into third-party surfaces such as Teams, Slack, ServiceNow, or Jira.
• Prior ownership of an AI or agent product within a B2B SaaS or data product.
• Familiarity with data observability, lineage, governance, or metadata platforms.
• Hands-on background in data engineering or data architecture, or experience building AI on top of enterprise data systems.
• Experience in a high-growth SaaS or developer-focused product environment.