Engineering Product Manager

The application window is expected to close on 10/12/2026.

About the role

The Cisco AI Software & Platform Group incubates and delivers mission‑critical Generative AI and platform infrastructure across Cisco's global product portfolio. The group builds foundational evaluation, benchmarking, and observability platforms that empower engineering teams to deploy autonomous AI agents and LLM‑powered systems safely, reliably, and with verified accuracy at enterprise scale. As an Engineering Product Manager you will lead the strategy, architecture, and organizational adoption of Cisco's end‑to‑end evaluation platform.

Responsibilities

• Define AI Evaluation Platform Strategy: Architect the product vision, staged roadmap, and platform architecture for Cisco's unified offline benchmarking suites, automated LLM judges, and continuous production evaluation systems.
• Operationalize Quality & Safety Standards: Convert ambiguous quality standards (e.g., factual accuracy, hallucination mitigation, safety policies, agentic tool‑use fidelity) into objective, repeatable test suites and structured human‑in‑the‑loop review pipelines.
• Establish Release Gating & Quality Bars: Establish pre‑deployment quality bars, risk‑based release criteria, and automated regression‑detection loops to govern AI model and application rollouts.
• Bridge Offline & Production Telemetry: Connect offline model evaluations directly with live telemetry, product analytics (Amplitude), and real‑time user feedback loops to rapidly detect and remediate performance regressions.
• Drive Multi‑Team Platform Adoption: Partner deeply with research scientists, engineering leaders, and product managers across business units to drive organization‑wide adoption of standardized evaluation frameworks.

Requirements

• Bachelor's degree in Computer Science, Engineering, Mathematics, or a related technical field with 12+ years of product management experience, OR Master's degree with 8+ years, OR PhD with 5+ years of experience.
• Prior experience leading product management for AI/ML platforms, model evaluation frameworks, Large Language Model (LLM) infrastructure, or data‑intensive software products.
• Applied background designing and operationalizing offline benchmarking systems, automated evaluation judges, or real‑time production quality monitoring pipelines.
• Previous experience defining risk‑based launch criteria, release‑gating frameworks, and regression mitigation workflows for software or AI systems.
• Proven experience driving multi‑team platform adoption across matrixed engineering and product organizations.

Qualifications

• Direct experience evaluating Generative AI, Retrieval‑Augmented Generation (RAG) pipelines, and autonomous multi‑agent systems.
• Familiarity with modern AI evaluation toolkits (e.g., DeepEval, Ragas, LangSmith, TruLens) and enterprise telemetry architectures.
• Prior experience operating across enterprise SaaS, cloud infrastructure, or hyperscaler environments.
• Demonstrated background in statistical evaluation methodologies, automated LLM‑as‑a‑judge calibration, and human‑in‑the‑loop annotation workflows.
• Practical experience with online evals for agent systems: observing production traces and trajectories to diagnose failure patterns and decide which techniques to apply to make the agent work better.

Benefits

• Medical, dental and vision insurance.
• 401(k) plan with Cisco matching contribution.
• Paid parental leave.
• Short and long‑term disability coverage.
• Basic life insurance.
• Eligibility for grants of Cisco restricted stock units (RSUs) that vest with continued employment.
• Paid time off: 10 paid holidays per year + 1 floating holiday for non‑exempt employees, 1 paid birthday‑day off, paid year‑end holiday shutdown, 4 paid personal‑wellness days.
• Vacation: non‑exempt employees receive 16 days per year (accrued 4.92 h/pay period); exempt employees participate in flexible vacation program with no defined limit.
• Sick time: 80 hours provided on hire date and each Jan 1, with up to 80 hours carry‑forward.
• Optional