Staff AI Product Analyst

Angi At a Glance

• Founded in 1995 as Angie’s List and rebranded in 2021
• Global company with 9 brands in 8 countries and employees worldwide
• Homeowners have turned to us for 300 million home projects and counting

About The Team

The Staff AI Product Analyst will define AI-native product analytics for Angi, turning visionary technology into prioritized use cases, robust evaluation disciplines, and a trusted data ecosystem.

• This role is central to building Angi’s agentic product analytics capability—defining where AI-native analytics creates real leverage, designing the evaluation frameworks that earn organizational trust, and establishing repeatable system standards.
• The ideal candidate is a systems-thinking data expert who bridges the gap between deep AI capabilities and practical business outcomes.

What You’ll Do

• AI Product Strategy & Metrics Design
• Flagship Journey Measurement: Partner with AI product, engineering, and domain experts to define north-star and guardrail metrics for flagship AI journeys, focusing on adoption, task completion, quality, and downstream business outcomes.
• Causal Framing & Experimentation: Align closely with data science partners on test design, interpretation, and causal framing when new AI products ship.
• Data Layer Systems Thinking: Map how agents interact with metrics, raw tables, documentation, and APIs to ensure requirements for data and analytics engineering are explicit, structured, and testable.
• Evaluation, Governance & Backlog Management
• Eval Framework Architecture: Design and maintain formal evaluation sets, "golden datasets," rubrics, and regression checks for agent outputs to guarantee accuracy, completeness, policy safety, and usefulness.
• Taxonomy & Failure Mode Analysis: Systematically categorize wrong answers, tool misuse, context gaps, and data edge cases, feeding those insights back into prompts, retrieval (RAG), tools, and data modeling pipelines.
• Performance Monitoring: Monitor live usage of Agentic AI analytics workflows—tracking success, abandonment, fallback patterns, and human override rates to maintain a prioritized engineering improvement backlog.
• Cross-Functional Leadership & Enablement
• Playbooks & Narrative: Own internal documentation defining when to use agents, required human checkpoints, and how to communicate uncertainty. Support enablement and leadership storytelling around agentic AI analytics.
• Strategic Collaboration: Partner with business leaders, the Principal Analytics Engineer, and data engineering teams to translate complex AI capabilities into measurable business outcomes.
• Defining Success: Deliver recurring measurement artifacts (dashboards, reviews, or written readouts) that leadership uses to prioritize roadmaps, while continuously reducing high-severity incidents where agents produce incorrect metrics.

Who You Are

• Minimum Qualifications
• 10+ years of experience across product analytics, data science, or technical product management within a complex, high-scale operating environment.
• 1+ years of direct, hands-on experience working with AI/LLM systems in an applied context, with a specific focus on evaluating, trusting, and deploying them responsibly.
• Deep technical proficiency with Python, ML frameworks, data visualization/analytics platforms, and standard data management/governance practices.
• Proven track record designing AI evaluation frameworks, including formal evals, golden datasets, production monitoring, or equivalent rigor in a related data discipline.
• Exceptional communication skills with a demonstrated ability to translate highly technical AI insights and tradeoffs into clear narratives for non-technical business stakeholders.

Preferred Qualifications

• Experience building or evaluating agentic analytics workflows, orchestration frameworks, and familiarity with multi-agent systems.
• Demonstrated ability to operate fluidly in ambiguity, run structured experiments or pilots, and default to fast action and iteration.
• A systems-oriented mindset with a healthy skepticism of AI hype, focusing strictly on organizational outcomes and driving technical investments that yield true business value.

Compensation & Benefits

• The salary band for this position ranges from $165,000 - $225,000 commensurate with experience and performance.
• Compensation may vary based on factors such as cost of living.
• This position will be eligible for a competitive year end performance bonus & equity package.
• Full medical, dental, vision package to fit your needs
• Flexible vacation policy; work hard and take time when you need it
• Pet discount plans & retirement plan with company match (401K)