Senior Product Manager Technical, AWS Applied AI Solutions - Core Services
Key job responsibilities
Own the product vision, strategy, and roadmap for AI agent evaluation capabilities, including automated evaluation pipelines, human review workflows, and quality benchmarking tools
Write detailed product requirements that articulate evaluation framework APIs, scoring algorithms, and integration patterns
Partner with applied scientists and engineers to translate research into production-grade product features
Drive customer discovery to generalize evaluation requirements from diverse agent architectures, domains, and deployment patterns
Define success metrics and instrumentation strategies that make evaluation product adoption, coverage, and impact visible to stakeholders
Own go-to-market strategy, including positioning evaluation tooling relative to existing AWS services and third-party alternatives
Drive alignment across dependent teams (identity, observability, data platform) to ensure evaluation signals integrate seamlessly with the broader Core Services AI Foundations platform
Key accountabilities
Own the product vision, strategy, and roadmap for AI agent evaluation capabilities, including automated evaluation pipelines, human review workflows, and quality benchmarking tools
Write detailed product requirements that articulate evaluation framework APIs, scoring algorithms, and integration patterns
Partner with applied scientists and engineers to translate research into production-grade product features
Drive customer discovery to generalize evaluation requirements from diverse agent architectures, domains, and deployment patterns
Define success metrics and instrumentation strategies that make evaluation product adoption, coverage, and impact visible to stakeholders
Own go-to-market strategy, including positioning evaluation tooling relative to existing AWS services and third-party alternatives
Drive alignment across dependent teams (identity, observability, data platform) to ensure evaluation signals integrate seamlessly with the broader Core Services AI Foundations platform
About the team
AWS is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses.
Work/Life Balance: We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why flexible work hours and arrangements are part of our culture.
Inclusive Team Culture: Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empowers us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity and AmazeCon conferences, inspire us to never stop embracing our uniqueness.
Qualifications
- Bachelor's degree
- Experience owning/driving roadmap strategy and definition
- Experience with feature delivery and tradeoffs of a product
- Experience contributing to engineering discussions around technology decisions and strategy related to a product
- Experience managing technical products or online services
- Experience in representing and advocating for a variety of critical customers and stakeholders during executive-level prioritization and planning