Sr Applied Scientist, Support Agent Intelligence and Evaluation

About the role

Amazon Advertising is one of Amazon's fastest growing and most profitable businesses. The role sits within Amazon Advertising's broader Agentic Intelligence organization, a community of multiple Applied Science and Engineering teams building the next generation of AI-powered experiences for advertisers.

Responsibilities

• Enhance support agent capabilities across the broad suite of Amazon Advertising products — expanding coverage, depth of resolution, and advertiser task completion across Sponsored Products, Sponsored Brands, DSP, AMC, and more
• Design and own the evaluation framework for agent quality — including automated LLM-based scoring of answer correctness, confidence calibration, and conversation-level resolution signals
• Develop novel metrics that capture whether advertisers actually got the help they needed (beyond surface-level deflection rates)
• Build and improve retrieval and generation models that power real-time advertiser interactions under strict latency SLAs
• Partner with product, engineering, and business teams to productize research and inform roadmap decisions with data

Qualifications

• 3+ years of building machine learning models for business application experience
• PhD, or Master's degree and 6+ years of applied research experience
• Experience programming in Java, C++, Python or related language
• Experience with neural deep learning methods and machine learning
• Experience with modeling tools such as R, scikit-learn, Spark MLLib, MxNet, Tensorflow, numpy, scipy etc.
• Experience with large scale distributed systems such as Hadoop, Spark etc.
• Experience in online or digital advertising