Applied Scientist II, Core Shopping Data Science

About the role

Some customer experience (CX) shopping defects are straightforward to detect and track. The interesting ones are not because they depend on what a customer perceives. For example, a search page may return legitimately different results, yet a shopper has no way to tell them apart. We turn these ambiguous perception questions into measurable artifacts using LLMs, and we build the frameworks to know exactly where model judgment can be trusted and where a human must decide—proving it against ground truth at Amazon scale.

This role involves owning LLM-based measurement pipelines, extending them to other parts of the shopping experience (e.g., homepage, detail page), and designing measurement methodologies from scratch. The larger goal is to set the standard for LLM-driven quality labeling across Amazon, building reusable tooling that scales and is adopted by teams independently solving the same problems.

Responsibilities

• Design and improve LLM-based labeling for perception-driven defects, including prompt design, sampling strategy, and determining the split between model judgment and human annotation.
• Validate labeling quality against human ground truth and build/maintain golden datasets for validation.
• Extend perceived-duplicate measurement to other parts of the shopping experience, designing methodologies where none exist and evaluating existing approaches.
• Build reusable, production-grade labeling and evaluation tooling (batch inference, quality sampling, prompt/model version control) that operates on Amazon-scale data.
• Define and publish standards for using LLMs to measure customer experience, ensuring adoption by other teams.
• Partner with science, engineering, and product teams across Amazon Stores to make quality measurement actionable in their decisions.
• Present metric results and methodology changes to stakeholders.

Qualifications

• PhD or Master’s degree and 4+ years of experience in CS, CE, ML, or a related field.
• Experience programming in Java, C++, Python, or a related language.
• Experience in one or more of the following: algorithms and data structures, parsing, numerical optimization, data mining, parallel/distributed computing, or high-performance computing.

Preferred Qualifications

• Experience using Unix/Linux.
• Experience in professional software development.

Pay

Base salary range: $142,800 – $193,200 USD annually (Seattle, WA). Final compensation will include sign-on payments, restricted stock units (RSUs), and will vary based on experience, qualifications, and location.

Benefits

• Comprehensive health insurance (medical, dental, vision, prescription, Basic Life & AD&D, with optional supplemental life plans).
• Employee Assistance Program (EAP), mental health support, and medical advice line.
• Flexible Spending Accounts (FSAs).
• Adoption and surrogacy reimbursement coverage.
• 401(k) matching.
• Paid time off and parental leave.

Learn more about our benefits at amazon.jobs/en/benefits.