Data Scientist, Security Issue Management
About the role
The Security Tooling team designs and builds high-performance AI systems using LLMs and machine learning that identify builder bottlenecks, automate security workflows, and optimize the software development lifecycle—empowering engineering teams worldwide to ship secure code faster while maintaining the highest security standards. As a Data Scientist on our Security Tool team, you will focus on building state-of-the-art ML models to enhance builder experience and productivity. You will identify builder bottlenecks and pain points across the software development lifecycle, design and apply experiments to study developer behavior, and measure the downstream impacts of security tooling on engineering velocity and code quality. Our team rewards curiosity while maintaining a laser-focus on bringing products to market that empower builders while maintaining security excellence. Competitive candidates are responsive, flexible, and able to succeed within an open, collaborative, entrepreneurial, startup-like environment. At the forefront of both academic and applied research in builder experience and security automation, you have the opportunity to work together with a diverse and talented team of scientists, engineers, and product managers and collaborate with other teams. This role offers a unique opportunity to work on projects that could fundamentally transform how builders interact with security tools and how organizations balance security requirements with developer productivity.
Key job responsibilities
- Design and run rigorous experiments to evaluate and improve security tooling performance, builder experience, and adoption across hundreds of thousands of builders, multiple security tools, and diverse business verticals.
- Lead the end-to-end lifecycle of data science and ML models — from research and experimentation through production launch — including defining success metrics, obtaining stakeholder sign-off, and managing rollout.
- Conduct online and offline analyses to measure the real-world impact of security tooling improvements beyond adoption metrics, including downstream effects on vulnerability resolution, builder productivity, and organizational security posture.
- Develop and deploy production-grade machine learning and statistical models using Python, SQL, and related tools to automate insights, detect patterns, and drive decision-making across STF's security tool ecosystem.
- Perform large-scale exploratory data analysis on builder feedback, ticket resolution, tool usage, and customer satisfaction data to uncover patterns, identify opportunities, and inform product and tooling decisions.
- Translate complex research findings into clear insights and recommendations for technical and non-technical stakeholders at all levels, including STF leadership metric reporting and customer satisfaction publications.
- Contribute to Amazon's scientific community and the broader research field through collaboration and publication in top-tier venues.
A day in the life
Morning — Review overnight pipeline health: nudge systems, ticket classification models, and adoption dashboards running as expected. Join daily standup with the SDI team to align on priorities and flag blockers. Dive into exploratory analysis — investigating a spike in unresolved tickets or segmenting builder feedback to understand adoption gaps.
Midday — Partner with security tool owners (e.g., Shepherd, Talos, Scorecard) to review experiment results — did the latest nudge improve resolution rates? Translate findings into actionable recommendations for leadership reviews or WBR updates. Analyze CSAT survey data to surface emerging dissatisfaction themes.
Afternoon — Write production code — building features for the classification pipeline, optimizing SQL for the metrics scorecard, or iterating on a model for predicting resolution timelines. Collaborate with STF stakeholders to define success metrics for an upcoming model launch. Document findings, update trackers, and queue next steps.
Basic Qualifications
- 2+ years of data scientist experience
- 3+ years of data querying languages (e.g. SQL), scripting languages (e.g. Python) or statistical/mathematical software (e.g. R, SAS, Matlab, etc.) experience
- 3+ years of machine learning/statistical modeling data analysis tools and techniques, and parameters that affect their performance experience
- 1+ years of guiding and coaching a group of researchers experience
- 1+ years of working with or evaluating AI systems experience
- 1+ years of creating or contributing to mathematical textbooks, research papers, or educational content experience
- Master's degree in Science, Technology, Engineering, or Mathematics (STEM), or experience working in Science, Technology, Engineering, or Mathematics (STEM)
- Experience applying theoretical models in an applied environment
Preferred Qualifications
- Ph.D. in Science, Technology, Engineering, or Mathematics (STEM)
- Knowledge of machine learning concepts and their application to reasoning and problem-solving
- Experience in Python, Perl, or another scripting language
- Experience in a ML or data scientist role with a large technology company
- Experience in defining and creating benchmarks for assessing GenAI model performance
- Experience working on multi-team, cross-disciplinary projects
- Experience applying quantitative analysis to solve business problems and making data-driven business decisions
- Experience effectively communicating complex concepts through written and verbal communication
Pay & compensation
USA, WA, Seattle — 136,000.00 - 184,000.00 USD annually. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location.
Benefits
- Health insurance (medical, dental, vision, prescription)
- Basic Life & AD&D insurance and option for Supplemental life plans
- Employee Assistance Program (EAP) and Mental Health Support
- Medical Advice Line
- Flexible Spending Accounts
- Adoption and Surrogacy Reimbursement coverage
- 401(k) matching
- Paid time off
- Parental leave