Senior Applied Scientist, Amazon AWS Agentic AI, AWS AI Fundamental Research
Key job responsibilities
The Senior Applied Scientist will lead the design and development of agentic evaluation frameworks and evaluation/critic model training that assess the quality and effectiveness of AI agents at scale. They will define evaluation methodologies, create benchmarks, and build evaluation models and automated systems that measure agent performance across critical dimensions. The scientist will stay at the forefront of the rapidly evolving field by studying and adopting state-of-the-art methods, conducting original research to advance the science of agent and evaluation. They will own the end-to-end lifecycle from research and data curation through model training to production deployment, working closely with engineering to deliver evaluation capabilities as managed AWS services. They will collaborate with cross-functional stakeholders to translate science insights into actionable improvements, mentor junior scientists, and contribute to the broader research community.
Basic Qualifications
- 3+ years of building machine learning models for business application experience
- PhD, or Master's degree and 6+ years of applied research experience
- Experience programming in Java, C++, Python or related language
- Experience with neural deep learning methods and machine learning
Preferred Qualifications
- Experience with modeling tools such as R, scikit-learn, Spark MLLib, MxNet, Tensorflow, numpy, scipy etc.
- Experience with large scale distributed systems such as Hadoop, Spark etc.