Applied Scientist, Artificial General Intelligence, Frontier AI
for section headings.
Use - for any list of duties/requirements/benefits, and
for prose.
We need to preserve all substantive information; do not invent or summarize away detail.
We need to ensure we don't create sections with filler like "commensurate with experience". Not present.
We need to include sections only when posting has real specific content.
Thus sections: About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay.
We need to decide what goes where.
About the role: maybe a paragraph summarizing role: "The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems."
That's from first sentence.
Responsibilities: list bullet points from responsibilities and day in the life.
We need to capture all responsibilities:
- Collaborate closely with core scientist team developing Amazon Nova models.
- Lead development of comprehensive quality strategies and auditing frameworks that safeguard integrity of data collection workflows.
- Design auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks.
- Perform expert-level manual audits.
- Conduct meta-audits to evaluate auditor performance.
- Provide targeted coaching to uplift overall quality capabilities.
- Develop and maintain LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment.
- Set up configuration of data collection workflows and communicate quality feedback to stakeholders.
- Enhance customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services.
- Support quality solution design.
- Conduct root cause analysis on data quality issues.
- Research new auditing methodologies.
- Find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards.
- Work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
We can group into bullet list.
Requirements (Basic Qualifications):
- Master's degree in computer science, mathematics, statistics, machine learning or equivalent quantitative field.
- Experience programming in Java, C++, Python or related language.
- Experience with SQL and an RDBMS (e.g., Oracle) or Data Warehouse.
Qualifications (Preferred Qualifications):
- Experience implementing algorithms using both toolkits and self-developed code.
- Have publications at top-tier peer-reviewed conferences or journals.
Skills: Could list: machine learning, statistics, quality assurance, auditing methodologies, automated evaluation systems, LLM-as-a-Judge, designing judge architectures, creating evaluation rubrics, building ML models, data collection workflow configuration, root cause analysis, research auditing methodologies, optimizing data quality, quality assurance best practices, collaboration with engineers/domain experts/vendors.
But we need to ensure we don't invent; we can pull from text: "robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems", "designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment", "set up the configuration of data collection workflows", "conduct root cause analysis on data quality issues", "research new auditing methodologies", "find innovative ways of optimizing data quality", "setting examples for the team on quality assurance best practices and standards", "work closely with talented engineers, domain experts, and vendor teams". So we can list those as skills.
Benefits: list:
- Sign-on payments and restricted stock units (RSUs).
- Comprehensive health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance, option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage).
- 401(k) matching.
- Paid time off.
- Parental leave.
We could also mention "Learn more about our benefits at https://amazon.jobs/en/benefits." but that's a link; maybe we can include as prose.
Pay: "The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. USA, WA, BELLEVUE - 136,000.00 - 184,000.00 USD annually"
We need to preserve all substantive info. So include that sentence.
We should not repeat sign-on payments etc if already in Benefits; but it's okay.
We need to avoid duplication? The instruction: "Preserve ALL substantive information; do NOT invent or summarize away detail." Duplication is okay if it's in original; but we might want to avoid repeating same info in two places? Not prohibited. But better to keep each piece once.
We can put sign-on payments and RSUs under Benefits; Pay section can mention base salary range and that final compensation based on experience etc.
We need to ensure we don't include boilerplate like EEO, accommodation, etc.
Now construct HTML.
We'll have:
About the role
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems.
Responsibilities
- Collaborate closely with core scientist team developing Amazon Nova models.
- Lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows.
- Design auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performance on benchmarks.
- Perform expert-level manual audits.
- Conduct meta-audits to evaluate auditor performance.
- Provide targeted coaching to uplift overall quality capabilities.
- Develop and maintain LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment.
- Set up the configuration of data collection workflows and communicate quality feedback to stakeholders.
- Enhance customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services.
- Support quality solution design.
- Conduct root cause analysis on data quality issues.
- Research new auditing methodologies.
- Find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards.
- Work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
Requirements
- Master's degree in computer science, mathematics, statistics, machine learning or equivalent quantitative field.
- Experience programming in Java, C++, Python or related language.
- Experience with SQL and an RDBMS (e.g., Oracle) or Data Warehouse.
Qualifications
- Experience implementing algorithms using both toolkits and self-developed code.
- Have publications at top-tier peer-reviewed conferences or journals.
Skills
- Machine learning
- Statistics
- Quality assurance
- Auditing methodologies
- Automated evaluation systems
- LLM-as-a-Judge systems (designing judge architectures, creating evaluation rubrics, building ML models)
- Data collection workflow configuration
- Root cause analysis
- Researching auditing methodologies
- Optimizing data quality
- Quality assurance best practices and standards
- Collaboration with engineers, domain experts, and vendor teams
Benefits
- Sign-on payments and restricted stock units (RSUs)
- Comprehensive health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance, option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage)
- 401(k) matching
- Paid time off
- Parental leave
Pay
The base salary range for this position is USA, WA, BELLEVUE - 136,000.00 - 184,000.00 USD annually. Final compensation will be determined based on factors including experience, qualifications, and location. Your Amazon package will include sign-on payments and restricted stock units (RSUs).
We need to ensure we didn't include Schedule section.
Check for any leftover boilerplate: we removed EEO, accommodation, etc.
Check for duplicate content: sign-on payments and RSUs appear both in Benefits and Pay; that's okay as it's in original. But we might want to avoid duplication? The instruction says preserve ALL substantive information; duplication is not prohibited. However, we could mention sign-on payments and RSUs only once, but then we would lose that info from Pay section if we remove. But Pay section already mentions base salary range and final compensation; we could keep sign-on payments and RSUs only in Benefits. However, the Pay paragraph originally includes "Your Amazon package will include sign-on payments and restricted stock units (RSUs)." If we remove that, we lose that piece of info from Pay section
for prose. We need to preserve all substantive information; do not invent or summarize away detail. We need to ensure we don't create sections with filler like "commensurate with experience". Not present. We need to include sections only when posting has real specific content. Thus sections: About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay. We need to decide what goes where. About the role: maybe a paragraph summarizing role: "The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems." That's from first sentence. Responsibilities: list bullet points from responsibilities and day in the life. We need to capture all responsibilities: - Collaborate closely with core scientist team developing Amazon Nova models. - Lead development of comprehensive quality strategies and auditing frameworks that safeguard integrity of data collection workflows. - Design auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performances on benchmarks. - Perform expert-level manual audits. - Conduct meta-audits to evaluate auditor performance. - Provide targeted coaching to uplift overall quality capabilities. - Develop and maintain LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment. - Set up configuration of data collection workflows and communicate quality feedback to stakeholders. - Enhance customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services. - Support quality solution design. - Conduct root cause analysis on data quality issues. - Research new auditing methodologies. - Find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards. - Work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice. We can group into bullet list. Requirements (Basic Qualifications): - Master's degree in computer science, mathematics, statistics, machine learning or equivalent quantitative field. - Experience programming in Java, C++, Python or related language. - Experience with SQL and an RDBMS (e.g., Oracle) or Data Warehouse. Qualifications (Preferred Qualifications): - Experience implementing algorithms using both toolkits and self-developed code. - Have publications at top-tier peer-reviewed conferences or journals. Skills: Could list: machine learning, statistics, quality assurance, auditing methodologies, automated evaluation systems, LLM-as-a-Judge, designing judge architectures, creating evaluation rubrics, building ML models, data collection workflow configuration, root cause analysis, research auditing methodologies, optimizing data quality, quality assurance best practices, collaboration with engineers/domain experts/vendors. But we need to ensure we don't invent; we can pull from text: "robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems", "designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment", "set up the configuration of data collection workflows", "conduct root cause analysis on data quality issues", "research new auditing methodologies", "find innovative ways of optimizing data quality", "setting examples for the team on quality assurance best practices and standards", "work closely with talented engineers, domain experts, and vendor teams". So we can list those as skills. Benefits: list: - Sign-on payments and restricted stock units (RSUs). - Comprehensive health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance, option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage). - 401(k) matching. - Paid time off. - Parental leave. We could also mention "Learn more about our benefits at https://amazon.jobs/en/benefits." but that's a link; maybe we can include as prose. Pay: "The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. USA, WA, BELLEVUE - 136,000.00 - 184,000.00 USD annually" We need to preserve all substantive info. So include that sentence. We should not repeat sign-on payments etc if already in Benefits; but it's okay. We need to avoid duplication? The instruction: "Preserve ALL substantive information; do NOT invent or summarize away detail." Duplication is okay if it's in original; but we might want to avoid repeating same info in two places? Not prohibited. But better to keep each piece once. We can put sign-on payments and RSUs under Benefits; Pay section can mention base salary range and that final compensation based on experience etc. We need to ensure we don't include boilerplate like EEO, accommodation, etc. Now construct HTML. We'll have:
About the role
The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Scientist with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to ensure the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems.
Responsibilities
- Collaborate closely with core scientist team developing Amazon Nova models.
- Lead the development of comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows.
- Design auditing strategies with detailed SOPs, quality metrics, and sampling methodologies that help Nova improve performance on benchmarks.
- Perform expert-level manual audits.
- Conduct meta-audits to evaluate auditor performance.
- Provide targeted coaching to uplift overall quality capabilities.
- Develop and maintain LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment.
- Set up the configuration of data collection workflows and communicate quality feedback to stakeholders.
- Enhance customer experiences through high-quality training and evaluation data that powers state-of-the-art LLM products and services.
- Support quality solution design.
- Conduct root cause analysis on data quality issues.
- Research new auditing methodologies.
- Find innovative ways of optimizing data quality while setting examples for the team on quality assurance best practices and standards.
- Work closely with talented engineers, domain experts, and vendor teams to put quality strategies and automated judging systems into practice.
Requirements
- Master's degree in computer science, mathematics, statistics, machine learning or equivalent quantitative field.
- Experience programming in Java, C++, Python or related language.
- Experience with SQL and an RDBMS (e.g., Oracle) or Data Warehouse.
Qualifications
- Experience implementing algorithms using both toolkits and self-developed code.
- Have publications at top-tier peer-reviewed conferences or journals.
Skills
- Machine learning
- Statistics
- Quality assurance
- Auditing methodologies
- Automated evaluation systems
- LLM-as-a-Judge systems (designing judge architectures, creating evaluation rubrics, building ML models)
- Data collection workflow configuration
- Root cause analysis
- Researching auditing methodologies
- Optimizing data quality
- Quality assurance best practices and standards
- Collaboration with engineers, domain experts, and vendor teams
Benefits
- Sign-on payments and restricted stock units (RSUs)
- Comprehensive health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance, option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage)
- 401(k) matching
- Paid time off
- Parental leave
Pay
The base salary range for this position is USA, WA, BELLEVUE - 136,000.00 - 184,000.00 USD annually. Final compensation will be determined based on factors including experience, qualifications, and location. Your Amazon package will include sign-on payments and restricted stock units (RSUs).
We need to ensure we didn't include Schedule section. Check for any leftover boilerplate: we removed EEO, accommodation, etc. Check for duplicate content: sign-on payments and RSUs appear both in Benefits and Pay; that's okay as it's in original. But we might want to avoid duplication? The instruction says preserve ALL substantive information; duplication is not prohibited. However, we could mention sign-on payments and RSUs only once, but then we would lose that info from Pay section if we remove. But Pay section already mentions base salary range and final compensation; we could keep sign-on payments and RSUs only in Benefits. However, the Pay paragraph originally includes "Your Amazon package will include sign-on payments and restricted stock units (RSUs)." If we remove that, we lose that piece of info from Pay section