Senior Efficacy Research and Psychometrics Analyst, Khan Kids (24 months fixed-term)
Khan Academy · San Mateo, CA · 1 mo ago
Analyst$138k–$155k/yrContract
We need to output HTML fragment with sections: About the role, Responsibilities, Requirements (or Qualifications?), Benefits, Pay, Schedule? The posting includes "About Khan Academy", "About Khan Academy Kids", "OUR COMMUNITY", "THE ROLE", "Responsibilities", "What You Bring", "Nice To Have", "Perks And Benefits", "More About Us", "Compensation band". Also includes "About Khan Academy" etc. We must not repeat job title as heading. Use for section headings (e.g., About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule). We need to include sections only when posting has real specific content. So we need to decide which sections to include.
We have content: About Khan Academy (mission, platform). About Khan Academy Kids (program). OUR COMMUNITY (diversity). THE ROLE description. Responsibilities (two bullet groups). What You Bring (qualifications). Nice To Have (early childhood interest). Perks And Benefits (list). Compensation band (pay). No explicit schedule. No explicit benefits beyond perks. So we can have sections: About the organization? Not in list but we could use "About the role"? Actually "About the role" is a heading we can use for THE ROLE description. Could also have "About Khan Academy" and "About Khan Academy Kids" maybe combine into "About the organization"? But the instruction says use headings like About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule. It doesn't forbid other headings? It says use
for section headings (e.g., About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule). We need to include sections only when posting has real specific content. So we need to decide which sections to include.
We have content: About Khan Academy (mission, platform). About Khan Academy Kids (program). OUR COMMUNITY (diversity). THE ROLE description. Responsibilities (two bullet groups). What You Bring (qualifications). Nice To Have (early childhood interest). Perks And Benefits (list). Compensation band (pay). No explicit schedule. No explicit benefits beyond perks. So we can have sections: About the organization? Not in list but we could use "About the role"? Actually "About the role" is a heading we can use for THE ROLE description. Could also have "About Khan Academy" and "About Khan Academy Kids" maybe combine into "About the organization"? But the instruction says use headings like About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule. It doesn't forbid other headings? It says use for section headings (e.g. About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule). Likely they expect those headings only. But we can also include other headings if needed? Safer to stick to those. We need to map content.
- About the role: could include THE ROLE description plus maybe About Khan Academy and Kids? But "About the role" should describe the role. We'll put THE ROLE content there.
- Responsibilities: we have responsibilities listed.
- Requirements / Qualifications: we have "What You Bring" (qualifications) and maybe "Nice To Have" could be under Skills? Or we could have Qualifications for required, Skills for nice-to-have. The instruction says use headings like Requirements, Qualifications, Skills. We can decide: Requirements for mandatory qualifications (advanced degree, grounding, SQL, etc). Qualifications maybe similar. But we can combine: Use Requirements for required qualifications, Skills for nice-to-have or technical skills. However we also have "Perks And Benefits" -> Benefits heading. Pay heading for compensation band. Schedule not present, so omit.
We must not include placeholder text like "commensurate with experience". We have actual pay range.
We need to preserve all substantive information; do not invent or summarize away detail. So we need to include details from each section.
We must remove boilerplate: Notice To Applicants (scam warning), "More About Us" (TED talks, team link, company values) maybe considered boilerplate? It's not exactly boilerplate but it's generic company info. The instruction: Remove cookie/navigation/legal/EEO/equal-opportunity/scam-warning boilerplate, duplicated lines, and application instructions. The "More About Us" includes values etc. Might be considered boilerplate? It's not legal/EEO but it's still generic company values. Might be okay to keep? The instruction says remove boilerplate, but not explicitly to remove company values. However "More About Us" includes Sal's TED talk links and values; could be considered extra but not required. The instruction: "Preserve ALL substantive information; do NOT invent or summarize away detail." The values might be substantive? They are generic but still part of posting. However they might be considered boilerplate. Safer to omit them? The instruction says remove cookie/navigation/legal/EEO/equal-opportunity/scam-warning boilerplate. Values not listed. But "More About Us" might be considered extra but not required. I'd keep it maybe as a section "About the organization"? But we don't have heading for that. Could incorporate into About the role? Might be too much.
Better to follow the instruction: keep only sections that map to the allowed headings and contain substantive role-specific info. So we will include:
- About the role: THE ROLE description (maybe also include brief about Khan Academy Kids? but that's about the product, could be part of role). We'll keep THE ROLE.
- Responsibilities: the bullet list.
- Requirements: the "What You Bring" (required qualifications). We'll list as - .
- Skills: the "Nice To Have" (early childhood interest) maybe also include proficiency with BI tools etc? Those are already in requirements. We'll put nice-to-have as Skills.
- Benefits: the "Perks And Benefits" list.
- Pay: the compensation band.
We must not include schedule.
We need to output only HTML fragment, no markdown fences.
Let's extract exact text.
About the role section: From "THE ROLE" paragraph:
"Khan Academy Kids is looking for a Senior Efficacy Research and Psychometrics Analyst (24 months fixed-term) to strengthen the evidence base behind our assessments and learning content for children in PreK through 2nd grade. This role spans two complementary areas. First, you will lead the technical side of our psychometrics and efficacy research: serving as the primary point of contact for our external research partners, managing recurring data extracts, analyzing and interpreting psychometric findings, and helping translate research into product improvements. Second, you will own analytics and reporting for Khan Academy Kids: building and maintaining our KPI dashboard and preparing the recurring reporting that our leadership, philanthropy, and school partnerships teams rely on to make decisions and tell our impact story. You will work on a small, interdisciplinary team alongside product managers, engineers, designers, and academic experts and collaborate closely with leading external research organizations. We are looking for someone with strong quantitative training — ideally in psychometrics, educational measurement, or a related field — who is energized by using rigorous evidence to improve learning experiences for young children."
We need to keep that as prose inside
.
Responsibilities: There are two headings: "Psychometrics & Efficacy Research" and "Analytics Reporting and Data Infrastructure". Under each there are bullet-like sentences starting with verbs. We need to convert to
- . Let's extract each bullet.
Psychometrics & Efficacy Research:
- Lead our psychometric research partnership: Serve as the primary technical point of contact for our external psychometric research partnership, supporting study design, data sharing, psychometric analyses, and interpretation of findings
- Extract, clean, and securely share assessment data with research partners on a recurring schedule
- Analyze and interpret psychometric findings, including item difficulty, score distributions, growth analyses, concurrent validity, and IRT-based scoring
- Partner with our product and engineering teams to translate findings into product improvements, such as updates to scoring models and assessment discontinuation rules
- Collaborate with research partners to develop and publish technical documentation, beginning with the technical manual for our PreK assessment
- Recruit and manage research partner sites: Recruit and onboard school and district sites for psychometric and efficacy studies; execute MOUs; align on consent procedures and participant incentives; and coordinate site visits for direct child assessment data collection
- Partner with our legal team to draft and negotiate data-sharing agreements with districts and research organizations, including requirements for pseudonymized and identified data
- Support efficacy research. Coordinate Khan Academy Kids' participation in ongoing efficacy studies and contribute to the design of future studies, aligning internal and external stakeholders on the research approach
Analytics Reporting and Data Infrastructure:
- Own Khan Kids’s KPI dashboard: Partner with our engineering team to build and maintain the Khan Academy Kids analytics dashboard, including integrations with App Store Connect and Google Play Console. Maintain the dashboard so it can be used by other teams including marketing, school partnerships, and philanthropy.
- Lead recurring reporting: Prepare recurring app performance reporting, including monthly and quarterly business reviews and periodic target setting. Field ad hoc data requests across teams — for example, writing new SQL queries to support targeted district outreach or grant reporting.
We need to keep each as
- .
Now Requirements (What You Bring):
- An advanced degree (PhD or Master's) in psychometrics, educational measurement, quantitative methods or a related field — or equivalent applied experience
- Strong grounding in psychometric methods, including item analysis, reliability and validity evidence, growth modeling, and Item Response Theory (IRT)
- Proficiency with SQL and a statistical programming language (e.g., R or Python) for extracting, cleaning, analyzing, and visualizing data
- Experience with (or eagerness to learn) business intelligence tools such as Looker, and comfort building dashboards and data pipelines
- Familiarity with responsible data-sharing practices in education research, including student data privacy considerations (e.g., FERPA, COPPA) and work with pseudonymized and deidentified data
- Excellent communication skills: able to translate complex psychometric findings into clear implications for product teams, funders, and school partners
- Strong project-management and organizational skills, with the ability to coordinate multiple external partners, recurring deliverables, and data-sharing schedules with close attention to detail
- Proven cross-cultural competency skills, demonstrating self-awareness, awareness of others, and the ability to adopt inclusive perspectives, attitudes, and behaviors to drive inclusion and belonging throughout the organization
- Motivated by Khan Academy’s mission “to provide a free, world-class education for anyone, anywhere”
We'll put these as
- under
Requirements
.
Skills (Nice To Have):
- Experience with (or strong interest in) early childhood education and assessment of young learners is a plus
We'll put as - under
Skills
.
Benefits (Perks And Benefits):
List:
- Competitive salaries
- Ample paid time off as needed – Your well-being is a priority
- 8 pre-scheduled Wellness Days in 2026 occurring on a Monday or a Friday for a 3-day weekend boost
- Remote-first culture - that caters to your time zone, with open flexibility as needed, at times
- Generous parental leave
- An exceptional team that trusts you and gives you the freedom to do your best
- The chance to put your talents towards a deeply meaningful mission and the opportunity to work on high-impact products that are already defining the future of education
- Opportunities to connect through affinity, ally, and social groups
- And we offer all those other typical benefits as well: 401(k) + 4% matching & comprehensive insurance, including medical, dental, vision, and life
We need to keep as - .
Pay:
- The compensation band for this role is $137,871 - $155,105 USD annually for candidates based in the United States and $186,306 - $209,595 CAD annually for candidates based in Canada.
- The pay range for this position is a general guideline only. The salary offered will depend on internal pay equity and the candidate’s relevant skills, experience, qualifications, and job market data.
We'll put as
maybe.
We must not include schedule.
Now ensure we don't include any boilerplate like "Notice To Applicants", "About Khan Academy", "About Khan Academy Kids", "OUR COMMUNITY", "More About Us", etc.
We must not repeat job title as heading. We are not using job title as heading.
Now produce HTML:
About the role
...
Responsibilities
- ...
...
Requirements
...
Skills
...
Benefits
...
Pay
...
Make sure to preserve all details exactly as in text, preserving punct
- .
- Skills: the "Nice To Have" (early childhood interest) maybe also include proficiency with BI tools etc? Those are already in requirements. We'll put nice-to-have as Skills.
- Benefits: the "Perks And Benefits" list.
- Pay: the compensation band.
We must not include schedule.
We need to output only HTML fragment, no markdown fences.
Let's extract exact text.
About the role section: From "THE ROLE" paragraph:
"Khan Academy Kids is looking for a Senior Efficacy Research and Psychometrics Analyst (24 months fixed-term) to strengthen the evidence base behind our assessments and learning content for children in PreK through 2nd grade. This role spans two complementary areas. First, you will lead the technical side of our psychometrics and efficacy research: serving as the primary point of contact for our external research partners, managing recurring data extracts, analyzing and interpreting psychometric findings, and helping translate research into product improvements. Second, you will own analytics and reporting for Khan Academy Kids: building and maintaining our KPI dashboard and preparing the recurring reporting that our leadership, philanthropy, and school partnerships teams rely on to make decisions and tell our impact story. You will work on a small, interdisciplinary team alongside product managers, engineers, designers, and academic experts and collaborate closely with leading external research organizations. We are looking for someone with strong quantitative training — ideally in psychometrics, educational measurement, or a related field — who is energized by using rigorous evidence to improve learning experiences for young children."
We need to keep that as prose inside
. Responsibilities: There are two headings: "Psychometrics & Efficacy Research" and "Analytics Reporting and Data Infrastructure". Under each there are bullet-like sentences starting with verbs. We need to convert to
- . Let's extract each bullet. Psychometrics & Efficacy Research: - Lead our psychometric research partnership: Serve as the primary technical point of contact for our external psychometric research partnership, supporting study design, data sharing, psychometric analyses, and interpretation of findings - Extract, clean, and securely share assessment data with research partners on a recurring schedule - Analyze and interpret psychometric findings, including item difficulty, score distributions, growth analyses, concurrent validity, and IRT-based scoring - Partner with our product and engineering teams to translate findings into product improvements, such as updates to scoring models and assessment discontinuation rules - Collaborate with research partners to develop and publish technical documentation, beginning with the technical manual for our PreK assessment - Recruit and manage research partner sites: Recruit and onboard school and district sites for psychometric and efficacy studies; execute MOUs; align on consent procedures and participant incentives; and coordinate site visits for direct child assessment data collection - Partner with our legal team to draft and negotiate data-sharing agreements with districts and research organizations, including requirements for pseudonymized and identified data - Support efficacy research. Coordinate Khan Academy Kids' participation in ongoing efficacy studies and contribute to the design of future studies, aligning internal and external stakeholders on the research approach Analytics Reporting and Data Infrastructure: - Own Khan Kids’s KPI dashboard: Partner with our engineering team to build and maintain the Khan Academy Kids analytics dashboard, including integrations with App Store Connect and Google Play Console. Maintain the dashboard so it can be used by other teams including marketing, school partnerships, and philanthropy. - Lead recurring reporting: Prepare recurring app performance reporting, including monthly and quarterly business reviews and periodic target setting. Field ad hoc data requests across teams — for example, writing new SQL queries to support targeted district outreach or grant reporting. We need to keep each as
- .
Now Requirements (What You Bring):
- An advanced degree (PhD or Master's) in psychometrics, educational measurement, quantitative methods or a related field — or equivalent applied experience
- Strong grounding in psychometric methods, including item analysis, reliability and validity evidence, growth modeling, and Item Response Theory (IRT)
- Proficiency with SQL and a statistical programming language (e.g., R or Python) for extracting, cleaning, analyzing, and visualizing data
- Experience with (or eagerness to learn) business intelligence tools such as Looker, and comfort building dashboards and data pipelines
- Familiarity with responsible data-sharing practices in education research, including student data privacy considerations (e.g., FERPA, COPPA) and work with pseudonymized and deidentified data
- Excellent communication skills: able to translate complex psychometric findings into clear implications for product teams, funders, and school partners
- Strong project-management and organizational skills, with the ability to coordinate multiple external partners, recurring deliverables, and data-sharing schedules with close attention to detail
- Proven cross-cultural competency skills, demonstrating self-awareness, awareness of others, and the ability to adopt inclusive perspectives, attitudes, and behaviors to drive inclusion and belonging throughout the organization
- Motivated by Khan Academy’s mission “to provide a free, world-class education for anyone, anywhere”
We'll put these as
- under
Requirements
. Skills (Nice To Have): - Experience with (or strong interest in) early childhood education and assessment of young learners is a plus We'll put as- under
Skills
. Benefits (Perks And Benefits): List: - Competitive salaries - Ample paid time off as needed – Your well-being is a priority - 8 pre-scheduled Wellness Days in 2026 occurring on a Monday or a Friday for a 3-day weekend boost - Remote-first culture - that caters to your time zone, with open flexibility as needed, at times - Generous parental leave - An exceptional team that trusts you and gives you the freedom to do your best - The chance to put your talents towards a deeply meaningful mission and the opportunity to work on high-impact products that are already defining the future of education - Opportunities to connect through affinity, ally, and social groups - And we offer all those other typical benefits as well: 401(k) + 4% matching & comprehensive insurance, including medical, dental, vision, and life We need to keep as- .
Pay:
- The compensation band for this role is $137,871 - $155,105 USD annually for candidates based in the United States and $186,306 - $209,595 CAD annually for candidates based in Canada.
- The pay range for this position is a general guideline only. The salary offered will depend on internal pay equity and the candidate’s relevant skills, experience, qualifications, and job market data.
We'll put as
maybe. We must not include schedule. Now ensure we don't include any boilerplate like "Notice To Applicants", "About Khan Academy", "About Khan Academy Kids", "OUR COMMUNITY", "More About Us", etc. We must not repeat job title as heading. We are not using job title as heading. Now produce HTML:
About the role
...
Responsibilities
- ... ...
Requirements
- ...
Skills
- ...
Benefits
- ...
Pay
...
Make sure to preserve all details exactly as in text, preserving punct
- .
Pay:
- The compensation band for this role is $137,871 - $155,105 USD annually for candidates based in the United States and $186,306 - $209,595 CAD annually for candidates based in Canada.
- The pay range for this position is a general guideline only. The salary offered will depend on internal pay equity and the candidate’s relevant skills, experience, qualifications, and job market data.
We'll put as
- under
- under