Jobs · Research

Senior Efficacy Research and Psychometrics Analyst, Khan Kids (24 months fixed-term)

CHEManager International · San Mateo, CA · 1 mo ago
Research$138k–$155k/yrContract
We need to output HTML fragment with sections for headings that have real content. Must not repeat job title. Must not include boilerplate like notice to applicants, EEO, etc. Must include sections: About the role? Actually the posting includes "ABOUT KHAN ACADEMY", "ABOUT KHAN ACADEMY KIDS", "OUR COMMUNITY", "THE ROLE", "Responsibilities", "What You Bring", "Nice To Have", "Perks And Benefits", "More About Us", "OUR COMPANY VALUES". We need to map to allowed headings: About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule. We can only use those headings. We need to decide which sections map. We have content: "THE ROLE" description (overview). Could map to About the role. Responsibilities: we have a Responsibilities section with bullet points. Requirements/Qualifications/Skills: we have "What You Bring" (requirements) and "Nice To Have" (nice-to-have). Could combine into Requirements and Skills? The instruction: use headings for section headings (e.g. About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule). So we can have multiple of those. We need to include only when posting has real specific content. We have Pay: compensation band given. So we can include Pay section. Benefits: we have Perks And Benefits section with list. So Benefits. Schedule: not mentioned (no schedule info). So omit. We also might have Qualifications separate from Requirements? Could treat "What You Bring" as Requirements (or Qualifications). "Nice To Have" could be Skills? But better: Requirements = mandatory qualifications; Skills = maybe technical skills? However we have both mandatory and nice-to-have. Could split: Requirements = advanced degree, strong grounding, proficiency SQL, etc. Skills = familiarity with BI tools, data-sharing practices, communication, project-management, cross-cultural competency, motivation. Nice To Have could be extra but we could omit or include as Skills? The instruction: prefer a few substantive sections over many thin ones. So maybe combine Requirements and Qualifications into one section? But we have both headings allowed; we could use Requirements for mandatory, Qualifications for nice-to-have? Actually Qualifications often similar to Requirements. But we can decide: Use Requirements for mandatory qualifications (advanced degree, psychometric methods, SQL, etc). Use Skills for technical and soft skills (BI tools, data-sharing practices, communication, project-management, cross-cultural, mission motivation). Use Benefits for perks. Use Pay for compensation. Use About the role for role overview. Use Responsibilities for responsibilities. We must not include filler like "commensurate with experience". Not present. We must remove boilerplate: notice to applicants, EEO, etc. Also remove duplicated lines, navigation, legal, etc. Also remove "More About Us" and "OUR COMPANY VALUES" as they are not in allowed headings; we should omit them unless we can map to allowed headings (maybe About the role? but that's already used). Better to omit. We must preserve all substantive information. So we need to include all content from sections we keep. Let's extract content: About Khan Academy (mission, platform, learners). About Khan Academy Kids (program, ages, characters). Our Community (diversity). The Role description (two complementary areas). Responsibilities (two sub-sections: Psychometrics & Efficacy Research, Analytics Reporting and Data Infrastructure). What You Bring (list of requirements). Nice To Have (early childhood education interest). Perks And Benefits (list). Compensation band. We need to decide what to put in About the role: likely the description of the role (THE ROLE). Could also include brief about Khan Academy and Kids? But those are about organization, not role. The instruction: About the role heading for section headings (e.g. About the role, Responsibilities...). So About the role should contain role description. We can also include brief about Khan Academy Kids as context? Might be considered part of role? But better to keep About the role as the role description only. However we must preserve ALL substantive information; if we omit the About Khan Academy and Kids sections, we lose substantive info about organization. But maybe those are considered boilerplate? They are not boilerplate; they describe the organization. The instruction says we can omit cookie/navigation/legal/EEO/equal-opportunity/scam-warning boilerplate, duplicated lines, and application instructions. It does not say to omit organizational description. So we should keep that info. But we have no heading for "About the company". Allowed headings list does not include About the company. However we could incorporate that info into About the role section? Might be okay to include a brief paragraph about the organization under About the role. The instruction: "Use

for section headings (e.g. About the role, Responsibilities, Requirements, Qualifications, Skills, Benefits, Pay, Schedule)." It doesn't forbid other headings, but says to use those for section headings. Likely they expect only those headings. So we need to fit all substantive content under those headings. We can put organizational info under About the role (maybe as a preamble). Or we could create a separate section like "About Khan Academy" but not allowed. So we must embed. Thus About the role will contain: description of Khan Academy, Khan Academy Kids, community, and the role overview. Responsibilities: as given. Requirements: mandatory qualifications from What You Bring. Qualifications: maybe we could treat Nice To Have as Qualifications? Or we could put Nice To Have under Skills? Actually Skills heading could contain technical skills like SQL, R/Python, BI tools, etc. But those are already in Requirements. Hmm. Better: Use Requirements for mandatory qualifications (advanced degree, psychometric methods, SQL, etc). Use Skills for technical and soft skills (BI tools, data-sharing practices, communication, project-management, cross-cultural, mission motivation). Use Qualifications maybe for Nice To Have? But Qualifications similar to Requirements. Could also omit Qualifications if not needed. The instruction: prefer a few substantive sections over many thin ones. So we could have About the role, Responsibilities, Requirements, Benefits, Pay. That's five sections. That seems reasonable. We can put all mandatory and nice-to-have under Requirements (or split). But we need to avoid filler. We'll include both mandatory and nice-to-have under Requirements, maybe separate with a note? But we can just list them all. But we must not repeat job title as heading. That's fine. We must not include Pay/Schedule/Benefits unless posting actually states them. We have Pay and Benefits. Schedule not present, omit. Now we need to format lists with
  • . For prose use

    . We need to ensure we preserve all substantive information: that includes the bullet points under Responsibilities, What You Bring, Nice To Have, Perks And Benefits, compensation band. We also need to preserve the description of the role (two complementary areas). Also the description of Khan Academy, Khan Academy Kids, Our Community. Let's extract exact text: From posting: "ABOUT KHAN ACADEMYKhan Academy is a nonprofit with the mission to deliver a free, world-class education to anyone, anywhere. Our proven learning platform offers free, high-quality supplemental learning content and practice that cover Pre-K - 12th grade and early college core academic subjects, focusing on math and science. We have over 181 million registered learners globally and are committed to improving learning outcomes for students worldwide, focusing on learners in historically under-resourced communities." Note there is no space after the heading; but we can parse. "ABOUT KHAN ACADEMY KIDSKhan Academy Kids is a free and fun learning program for children ages 2 to 8. Kodi Bear and a cast of animated characters lead children on a personalized education journey filled with fun, standards-aligned activities in early literacy, math, executive functioning and social emotional skills. Built by a small but mighty team, Khan Academy Kids is used by millions of children in homes and classrooms around the world." "OUR COMMUNITYOur students, teachers, and parents come from all walks of life, and so do we. Our team includes people from academia, traditional/non-traditional education, big tech companies, and tiny startups. We hire great people from diverse backgrounds and experiences because it makes our company stronger. We value diversity, equity, inclusion, and belonging as necessary to achieve our mission and impact the communities we serve. We know that transforming education starts in-house with learning about ourselves and our colleagues. We strive to be world-class in investing in our people and commit to developing you as a professional." "THE ROLE Khan Academy Kids is looking for a Senior Efficacy Research and Psychometrics Analyst (24 months fixed-term) to strengthen the evidence base behind our assessments and learning content for children in PreK through 2nd grade. This role spans two complementary areas. First, you will lead the technical side of our psychometrics and efficacy research: serving as the primary point of contact for our external research partners, managing recurring data extracts, analyzing and interpreting psychometric findings, and helping translate research into product improvements. Second, you will own analytics and reporting for Khan Academy Kids: building and maintaining our KPI dashboard and preparing the recurring reporting that our leadership, philanthropy, and school partnerships teams rely on to make decisions and tell our impact story. You will work on a small, interdisciplinary team alongside product managers, engineers, designers, and academic experts and collaborate closely with leading external research organizations. We are looking for someone with strong quantitative training - ideally in psychometrics, educational measurement, or a related field - who is energized by using rigorous evidence to improve learning experiences for young children." Responsibilities section: "Responsibilities Psychometrics & Efficacy ResearchLead our psychometric research partnership: Serve as the primary technical point of contact for our external psychometric research partnership, supporting study design, data sharing, psychometric analyses, and interpretation of findingsExtract, clean, and securely share assessment data with research partners on a recurring scheduleAnalyze and interpret psychometric findings, including item difficulty, score distributions, growth analyses, concurrent validity, and IRT-based scoringPartner with our product and engineering teams to translate findings into product improvements, such as updates to scoring models and assessment discontinuation rulesCollaborate with research partners to develop and publish technical documentation, beginning with the technical manual for our PreK assessmentRecruit and manage research partner sites: Recruit and onboard school and district sites for psychometric and efficacy studies; execute MOUs; align on consent procedures and participant incentives; and coordinate site visits for direct child assessment data collectionPartner with our legal team to draft and negotiate data-sharing agreements with districts and research organizations, including requirements for pseudonymized and identified dataSupport efficacy research. Coordinate Khan Academy Kids' participation in ongoing efficacy studies and contribute to the design of future studies, aligning internal and external stakeholders on the research approach Analytics Reporting and Data InfrastructureOwn Khan Kids's KPI dashboard: Partner with our engineering team to build and maintain the Khan Academy Kids analytics dashboard, including integrations with App Store Connect and Google Play Console. Maintain the dashboard so it can be used by other teams including marketing, school partnerships, and philanthropy.Lead recurring reporting: Prepare recurring app performance reporting, including monthly and quarterly business reviews and periodic target setting. Field ad hoc data requests across teams - for example, writing new SQL queries to support targeted district outreach or grant reporting." What You Bring: "What You Bring An advanced degree (PhD or Master's) in psychometrics, educational measurement, quantitative methods or a related field - or equivalent applied experienceStrong grounding in psychometric methods, including item analysis, reliability and validity evidence, growth modeling, and Item Response Theory (IRT)Proficiency with SQL and a statistical programming language (e.g., R or Python) for extracting, cleaning, analyzing, and visualizing dataExperience with (or eagerness to learn) business intelligence tools such as Looker, and comfort building dashboards and data pipelinesFamiliarity with responsible data-sharing practices in education research, including student data privacy considerations (e.g., FERPA, COPPA) and work with pseudonymized and deidentified dataExcellent communication skills: able to translate complex psychometric

Similar jobs

Senior Psychometrician

National Board for Certification in Occupational Therapy (NBCOT)Gaithersburg, MD· 4 wk ago
RemoteHealthcare$105k–$135k/yrapply on nationalboardforcertificationinoccupationaltherapyinc.applytojob.com

Senior Psychometrician

RTI InternationalUnited States· 1 mo ago
RemoteHealthcare$120k–$185k/yrapply on careers.rti.org

Senior Psychometrician

RTI Health SolutionsRaleigh, NC· 1 mo ago
Healthcareapply on globalcareers-rtiinc.icims.com

Senior Psychometrician

Western Governors UniversitySalt Lake City Metropolitan Area· 2 mo ago
Healthcare$89k–$133k/yrapply on wgu.wd5.myworkdayjobs.com

Senior Psychometrician

NBMEPhiladelphia, PA· 2 mo ago
RemoteHealthcare$139k–$173k/yrapply on nbme.applicantpro.com