Lead Assessment Developer

What You’ll Be Doing

• Manage the full assessment development lifecycle at Outsmart — from item authoring through field testing, standard setting, scoring, and reporting.
• Work closely with psychometricians, faculty, product managers, and engineers to build rigorous, scalable assessments.
• Help shape how AI tools are responsibly woven into our development and QA workflows as an AI-forward organization.

Who You Are

• Master's degree (Ph.D. preferred) in educational measurement, assessment, or a related field.
• 3+ years managing operational, high-stakes assessment lifecycles end-to-end.
• Proven experience leading standard-setting studies and managing diverse SME panels.
• Experience managing external vendors for field tests and pilot studies.
• Familiarity with validity, reliability, and fairness standards (e.g., AERA/APA/NCME).
• Familiarity with cognitive complexity frameworks (DOK, Bloom's).
• Strong communicator who can translate psychometric concepts for non-technical audiences.
• Bonus: knowledge of CTT, IRT, CDM, DIF, AI-assisted item generation, or accreditation reviews.

What You’ll Own

• End-to-end assessment development operations for assigned assessments.
• SME network — recruitment, training, calibration, and quality.
• Standard-setting studies — method selection, panel facilitation, and final reporting.
• Vendor and external partner relationships, SLAs, and security protocols.
• Field testing and pilot operations.
• Scoring operations for constructed-response and performance-based assessments.
• Project timelines, risk identification, and stakeholder communication.
• Core assessment documentation — specs, blueprints, training materials, technical reports.

What Helps You Succeed

• You thrive in ambiguity and are energized by building from scratch.
• You balance psychometric rigor with startup pragmatism — you know when to ship and iterate.
• You communicate with unusual clarity across technical and non-technical audiences.
• You take ownership without being asked and surface risks early.
• You're excited about AI's role in assessment and know where validation is non-negotiable.
• You care deeply about fairness and the learners behind every score.

Salary Range

$120K-$180K

Perks

• Competitive salary, equity, and benefits.
• Healthcare for you and your dependents (medical, dental, and vision).
• Parental leave for any full-time employee (birthing, non-birthing, and adoptive parents).
• 15 PTO days per year, a winter break, and 12+ paid company holidays per year.
• Remote work days.
• Lunch provided in-office daily.
• Working with super talented people who want to make a difference in the world.

About the Role

Outsmart is a mission-driven, early-stage startup reimagining higher education for the age of AI by creating an entirely new collegiate experience that expands access and opportunity. Backed by $38M from top venture firms including Khosla Ventures, Forerunner, DST Global Partners, and Lightspeed, the company is led by former Duolingo executives and a founding team that also includes Emmy Award winners. With a strong culture centered around high-impact, detail-oriented team members, Outsmart is already seeing significant traction, generating 1–5 million weekly organic views on social media.