Psychometrician

WIDA is an educational services organization within UW-Madison’s School of Education that advances language development and academic achievement for multilingual children in early childhood and grades K-12. We provide educators worldwide with high-quality language standards, assessments, professional learning, and research.

About the role

The Psychometrician is located within WIDA’s Assessment Department and reports to the Psychometrics Manager. This position supports psychometric operational and research activities for WIDA’s language assessments, monitors and evaluates vendor-conducted psychometric projects, and contributes to research reports and psychometric deliverables. Key responsibilities include supporting the MODEL assessment program, automated scoring initiatives, test security analyses for ACCESS for ELLs, item bank maintenance, and quality assurance for ACCESS and Alternate ACCESS programs.

Remote work is an option on a voluntary basis; the role is headquartered in Madison, WI, and occasional work-related travel is required. WIDA operates in a Windows-based environment, and a Dell PC laptop will be provided.

Responsibilities

• Analyze, research, and propose solutions to psychometric data and problems; communicate solutions with internal and external stakeholders.
• Design and implement psychometric tools to assess learner ability and enhance learning outcomes in collaboration with stakeholders.
• Serve as a subject matter expert, disseminating technical psychometric concepts to non-technical audiences.
• Observe and evaluate learner performance, adjusting procedures as needed.
• Proctor psychological tests.
• Support psychometric operations and continuous improvement of the MODEL assessment program, including automated scoring initiatives.
• Conduct and review test security analyses for ACCESS for ELLs.
• Support item bank maintenance and quality assurance activities for ACCESS and Alternate ACCESS programs.
• Contribute to operational and research projects that support the validity, reliability, and fairness of WIDA assessments.

Requirements

• Ph.D. in Educational Measurement or a related field with an emphasis on psychometrics or quantitative methods.
• Minimum 1 year of experience in psychometrics theory, research, and statistics (e.g., classical test theory, item response theory, scaling and equating, sampling, generalizability theory, linear modeling, or structural equation modeling).
• Experience with data manipulation and statistical/psychometric analyses using Winsteps, SAS, R, or related software packages.
• Familiarity with multistage adaptive testing, Rasch model, and graded response IRT models.
• Ability to write technical and research reports.
• Ability to work independently and collaboratively within a team.
• Experience collaborating with external vendors or partners.

Preferred Qualifications

• Programming skills, including proficiency in Python or similar languages for psychometric modeling and statistical analysis.
• Experience applying artificial intelligence (AI) methods to assessment and measurement (e.g., natural language processing, automated scoring, machine learning).
• Test security-related analysis, training, or experience.
• Experience with K–12 educational assessments, language proficiency assessments, or large-scale standardized testing programs.

Benefits

• Generous vacation, holidays, and paid time off.
• Competitive health, dental, vision, and life insurance.
• Tax-advantaged savings accounts.
• Participation in the Wisconsin Retirement System (WRS) pension fund.

Pay

Minimum $80,000.00, commensurate with qualifications and experience.