Machine Learning Evaluation Specialist

About The Role

What if your years of research and domain expertise could directly shape how the next generation of AI models think and reason? That's exactly what this role offers. We're looking for ML experts and domain specialists to design fiendishly difficult evaluation problems that push state-of-the-art AI systems to their limits. This isn't about textbook prompts — it's about crafting the kind of challenges that only someone with deep, hard-earned expertise can build and judge. Your work will directly influence how leading AI models are measured, improved, and deployed.

Organization

Alignerr
Type: Hourly Contract
Location: Remote
Commitment: 10–40 hours/week

Who You Are

• Hold graduate-level expertise (MS or PhD preferred) in a scientific or technical domain intersecting with machine learning
• Have strong working knowledge of core ML methods — model selection, feature engineering, evaluation metrics, and pipeline design
• Be deeply familiar with active, open research problems in your field
• Be able to pinpoint exactly where general ML knowledge falls short and specialized domain insight becomes essential
• Have experience conducting or publishing original research (highly valued)
• Communicate complex ideas clearly and precisely in writing
• Be self-motivated and thrive when working independently on intellectually demanding problems

Example Domains (Not Exhaustive)

• Computational biology, genomics, or bioinformatics
• Climate science and environmental modeling
• Medical imaging and healthcare
• Materials science and computational chemistry
• Astrophysics and signal processing
• NLP for low-resource or specialized corpora
• Robotics, control theory, or reinforcement learning
• Financial modeling and quantitative analysis