Machine Learning Scientist - Open Source Lead

About the role

Arena Intelligence is seeking a Machine Learning Scientist to lead our open-source research. Key responsibilities include:

• Design, run, and share new methods and experiments to reveal what makes models useful, trustworthy, and capable.
• Develop new metrics, methodologies, and evaluation protocols that go beyond traditional benchmarks.
• Analyze large-scale human voting and interaction data to uncover insights into model performance and user preferences.
• Communicate results with the broader research community via academic papers, educational content, conference talks.
• Collaborate with engineers to implement and scale research findings into production systems.
• Prototype and test research ideas rapidly, balancing rigor with iteration speed.
• Partner with model providers to shape evaluation questions and support responsible model testing.
• Contribute to the scientific integrity and transparency of the LMArena leaderboard and tools.

Requirements

Qualifications include:

• Hands-on experience training large-scale models, including reward models, preference models, and fine-tuning LLMs with methods like RLHF, DPO, and contrastive learning.
• Strong foundation in ML and statistics, with a track record of designing novel training objectives, evaluation schemes, or statistical frameworks to improve model reliability and alignment.
• Fully experienced in the full experimental stack, from dataset design and large-batch training to rigorous evaluation and ablation.
• Deeply collaborative mindset, working closely with engineers to productionize research insights and iterating with product teams to align research with user needs.

Skills

Desired skills include:

• Fluent in the full experimental stack, from dataset design and large-batch training to rigorous evaluation and ablation.
• Comfortable being a visible representative of Arena Intelligence, engaging openly with the research community, and building a strong personal brand to help shape AI research culture.
• Strong understanding of LLMs and modern deep learning architectures (e.g., Transformers, diffusion models, reinforcement learning with human feedback).
• Proficiency in Python and ML research libraries such as PyTorch, JAX, or TensorFlow.
• Demonstrated ability to design and analyze experiments with statistical rigor.
• Experience publishing research or working on open-source projects in ML, NLP, or AI evaluation.
• Ability to translate research questions into practical systems and collaborate across engineering and product teams.
• Passion for open science, reproducibility, and community-driven research.

Bonus points

• Skilled at public speaking, writing, and presenting research work to diverse audiences.
• Actively participates in conferences, panels, and online forums to foster relationships and thought leadership.
• Builds trust through transparent communication and consistent community engagement.
• Serves as a go-to contact for external researchers, journalists, and partners.