Founding Research Engineer

About the role

Nolla Health is building AI doctors and healthcare applications, including Nolla Skin and NollaMD, to make high-quality healthcare accessible. The Founding Research Engineer will develop clinical evaluation benchmarks, grading infrastructure, clinician data workflows, and research studies while collaborating with the founding team and post-training partners.

Responsibilities

• Design and maintain the benchmarks we use to judge clinical accuracy, safety, and documentation quality
• Extend them to our agentic system: tool use, long-horizon tasks that span many visits, memory of a patient's history, uploaded records, labs, and escalation behavior
• Build the grading stack: deterministic checks, rubric graders, and LLM judges calibrated against clinician ratings
• Improve the production harness the product runs on
• Turn real, consented cases into eval cases and training examples, with clinician review, de-identification, and provenance built in
• Run the clinician review workflows that produce rubrics, labels, and feedback at scale
• Lead our clinical evaluations from protocol through publication, and publish our benchmarks for the field
• Collaborate with post-training on reward design and data curation, and run experiments where it helps