Machine Learning Research Engineer, Model Evaluation
WindBorne Systems · Palo Alto, CA · Today
On-siteEngineering$140k–$240k/yrFull-time
Responsibilities
- Evaluation strategy — Work with our Meteorology team to develop a rigorous, meteorologically valid strategy for comparing WeatherMesh with leading AI and physics-based models. Choose the metrics, datasets, baselines, and case studies that provide an honest picture of forecast quality.
- Fast feedback and durable systems — Build quick evaluations that give researchers useful signals, then turn recurring analyses into reliable, reusable infrastructure.
- Think systematically about reproducibility, provenance, and how evaluation tools fit into the broader research workflow.
- Agentic tooling for evals — Improve our existing evaluation infrastructure, including agentic AI-based tools for investigating forecasts and synthesizing results.
- Technical communication — Produce clear scorecards, visualizations, and explanations for researchers, leadership, customers, and external partners. Communicate model performance precisely, including uncertainty and important caveats.
Requirements
- Excellent scientific judgment and healthy skepticism. You ask whether a comparison is fair, what else could explain a result, and what evidence would change your mind.
- Strong experimental taste: you can identify the evaluation that answers the question that matters and distinguish robust improvement from noise.
- Systems thinking: you can solve today’s problem while recognizing what should become reusable infrastructure for future work.
- Experience evaluating ML systems using large, scientific, geospatial, multidimensional, or time-series datasets.
- Strong Python skills and experience with scientific and ML tools such as PyTorch, NumPy, pandas, or xarray.
- Able to investigate ambiguous results independently, synthesize evidence, and communicate conclusions clearly.
Skills and Qualifications
- Experience with weather, climate, forecasting, physical science, or AI-assisted research tools is helpful, but not required.
Benefits
401(k), Dental, health, and vision insurance, Unlimited PTO, Stock Option Plan, Office food and beverages
Pay
$140k–$240k. We consider a range of backgrounds and experience levels and adjust offers to be competitive with market rates.
Schedule
Hybrid or in-person.