Data Scientist, Product Analytics

San Francisco, CA · Full-time · Mid-level · On-site

About This Role

Zof AI is seeking a Data Scientist to measure how well our agent fleets actually find and fix defects. This role owns the numbers behind the product: detection precision and recall, remediation success rates, experiment design, and the statistical rigor that separates a real improvement from noise. If you have worked as an Applied Scientist, Product Data Scientist, Quantitative Analyst, or Machine Learning Analyst, this is that discipline at Zof AI. The ideal candidate is fluent in Python and SQL, reasons about causes rather than correlations, and would rather report an honest number than a flattering one.

Responsibilities

• Define and own the metrics that describe how well our agent fleets find and fix defects.
• Measure detection precision and recall against real customer codebases.
• Quantify remediation success rates and the strength of the validation evidence behind them.
• Design experiments that show whether a product or model change actually helped.
• Partner with engineering on eval design so agent runs are scored consistently over time.
• Build analysis pipelines in Python and SQL that other people can rerun and trust.
• Translate findings into clear recommendations for engineering, product, and leadership.
• Own the reporting that keeps our claims about the product grounded in evidence.

Requirements

• Strong working fluency with Python and SQL.
• Solid grounding in statistics, experiment design, and inference.
• Experience analyzing messy real-world data and drawing defensible conclusions.
• Ability to turn a fuzzy question into a metric someone can act on.
• Comfort reasoning about causality, bias, and confounding.
• Clear written and verbal communication.
• Comfort operating in a fast-moving environment.
• Evidence of analysis that changed a decision.

Nice to have

• Experience measuring the quality of machine learning or AI systems.
• Experience with causal inference methods or Bayesian analysis.
• Experience with data warehouses, dbt, or notebook-to-production workflows.
• Familiarity with software testing, CI pipelines, or developer tooling metrics.

What we provide

• MacBook Pro
• Premium AI development tools (Cursor Ultra, Claude Code Ultra, OpenAI Codex Max or equivalent advanced AI tooling)
• Access to a high-performance AI product environment
• Close collaboration with leadership, engineering, and customers
• Opportunity to work in the San Francisco AI ecosystem
• Wellness and productivity support where applicable
• Competitive startup environment with high ownership and direct product impact