Data Analyst, AI Agent Analytics — New Grad

About the role

As a New Grad Data Analyst focused on AI Agent Analytics, you will measure agent quality, understand user interactions, and turn complex behavioral data into insights that improve our AI systems.

• Build real, production AI agents used by real users
• High ownership and impact
• Work at the intersection of AI, agents, and product
• Shape how people experience AI-driven job search

Responsibilities

• Define metrics for agent task completion, quality, latency, reliability, and cost
• Analyze agent conversations, tool usage, errors, and user feedback
• Build SQL queries, dashboards, and recurring reports
• Identify failure patterns and improvement opportunities
• Support experiments involving prompts, models, tools, and workflows
• Present actionable recommendations to AI and product teams

Qualifications

• Recent graduate in Data Science, Statistics, Computer Science, Mathematics, Economics, or a related field
• Strong SQL skills and familiarity with Python
• Understanding of statistics, experimentation, and data visualization
• Strong analytical and communication skills
• Ability to work effectively in a remote environment

Preferred

• Experience analyzing LLM, chatbot, or agent interactions
• Familiarity with product analytics or event-tracking systems
• Experience with pandas, Tableau, Looker, or similar tools
• Interest in AI evaluation, prompt engineering, or human-AI interaction