Applied Research PhD Intern

Description

Work directly with members of Technical Staff in the Office of the CTO, on the evals and experimentation layer that BMC AI products are built on.

Design evaluations that catch the failure modes of enterprise agents: hallucinated tool calls, policy violations, context collapse, regression under distribution shift, etc.

Build the Agent Gym — task definitions, graders, reward signals, and trajectory capture — for multi-step agentic workflows.

Run experimentation sweeps across prompts, models, and scaffolds; quantify trade-offs between accuracy, cost, and latency.

Turn eval results into promotion gates and readiness reports that product teams can act on.

Contribute to our Responsible AI tooling — grounding checks, policy enforcement, and human-in-the-loop escalation paths.

Requirements

Your project will be part of the BMC AI Foundation’s active workstreams and shaped as a focused PhD-level research internship: Agent Gym evaluations, grader design, experimentation tooling, dataset curation, or trace / replay infrastructure.

Exact scope is matched to your doctoral research strengths during onboarding, with your technical mentors, and is sized to produce a concrete research artifact, prototype, or evaluation result within 12 weeks.

Qualifications

• Currently pursuing a PhD in Computer Science, Machine Learning, AI, or a closely related field, with active research in LLMs, agents, reinforcement learning, AI safety, or evaluation methodology.
• Have produced non-trivial research or systems that work on modern LLM and agent stacks — multi-step tool-using agents, RAG pipelines, evaluation harnesses, and post-training.
• Can turn an open research question into testable hypotheses, choose strong baselines and ablations, interpret learning curves or reward trajectories honestly, and communicate findings clearly.
• Treat evaluation as a first-class AI research and engineering problem, not just a reporting layer.
• Published, submitted, or in-progress PhD research on LLM evaluation, agent benchmarks, alignment, RL environments, or related systems.
• Hands-on research experience with RLHF / RLVR, reward modeling, synthetic data generation, red-teaming, or scalable evaluation design.
• Contributions to open-source eval harnesses, agent scaffolds, observability tooling, or reproducible research infrastructure.
• Clear thinking about AI safety, deployment risk, benchmark validity, and the gap between academic results and enterprise production use.