Program Manager II, Tech

You will lead complex programs that bridge technical vision and operational execution, focusing on the "how" of technical delivery. This role scales systems like AI/ML human-in-the-loop workflows or global supply chains by navigating high-ambiguity environments while maintaining disciplined operational excellence.

Responsibilities

• Technical program manager for GenAI programs, designing and implementing comprehensive Machine Translation (MT) and Large Language Model (LLM) programs that support both Uber's internal operations and external clients.
• Architect Reinforcement Learning from AI Feedback (RLAIF) frameworks to scale model alignment and ensure consistency with global communication and localization strategies.
• Build seamless integration points between Uber's internal services and tools, connecting them to the MT/LLM pipeline.
• Serve as a technical liaison, using Python scripting to automate data flows and optimize the pipeline for efficiency, specifically regarding the deployment of LLM-as-a-Judge for automated quality gatekeeping.
• Partner with the Localization Operations (LocOps) and Language Model Engineering teams to select, evaluate, train, and deploy the most effective models.
• Leverage Python to develop and run LLM-based evaluation rubrics that act as a judge for model performance, enhancing both internal productivity and external localization outcomes.
• Lead cross-functional initiatives, engaging with internal localization teams and external partners to gather requirements, draft Product Requirement Documents (PRDs) for RLAIF pipelines, and manage execution of tool enhancements and process improvements.

Requirements

• Master's degree in Program Management, Computer Science, Information Management, or a related field, and 6 months of experience in the job offered or a related occupation.
• Experience with Large Language Models (LLMs) and Prompt Engineering.
• Demonstrated experience implementing Reinforcement Learning from AI Feedback (RLAIF) or Human Feedback (RLHF) to scale model alignment.
• Experience fine-tuning and deploying production-grade LLMs or MT models (such as GPT, Mistral, T5, or OpenNMT) within scalable enterprise environments.
• Ability to design and architect automated evaluation pipelines, specifically utilizing "LLM-as-a-Judge" frameworks to optimize AI-assisted translation and human-in-the-loop (HITL) workflows.
• LLMOps/MLOps experience, including reviewing architecture diagrams, API documentation, and monitoring model lifecycle management and deployment performance.
• Experience leading cross-functional AI/ML initiatives by translating complex business requirements into technical Product Requirement Documents (PRDs) for engineering and product teams.
• Python scripting for data flow automation and SQL for analyzing judge-based quality metrics and reinforcement learning progress.
• Understanding of data privacy, redaction layers, and secure model access protocols within enterprise GenAI environments to ensure compliance with global data protection regulations.
• Experience curating instruction-tuning datasets and utilizing LLMs for synthetic data generation to improve dataset diversity for model alignment training.

Pay

$135,000.00 to $165,000.00 per year.

Benefits

• Eligibility to participate in Uber's bonus program and other compensation types, including equity.
• Access to a range of benefits. More details here.

Schedule

Unless approved for full remote work, employees must spend at least 50% of their time in-office. Some roles, like those at greenlight hubs, require full-time in-office presence. May telecommute.