Senior Machine Learning Engineer - LLM Evaluation & Agent Systems

We need to output HTML fragment with sections: About the Team, About the Role, Responsibilities, Qualifications (maybe split into Minimum and Preferred), About USDS, Why Join Us, Diversity & Inclusion, USDS Reasonable Accommodation? The instructions: include a section only when posting has real, specific content for it. Remove boilerplate like EEO, legal, etc. The "USDS Reasonable Accommodation" is maybe boilerplate but it's specific content about accommodation; but it's likely considered boilerplate? It's about accommodation process; might be considered boilerplate but it's specific. However instructions: Remove cookie/navigation/legal/EEO/equal-opportunity/scam-warning boilerplate. Reasonable accommodation is not exactly EEO but could be considered legal. Might be safe to omit. Also "Job Information" includes pay transparency and benefits; we need to include Pay and Benefits sections if present. The posting includes compensation description and benefits details. So we should include Pay section with salary range and maybe note about bonuses etc. Benefits section with list of benefits. Schedule: posting mentions shift from hybrid to fully in-person schedule up to 5 days a week. So we can include Schedule section. We must not repeat job title as heading. No Job ID. We must use

for section headings. Use

• for any list of duties/requirements/benefits. Use

  for prose. We need to preserve all substantive information; do not invent or summarize away detail. Must not emit heading with filler text. We need to decide sections: About the Team, About the Role, Responsibilities, Qualifications (maybe combine Minimum and Preferred under Qualifications), About USDS, Why Join Us, Diversity & Inclusion, Pay, Benefits, Schedule. Check if any section content is just filler like "commensurate with experience" - not present. We need to extract content. Let's parse the posting: - Responsibilities About the Team: Actually heading "About the Team" then paragraph about TikTok U.S. Data Security team and Data Platform Team. - About the Role: paragraph describing role. - Key Responsibilities: bullet list. - Qualifications: Minimum Qualifications bullet list, then Preferred Qualifications bullet list. - About USDS: paragraph about USDS Joint Venture etc. - Why Join Us: paragraph. - Diversity & Inclusion: paragraph. - USDS Reasonable Accommodation: paragraph about accommodation. - Job Information: includes compensation description and benefits. We need to decide whether to include USDS Reasonable Accommodation. It's specific but maybe considered boilerplate; but it's not EEO. However it's about accommodation process; could be considered legal. I'd lean to omit to be safe. Pay section: compensation description includes base salary range, mention of bonuses, RSU, etc. We'll put that in

  . Benefits: list of benefits: medical, dental, vision insurance, 401(k) match, paid parental leave, short/long term disability, life insurance, wellbeing benefits, 10 paid holidays, 10 paid sick days, 17 days Paid Personal Time, etc. We'll put in

  . Schedule: mention shift from hybrid to fully in-person schedule up to 5 days a week. We'll put in

  . Now produce HTML. Structure:

  About the Team

  ...

  About the Role

  ...

  Responsibilities

  • ... each bullet

  Qualifications

  Maybe we need subheadings? Not allowed to create extra headings besides h3 for sections. Could we embed Minimum and Preferred as inside

  ? But we need to preserve list. Could do two

  with preceding

  indicating Minimum Qualifications and Preferred Qualifications. That's okay. So:

  Qualifications

  Minimum Qualifications

  ...

  Preferred Qualifications

  ...
  But the instruction: Use
  • for any list of duties/requirements/benefits, and

    for prose. It didn't forbid using

    for labels. So okay.

    About USDS

    ...

    Why Join Us

    ...

    Diversity & Inclusion

    ...

    Pay

    ...

    Benefits

    ...

    Schedule

    ...

    Make sure not to include empty sections. Now extract exact text. About the Team paragraph: "The TikTok U.S. Data Security (USDS) team is responsible for the security, integrity, and compliance of TikTok’s data within the United States. The Data Platform Team builds a scalable, reliable, cost-efficient infrastructure that ensures data integrity and empowers every business unit to win through self-service tools and scientific insights." We need to keep that exactly. About the Role paragraph: "We are seeking a Senior Machine Learning Engineer to join our team as a technical lead, driving the architecture and execution of our next-generation AI Risk & Compliance Intelligence Platform and Intelligent Data Agent Ecosystem. In this role, you will be instrumental in bridging the gap between standard prompt engineering and native, model-driven reasoning frameworks. You will tackle complex ML challenges around offline/online evaluation, synthetic data generation for small-sample scenarios, and automated agent skill optimizations across Data & Engineering workflows (DA/DS/DE/SRE)." Responsibilities bullets: - Design, build, and scale robust LLM evaluation architecture, including automated offline datasets, metric suites, and benchmarking pipelines to measure prompt and model performance. - Innovate on LLM-driven synthetic data generation methodologies to tackle data scarcity bottlenecks and systematically evaluate long-tail/edge-case risk scenarios. - Advance beyond criteria-based prompt engineering by extracting deep risk patterns directly from complex case data to build native, ML-driven decision models and reasoning abstractions. - Architect, evaluate, and continuously optimize modular agent skills and tools tailored for Data Science, Analytics, Data Engineering, and SRE workflows to maximize tool-use accuracy, execution efficiency, and task success rates. - Implement advanced evaluation methodologies (such as LLM-as-a-judge and multi-agent benchmarking) across both risk intelligence and agentic ecosystems. - Provide technical leadership by establishing core ML system design patterns, benchmarking standards, and engineering best practices, while translating complex operational requirements into scalable machine learning solutions across cross-functional partnerships. Qualifications Minimum: - Experience: 5+ years of software/ML engineering experience with a strong track record of designing, building, and deploying ML/LLM systems in production. - Generative AI & LLMs: Deep hands-on expertise with LLM architectures, fine-tuning, RAG, prompt tuning, and agentic frameworks. - Evaluation & Data Bootstrapping: Demonstrated experience building robust LLM evaluation frameworks (LLM-as-a-judge, automated benchmark pipelines) and tackling data scarcity/imbalance problems. - Core Engineering: Strong coding skills alongside production experience with scalable data processing pipelines. Preferred Qualifications: - Prior domain experience in Risk & Compliance Intelligence and Automated Decision Systems. - Hands-on experience with synthetic dataset generation or bootstrapping models for cold-start scenarios. - Experience building AI tools or agents for technical personas (Data Analysts, Data Scientists, Data Engineers, Data SRE). - Demonstrated ability to serve as a founding technical lead on ambiguous, high-impact initiative pods. About USDS paragraph: "TikTok USDS Joint Venture LLC is dedicated to the safety and security of millions of Americans who create, discover, and connect with what they love on the apps we operate. The Joint Venture has been established in compliance with the Executive Order signed by President Trump on September 25, 2025. Our foundation is a comprehensive data privacy and cybersecurity program we operate under defined safeguards to protect national security and secure U.S. user data, apps and the algorithm. We safeguard the U.S. content ecosystem, holding decision-making authority for trust and safety policies and moderation. USDS Joint Venture helps ensure Americans can continue to express their creativity, discover new hobbies and interests, and build thriving communities and businesses on a global scale. On-site presence across teams allows the company to operate with greater speed, alignment, and agility — especially in areas like real-time decision-making, team development, and integrated execution. As such, the company is shifting from a hybrid work model to a fully in-person schedule up to 5 days a week." Why Join Us paragraph: "Inspiring creativity is at the core of TikTok's mission. Our innovative product is built to help people authentically express themselves, discover and connect – and our global, diverse teams make that possible. Together, we create value for our communities, inspire creativity and bring joy - a mission we work towards every day. We strive to do great things with great people. We lead with curiosity, humility, and a desire to make impact in a rapidly growing tech company. Every challenge is an opportunity to learn and innovate as one team. We're resilient and embrace challenges as they come. By constantly iterating and fostering an "Always Day 1" mindset, we achieve meaningful breakthroughs for ourselves, our company, and our users. When we create and grow together, the possibilities are limitless. Join us." Diversity & Inclusion paragraph: "TikTok is committed to creating an inclusive space where employees are valued for their skills, experiences, and unique perspectives. Our platform connects people from across the globe and so does our workplace. At TikTok, our mission is to inspire creativity and bring joy. To achieve that goal, we are committed to celebrating our diverse voices and to creating an environment that reflects the many communities we reach. We are passionate about this and hope you are too." Pay paragraph: "The base salary range for this position in the selected city is $177688 - $341734 annually. Compensation may vary outside of this range depending on a number of factors, including a candidate’s qualifications, skills, competencies and experience, and location. Base pay is one part of the Total Package that is provided to compensate and recognize employees for their work, and this role may be eligible for additional discretionary bonuses/incentives, and restricted stock units." Benefits list: - Employees have day one access to medical, dental, and vision insurance, - a 401(k) savings plan with company match, - paid parental leave, - short-term and long-term disability coverage, - life insurance, - wellbeing benefits, - among others. - Employees also receive 10 paid holidays per year, - 10 paid sick days per year - and 17 days of Paid Personal Time (prorated upon hire with increasing accruals by tenure). - The Company reserves the right to modify or change these benefits programs at any time, with or without notice. We need to keep as list items. Some sentences have commas; we can split each bullet as a separate item. Schedule paragraph: "As such, the company is shifting from a hybrid work model to a fully in-person schedule up to 5 days a week." But that sentence already appears in About USDS paragraph. Actually it's in About USDS. We could still have Schedule section with that sentence. However it's duplicate; but we need to preserve all substantive information; we can keep it in About USDS and also maybe not duplicate. Better to keep Schedule section with that sentence, but we already have it in About USDS. To avoid duplication, we could keep it only in About USDS and not create Schedule section. However the instruction: include a section ONLY when the posting has real, specific content for it. The posting does have schedule info (shift to fully in-person up to 5 days a week). It's present in About USDS paragraph. We could either keep it there or create Schedule section. Since we already have About USDS, we could keep schedule info there and not create separate Schedule section to avoid duplication. But the instruction says do