Principal Engineer - AI Platform

About the role

The Principal Engineer operates at the intersection of deep technical execution and broad organizational influence. This role will resolve the hardest architectural questions, set the AI-native engineering bar, and ensure that iHerb's platform decisions compound over time rather than accumulate debt. Reporting to the VP of Engineering, this role collaborates closely with peer Principals, Engineering Managers, and peer VPs across the organization.

Responsibilities

• Define and own the AI platform architecture: retrieval infrastructure, model lifecycle, evals framework, guardrails, and GenAI product feature design.
• Lead the build of the shared AI platform layer consumed by all AI product features and reusable by internal business teams.
• Hands-on contributor: build production AI systems, write proofs of concept, and validate architecture through working software.
• Set and enforce technical standards for the AI Platform team; drive architecture reviews and model quality reviews.
• Cook up complex multi-team technical efforts by coordinating architecture reviews, aligning peer Principals and EMs, and resolving competing approaches.
• Mentor and raise the technical bar across the engineering organization through code review, architecture review, and direct coaching of senior engineers.
• Represent engineering in cross-functional conversations with product, data science, security, and infrastructure.
• Feed architectural decisions into the shared knowledge base so institutional knowledge compounds across the organization.

Required Knowledge, Skills and Abilities

• AI-driven SDLC (required): demonstrated use of AI-assisted development tools such as Claude Code, GitHub Copilot, or Cursor to ship production systems. Can articulate workflow changes, quality tradeoffs, and guardrail strategies.
• Architecture at scale: experience designing and evolving large-scale distributed systems across multiple teams and years: APIs, data pipelines, event-driven architectures, or high-traffic platforms.
• Cross-org technical leadership: track record of driving architectural standards, technical roadmaps, or platform initiatives that span multiple teams or organizations.
• Engineering quality mindset: deeply held opinions on code quality, observability, CI/CD, test automation, and maintaining velocity without accumulating hidden debt.
• Communication and influence: able to write clear architecture documents, present to technical and non-technical audiences, and build consensus without formal authority.
• Experience working in distributed teams across the US, China, and Latin America.
• Production experience designing and operating AI platform systems: RAG pipelines, vector search, embedding infrastructure, or retrieval-augmented applications at scale.
• Hands-on experience with LLM evaluation: building eval frameworks, defining quality metrics, and iterating on model and prompt quality using data.
• MLOps experience: model deployment, monitoring, lifecycle management, and cost governance in a production environment.
• Experience building and operating agentic systems using frameworks such as LangChain, LlamaIndex, or equivalent.
• GenAI in production is required; prototype or research-only experience is not sufficient at this level.
• High degree of accuracy and attention to detail.
• Excellent organization skills and ability to multi-task.

Experience and Education Requirements

• Generally requires a minimum of 10+ years of software engineering experience, with a significant portion at senior, staff, or principal IC level.
• Bachelor’s Degree in Computer Science or related field preferred, or a combination of education and equivalent work experience.

Physical Demands

• The physical demands described here are representative of those that must be met by a Team Member to successfully perform the essential functions of this job.
• While performing the duties of this job, the Team Member is regularly required to talk and hear.
• The Team Member is frequently required to sit, walk, climb stairs, use hands and fingers, bend, stoop and reach with hands and arms.
• Reaching above shoulder heights, below the waist or lifting as required to file documents or store materials throughout the work day.
• The Team Member may occasionally lift or move office products and supplies up to 25 pounds.
• Proper lifting techniques required.

Work Environment

• Hectic, fast-paced with multi-level distractions.
• Professional, yet casual work environment.
• Office / Warehouse environment.
• Ability to work extended hours as required.