Principal AI Engineer - Agent Ops / SRE

Responsibilities

• Serve as technical liaison between AI COE and Platform Engineering & Enterprise SRE teams.
• Ensure AI systems meet requirements for performance, latency, throughput, resiliency, recovery, observability and reliability.
• Partner with AI engineers, Applied AI Scientist, and AI Architects to design, build and maintain scalable, fault tolerant AI systems as per SLO.
• Partner with Platform engineering team to design and implement CICD, GITOps, and IAC (Terraform) modules.
• Making sure we use our AgentOps NSA, standards, Ref. architecture and tooling.
• Partner with enterprise release management and AI Governance team to build & deploy AI solutions using their platform tooling.
• Supporting entire AI lifecycle as per the standard work template.
• Build standardized deployment templates, reference architecture, automation scripts, terraform modules, CICD pipelines, and operational runbooks for AI workloads.
• Design and build IDP (Harness) catalogs, templates & pipelines partnering with enterprise platform engineering team.
• Manage production systems to ensure our enterprise SLOs are met.
• Manage incident response for production systems, including triaging, escalating, RCA and implementing corrective actions.

Qualifications

• Bachelor's degree in Computer Science, Computer Engineering, or a technical field.
• 10+ years building and shipping software and/or platform solutions for enterprises.
• Programming experience with Python is required.
• 3+ years of experience with IAC (Terraform).
• 5+ years of experience owning production CICD, GitOps and release management gating.
• 3+ years of experience in implementing observability, performance & reliability solutions: SLO, P99-95 latency, alert tuning, & dashboards.
• Experience with AI observability/monitoring tools such as Dynatrace, Splunk, Arize & OpenTelemetry/OpenInference is must.
• Proven experience with Google's Gemini Enterprise Agent platform is a plus.
• Experience with GKE/Docker/Registry is a plus.
• Experience with Automated Testing, Automated Deployments, Agile methodologies, Unit Testing, and Integration Testing tools.
• Conversational UX/UI design (multi-turn chatbots) and Human-Agent-Interaction (HAI) is a plus.
• Experience with IR, vector embedding, and Hybrid/Semantic search technologies.
• Experience with LLM orchestration frameworks like Langchain, LlamaIndex, LangSmith, LangGraph, Google Agent Development Kit, is a plus.
• Experience with Generative AI Guardrails, responsible AI, adversarial attack mitigation, and red teaming is a plus.
• Foundational understanding of Natural Language Processing and Deep Learning.
• Excellent problem-solving skills and the ability to work in a collaborative team environment.
• Excellent communication skills.