Sr Principal AI Architect - Re-location Consider - Full-time

Role: Sr Principal AI Architect Location: Bay area, CA Re-location Consider Duration: Full-time Job Summary Will work on the intelligence layer for multiple programs owns all model quality, RAG accuracy, prompt engineering, and AI safety across applications Socratic tutor persona, adaptive learning recommendation engine, multi-modal AI (text and voice), RAG evaluation framework, and feedback loop into retrieval6-LLM call chain orchestration (NeMoGuardrails intent classification query rewriting RAG synthesis), , and compatibility check logicProduction-grade AI quality from launch this is not a research or prototyping role; accuracy thresholds, latency requirements, and safety guardrails must pass InfoSec adversarial testing before Release 1 Required Skills. Experience Total IT 15+ Years 4 7 years of software engineering with at least 2 years focused on LLM application development in production not research, not demos, not internal tools with 10 users Has shipped an LLM-powered feature or product to production where real users depend on the accuracy and the engineer owns the quality metrics Has owned an AI safety or guardrails implementation for a customer-facing product not just added an off-the-shelf filter; designed and tested the safety layer Has built RAG evaluation pipelines and used them to make go/no-go release decisions accuracy gating is part of the workflow. Has profiled and optimized a multi-step LLM call chain for latency LLM Application Development LLM prompt engineering system prompts, few-shot examples, chain-of-thought, instruction following Expert Must-haveMulti-step LLM chain orchestration LangChain, LlamaIndex, or custom orchestration Expert Must-haveMulti-turn conversation design context window management, conversation summarization, session memory Advanced Must-haveStreaming LLM response handling token-by-token streaming, partial response rendering Advanced Must-haveModel selection and benchmarking matching model size to task; balancing latency, cost, and accuracy Advanced Must-have RAG Pipeline Design & Quality RAG pipeline design chunking strategy, embedding model selection, retrieval configuration Expert Must-haveVector similarity search tuning index parameters, similarity thresholds, retrieval depth Advanced Must-haveReranking cross-encoder rerankers, relevance scoring Advanced Must-haveRAG evaluation frameworks RAGAS, TruLens, or equivalent; automated eval pipelines Advanced Must-haveHybrid search combining dense vector retrieval with BM25 or keyword search Proficient Nice to have AI Safety & Guardrails Prompt injection detection and mitigation Advanced Must-haveJailbreak testing and red-teaming LLM systems Advanced Must-haveContent safety classifier integration Advanced Must-haveHallucination detection and mitigation strategies Advanced Must-haveTopical control enforcing scope boundaries on LLM responses Advanced Must-have Evaluation & Production Quality Automated evaluation pipeline design test set curation, metric selection, regression detection Advanced Must-haveA/B evaluation methodology for prompt and model changes Proficient Must-haveLatency profiling for LLM call chains identifying bottlenecks across multi-step pipelines Proficient Must-haveFeedback loop design user signal collection, signal-to-retrieval-weight integration Proficient Must-haveProduction model monitoring accuracy drift detection, quality degradation alerting Proficient Must-have Development Python ML/AI application development, async programming Expert Must-haveAPI design for AI services streaming endpoints, error handling, timeout management Advanced Must-haveEmbedding model operations model selection, batch embedding, index updates Advanced Must-have Nice to Have Adaptive learning systems or personalization engine experienceKnowledge graph integration with RAGMulti-agent orchestration patterns ServiceNow API integrationPrior experience building AI products on NVIDIA infrastructure