Senior AI Architect (Hybrid Eligible)

About the role

The Senior AI Architect at Oak Ridge National Laboratory (ORNL) operates at the intersection of advanced machine learning, software and systems architecture, and ORNL’s enterprise and high-performance computing (HPC) environments. The role involves designing, integrating, and governing end-to-end AI systems that can be deployed and sustained across research programs and mission operations.

Responsibilities

• Define reference architectures for “AI at ORNL,” including patterns for data ingestion and curation, training and fine-tuning pipelines, model serving and inference at scale, and integration into scientific workflows.
• Guide technology selection and implementation for core capabilities such as distributed training and inference, workflow orchestration, GPU/accelerator utilization, model registries and artifact management, vector search and retrieval-augmented generation (RAG), and MLOps/MLOps practices (CI/CD for models, automated evaluation, and monitoring).
• Establish performance and cost models to decide when workloads should run on HPC versus cloud, and how to engineer systems for throughput, latency, and resource efficiency under real-world constraints.
• Work closely with cybersecurity and compliance stakeholders to implement robust identity and access management, network segmentation, audit logging, and data-governance controls.
• Lead architecture reviews, produce technical roadmaps, and coordinate cross-functional teams of researchers, platform engineers, and application developers to move AI capabilities from prototype to production.
• Mentor teams on best practices, standardize reusable components, and ensure AI systems remain maintainable and supportable over time.

Requirements

• Bachelor’s or Master’s degree in Computer Science, AI/ML, Software Engineering, or related field
• 7+ years of experience in software architecture, machine learning systems, or distributed systems
• Strong understanding of:
• Machine learning fundamentals and LLM architectures
• API-based and local model inference workflows
• Experience using both:
• Cloud-hosted AI services (e.g., OpenAI, AWS Bedrock, Azure AI, Vertex AI, or similar)
• Locally deployed models (e.g., via Hugging Face, Ollama, vLLM, or similar)
• Familiarity with Model Context Protocol (MCP) or comparable orchestration/integration patterns
• Proficiency in Python and backend system design
• Experience with containerization and deployment (Docker, Kubernetes)

Qualifications

• Experience architecting hybrid AI systems that dynamically route between local and cloud models
• Hands-on experience setting up on-demand local model inference (GPU-backed or CPU-optimized deployments)
• Deep familiarity with LLM system design patterns:
• Retrieval-Augmented Generation (RAG)
• Tool use / function calling
• Agent-based workflows
• Experience with vector databases (e.g., FAISS, Pinecone, Weaviate, Milvus)
• Experience with inference optimization frameworks (vLLM, TensorRT, ONNX Runtime, quantization techniques)
• Knowledge of model serving infrastructure (Ray Serve, KServe, Triton, FastAPI-based services, MLFlow, etc.)
• Experience designing low-latency and high-throughput inference pipelines
• Experience integrating AI systems into enterprise applications and data platforms
• Understanding of observability for AI systems (logging, tracing, evaluation metrics, drift detection)
• Experience with GPU infrastructure, scheduling, and cost optimization strategies
• Familiarity with security practices for AI systems, including prompt injection mitigation and data isolation
• Experience working with multimodal models (text, image, geospatial, etc.)