Principal AI Engineer

About the role

Loyal Source is bringing AI into the way the world's most demanding buildings operate — from datacenters and hospitals to pharmaceutical facilities and commercial campuses. We are transforming our smart building products into AI-native platforms capable of autonomous operations, intelligent operator assistance, and scalable AI deployment across cloud, edge, and on-premises environments. This is a focused, time-boxed engagement for a Principal-level AI engineer to embed within the Controls Software team and produce the foundational architecture assets and deployment playbooks that will accelerate AI adoption across our smart building platform at scale.

Required Qualifications

• AI Systems & Architecture: 10+ years of hands-on software or systems engineering experience, with at least 6 years focused on AI/ML in production environments
• Proven experience designing and deploying AI/ML systems at scale — from data ingestion through inference and monitoring
• Deep knowledge of MLOps: model deployment pipelines, versioning, observability, drift detection, and continuous improvement
• Experience with edge-to-cloud AI execution strategies: balancing latency, cost, and resiliency across distributed environments, including LLM cost optimization (model selection, caching, routing)
• Strong command of data pipeline architecture, time-series data, event-driven systems, and API/microservices patterns
• GenAI & LLM: Hands-on experience architecting production GenAI applications across multiple LLM providers (e.g., Anthropic, OpenAI, AWS Bedrock, Azure OpenAI, and open-source models)
• Deep knowledge of RAG architectures, vector databases, embedding pipelines, and retrieval strategies at production scale
• Experience with agentic architectures, multi-agent orchestration, and tool-calling patterns — including emerging standards like Model Context Protocol (MCP)
• Experience with LLM observability and tracing — instrumenting model calls, tool calls, and retrievals in production (e.g., LangSmith, LangFuse, or OpenTelemetry GenAI conventions)
• Craft & Communication: Ability to produce clear, durable architecture artifacts — reference architectures, decision records, playbooks — that engineers can execute without you in the room
• Comfortable working collaboratively in an embedded team model; can give and receive direct technical feedback
• Capable of running technical workshops or design sessions when needed

Preferred Qualifications

• Experience in industrial, OT, or IoT environments (building automation, manufacturing, energy, or similar)
• Familiarity with protocols such as BACnet, MQTT, Modbus, or OPC UA
• Exposure to cybersecurity frameworks in OT environments (e.g., IEC 62443, NIST CSF)
• Experience with AI use cases in buildings or critical infrastructure: FDD, energy optimization, predictive maintenance, alarm intelligence
• Experience with containerization, CI/CD tooling, and observability platforms
• Experience operationalizing AI safety guardrails, content filtering, and governance controls in production

What Success Looks Like

• A validated AI reference architecture that the Controls SW team can build against confidently
• Deployment playbooks that are immediately usable — not aspirational documents
• Engineers who have meaningfully leveled up through collaboration with you
• A clear architectural foundation for our smart building platform to evolve toward autonomous, AI-native building operations

Pay

$111.11

About Loyal Source

Loyal Source provides healthcare and technology solutions designed to improve outcomes and support mission-critical operations. Guided by innovation and purpose, we help organizations achieve success across complex challenges worldwide. Loyal Source is a military friendly employer and proud partner of the Military Spouse Employment Partnership program. Loyal Source uses AI-powered voice screening as part of our application process. After you apply, you will receive a brief qualification survey via text or email. Candidates who meet the position requirements will then be invited to complete a quick AI-powered voice assessment. This easy, no-scheduling-required process helps us and the candidate move the hiring process forward faster. We are committed to providing equal employment opportunities to all applicants. If you have a disability or medical condition and need a reasonable accommodation to participate in this step of the hiring process, you may request an alternative method or accommodation at any time by contacting Human Resources at HRmgt@loyalsource.com.