AI Engineer V (Principal)

Overview The AI Engineer V (Principal) is the most senior individual-contributor technical authority for artificial intelligence and machine learning within AssistRx Engineering. Reporting to the Vice President, Engineering, this role defines and drives the company’s AI technical strategy and leads the design, build, and deployment of production-grade AI solutions across the specialty medication access ecosystem, including large language model applications, predictive models, intelligent document processing, and agentic automation that improve speed, accuracy, and scale in patient onboarding, benefits verification, prior authorization, document intake, and patient and provider engagement. The AI Engineer V (Principal) shapes AI direction at the enterprise level, solves the company’s most complex and ambiguous technical problems, and sets the architecture, evaluation standards, and responsible AI practices for all AI work at AssistRx. The role serves as a trusted advisor to engineering and executive leadership, represents AssistRx’s AI capabilities with clients, partners, and auditors, and develops the next generation of AI technical leaders. The Principal AI Engineer partners closely with Software Engineering, Product, Architecture, Data, Security, Compliance, and Client Services, and is accountable for the quality, performance, cost efficiency, and safe and compliant use of AI with protected health information across the enterprise. Responsibilities AI Strategy & Solution Design Define the multi-year AI technical strategy and roadmap in partnership with the Vice President, Engineering, Product, and Architecture, aligning AI investments to business value, feasibility, and riskLead the design of the most complex enterprise-scale AI and machine learning solutions, including generative AI, predictive models, and intelligent document processing that span multiple platforms, products, and clientsArchitect large language model applications using retrieval augmented generation, prompt engineering, structured outputs, function and tool calling, and agent-based orchestrationSet the patterns for agentic workflows, including task decomposition, tool and function calling, memory and state management, routing, clear boundaries, human review checkpoints, and fallback paths for high-impact decisions such as benefits verification and prior authorizationIdentify new AI opportunities and emerging technologies that create competitive advantage, and lead proofs of concept that move proven pilots to scaleDetermine the right approach for each problem, including prompting, retrieval augmentation, tool use, task-specific models, and conventional software or deterministic logic when a model is not the right answerSet the standard for design documentation of AI features, covering intended behavior, context and retrieval design, prompt and agent structure, evaluation approach, and fallback behavior AI Foundations & Platform Own the enterprise AI reference architecture, platform strategy, and engineering standards, including model access, prompt and version management, and retrieval services, and ensure their adoption across all engineering teamsLead evaluation of commercial models, AI platforms, and vendor tools, and make build-versus-buy recommendations to executive leadership based on quality, cost, security, strategic fit, and time to valueGuide vendor and cloud provider relationships from a technical perspective, including model selection, capacity, pricing, and riskAnticipate scaling, reliability, and cost challenges in AI systems, and lead architectural investments that address them before they affect clients Model Development, Evaluation & Optimization Set organization-wide targets for accuracy, groundedness, hallucination rate, task completion, latency, and cost, and guide model selection, tuning, and evaluation against themDefine the evaluation strategy, benchmark datasets, and automated test frameworks used across the organization to measure output quality and detect regressions before each releaseGuide optimization of retrieval pipelines, embeddings, chunking strategies, and inference performance to improve quality and reduce cost per transaction at scaleDirect error analysis and human-in-the-loop review programs with operations and clinical subject matter experts, and ensure findings drive improvement of prompts, models, and guardrailsEstablish the experimentation approach, including A/B testing across prompts, retrieval configurations, model selection, and workflow designDefine and enforce quality gates and regression thresholds so AI features must meet defined accuracy and safety criteria before release Data Engineering & Integration Set the technical direction for the pipelines that prepare structured and unstructured healthcare data, such as enrollment forms, payer documents, and clinical notes, for training, retrieval, and inferencePartner with Data Engineering and Architecture to define enterprise data contracts, feature sets, vector stores, and lineage for AI workloadsDefine enterprise standards for de-identification, masking, and data minimization when working with protected health information Machine Learning Operations & Production Support Define enterprise standards for CI/CD, source control, containerization, and infrastructure-as-code practices for AI workloadsEstablish versioning, promotion, and rollback practices for prompts, agents, and model configurations, treating them as versioned artifacts alongside application codeSet the observability strategy for accuracy drift, groundedness, task completion, token consumption, latency, cost, and reliability, including alerting and incident response practicesOwn the strategy for model and inference spend, applying caching, routing, and model right-sizing to keep AI solutions cost-effective at scaleServe as the final technical escalation point for critical production issues and client-reported defects in AI-enabled features, and drive systemic improvements that prevent recurrence Responsible AI, Security, Privacy & Compliance Lead the company’s responsible AI approach and ensure solutions satisfy healthcare privacy, data security, and regulatory requirements, including HIPAA and HITRUSTPartner with Security, Compliance, and Legal to address AI-specific risks such as prompt injection, data leakage, and insecure output handling, and to confirm vendor agreements (including Business Associate Agreements) are in place before PHI is processedDefine guardrails, content filtering, human oversight, and audit trails appropriate to the risk profile of each use caseContribute to internal AI governance, and represent AssistRx’s AI practices in audits, client due diligence, and security reviewsTrack emerging AI regulation and industry guidance, and translate it into engineering policy and practice Technical Leadership, Influence & Enablement Serve as a trusted technical advisor to the Vice President, Engineering and executive leadership on AI strategy, risk, investment, and capabilityProvide technical leadership across the engineering organization, influencing architecture and engineering direction at the enterprise level without direct authorityMentor and develop Staff, Senior, and mid-level engineers, build the AI technical bench, and raise AI fluency across engineering and the broader companyLead cross-functional technical initiatives and resolve complex technical disagreements across teamsWork with Product Management, Client Services, and business leaders to identify high-value AI use cases and define clear success measures before build beginsCommunicate AI capabilities, limitations, risks, and results in plain language to technical and non-technical audiences, including executives, clients, and external partnersContribute to hiring strategy, interviewing, and role definition for AI and related engineering rolesAdvance AI-assisted and agentic development practices across the engineering organization, including specification-driven workflows and agentic coding tools, and share what works with engineering peersParticipate in agile team activities as needed, including sprint reviews and retrospectives, and conduct regular self-guided study to stay at the forefront of a fast-moving AI landscape Qualifications Bachelor’s degree in Computer Science, Engineering, Data Science, or related technical field (or equivalent experience); Master’s degree preferred10+ years of software engineering experience, including 6+ years building and deploying AI or machine learning solutions in productionTrack record of leading the strategy and delivery of enterprise-scale AI solutions used by real customers or operations teams, with significant, measurable business outcomesExpert programming skills in Python, along with experience in .NET/C#/Java or a comparable enterprise language, including building backend services and REST APIs (for example FastAPI, Flask, or ASP.NET)Deep, hands-on expertise with large language models and generative AI patterns, including retrieval augmented generation, prompt engineering, embeddings, vector and hybrid search, reranking, structured outputs, function and tool calling, context window management, and model evaluationExtensive experience architecting and operating production AI workloads on cloud-based AI and machine learning platforms (AWS preferred, including services such as Amazon Bedrock and Amazon SageMaker; equivalent Azure experience with Azure AI Foundry, Azure OpenAI Service, or Azure Machine Learning also acceptable)Expert knowledge of APIs, microservices, distributed systems, SQL, and modern development practices including Git, CI/CD, containerization, and automated testingDemonstrated ability to deliver in an environment where data privacy, security, and regulatory requirements shape solution designDemonstrated enterprise-level technical leadership, including setting technical strategy, driving architectural decisions across teams, and developing other technical leadersAbility to work independently on highly ambiguous, high-impact problems, set direction, and make sound tradeoffs with limited precedentHands-on experience building software with agentic coding tools such as Claude Code, GitHub Copilot agents, Codex, or Cursor, including specification-driven workflows and multi-step agent executionDeep practical understanding of how large language models behave in production, including hallucination, prompt sensitivity, non-determinism, and latency and cost tradeoffsExpert knowledge of evaluation methods for LLM systems, including automated scoring, LLM-as-judge techniques, and human review workflowsExceptional written and verbal communication skills, with the ability to influence executives, clients, and engineers and to represent the company’s AI approach externally Preferred Qualifications Master’s or doctoral degree in Computer Science, Machine Learning, Data Science, or related fieldExperience in healthcare technology, specialty pharmacy, HUB services, ePA, or EHR/EMR integrationsExperience with intelligent document processing, OCR, and data extraction from unstructured enrollment or clinical documentsApplied machine learning or data science background, including experiment design, model evaluation, and error analysisExperience with Docker, Kubernetes, or infrastructure-as-code at scaleExperience with agent and orchestration frameworks such as LangChain, LangGraph, LlamaIndex, Semantic Kernel, Strands, or Model Context ProtocolExperience with machine learning operations tooling, model registries, feature stores, and LLM observability or evaluation platformsExperience with model fine-tuning or smaller task-specific models for classification and extractionFamiliarity with AI security guidance such as the OWASP Top 10 for LLM ApplicationsFamiliarity with HIPAA, HITRUST, or responsible AI frameworks such as the NIST AI Risk Management FrameworkPrior experience establishing an AI function or serving as the senior AI technical leader for an organizationExperience representing technical capabilities in client due diligence, security reviews, or external audits COMPETENCIES Technical DepthStrategic ThinkingAnalytical Rigor & Problem SolvingExperimentation & Measurable OutcomesAccuracy & QualitySound Judgment & Risk AwarenessOwnership & AccountabilityPragmatism & Bias for DeliveryCommunicationCollaboration & InfluenceTechnical Leadership & Talent DevelopmentCuriosity & Learning AgilityAdaptabilityInitiative & InnovationEthics & Integrity Pay Range USD $162,239.00 - USD $204,048.00 /Yr.