Senior AI Engineer

About the Role

The Senior AI Engineer designs, builds, integrates, evaluates, and continuously improves production-grade AI-enabled capabilities that support priority AI use cases. This role focuses on applied AI engineering—taking LLM-powered experiences, agent workflows, RAG patterns, tool/API integrations, evaluation assets, and context-management approaches from concept and prototype into reliable production implementation.

The Senior AI Engineer partners closely with product managers, software engineers, AI architects, data product owners, ontology and knowledge teams, quality engineering, design, journey, clinical, operational, privacy, security, and compliance stakeholders. The role translates customer jobs-to-be-done, workflow requirements, trusted data and knowledge sources, and enterprise architecture patterns into secure, observable, measurable, and maintainable AI capabilities.

This role requires strong software engineering fundamentals, hands-on LLM application experience, and practical judgment about how to make AI systems useful, grounded, evaluated, cost-aware, and safe enough for real workflows.

Responsibilities

• Applied AI Engineering & Product Delivery
  • Design, build, test, and operate AI-enabled product capabilities across backend services, APIs, data flows, agent workflows, and user-facing experiences.
  • Translate product requirements, customer jobs-to-be-done, clinical or operational workflows, and stakeholder feedback into concrete technical designs and working AI capabilities.
  • Take ownership of implementation quality from prototype through production, including maintainable code, documentation, testing, observability, issue resolution, and iterative improvement.
  • Partner with product and engineering teams to sequence work into shippable increments that deliver measurable value while managing technical risk.
• LLM Applications, Agents & Workflow Engineering
  • Build LLM-powered applications, agents, agent components, and multi-step workflows that can use tools, retrieve context, follow defined instructions, and complete targeted tasks reliably.
  • Define and implement agent behavior, task flow, prompt structures, tool-use patterns, fallback behavior, escalation paths, and workflow control logic.
  • Develop reusable patterns for agent workflows, tool invocation, state handling, structured outputs, context windows, memory, and human-in-the-loop checkpoints.
  • Diagnose agent failures and improve behavior through prompt refinement, retrieval tuning, workflow redesign, evaluation results, telemetry, and user feedback.
• Retrieval, Context & Knowledge Integration
  • Design and implement retrieval-augmented generation workflows that ground AI outputs in trusted enterprise data, content, knowledge, and source systems.
  • Partner with data, ontology, knowledge, and engineering teams to define retrieval strategies, source attribution, indexing, chunking, metadata, ranking, traceability, and response-grounding expectations.
  • Build and tune context pipelines, vector or hybrid retrieval patterns, knowledge integrations, and reusable data-access patterns that support high-quality AI outputs.
  • Identify reusable retrieval, context, tool, API, and integration patterns that can be shared across multiple AI use cases and product teams.
• Evaluation, Observability & Reliability
  • Build and maintain evaluation assets such as test datasets, golden-answer sets, scoring rubrics, prompt/version comparisons, regression tests, and quality dashboards for AI-enabled capabilities.
  • Measure and improve agent and LLM application performance across accuracy, groundedness, task completion, latency, cost, safety, response quality, and workflow reliability.
  • Use logs, traces, telemetry, user feedback, evaluation harnesses, and production signals to identify failure patterns and improve system behavior over time.
  • Partner with Quality Engineering and Product teams to ensure AI capabilities meet acceptance criteria, responsible AI expectations, and release-readiness standards.
• Tool, API & Enterprise System Integration
  • Integrate AI capabilities with approved APIs, enterprise systems, tools, services, data products, knowledge repositories, and platform capabilities using secure and governed patterns.
  • Build tool wrappers, action interfaces, integration utilities, and workflow components that allow AI systems to interact with business processes in controlled and auditable ways.
  • Collaborate with engineering, architecture, security, privacy, and operations teams to ensure integrations follow enterprise standards for authentication, authorization, monitoring, error handling, and supportability.
  • Design AI-enabled workflows with clear boundaries around what the system can retrieve, recommend, automate, escalate, or hand off to a human.
• Production Engineering & Technical Leadership
  • Apply strong software engineering practices to AI systems, including code quality, tests, CI/CD, deployment discipline, monitoring, runbooks, incident follow-up, and operational support.
  • Optimize AI capabilities for latency, cost, reliability, scalability, maintainability, traceability, and user experience.
  • Document architecture decisions, implementation patterns, evaluation approaches, known limitations, and operational handoffs so teams can maintain and extend AI capabilities responsibly.
  • Mentor peers and contribute to shared engineering standards, reusable patterns, code reviews, design reviews, and practical guidance for applied AI delivery.

Requirements

• Education
  • Bachelor’s degree in Computer Science, Engineering, Data Science, Information Systems, Mathematics, or related technical field.
  • Equivalent combination of education and relevant technical experience may be considered.
• Experience
  • 6+ years of production software engineering, applied AI engineering, ML engineering, data-intensive application development, backend platform engineering, or related technology experience.
  • Meaningful hands-on experience building LLM-powered applications, AI agents, RAG workflows, prompt/context systems, evaluation assets, AI-assisted workflows, or comparable applied AI capabilities.
  • Experience designing and deploying end-to-end technical capabilities from requirements and prototypes through production implementation, monitoring, and iteration.
  • Experience working with product, design, engineering, data, analytics, quality, security, privacy, and business stakeholders to ship reliable software or AI-enabled product features.
  • Healthcare, life sciences, financial services, or other regulated-industry experience preferred but not required.
• Technical Expertise
  • Strong programming skills, preferably Python, with backend engineering experience across services, APIs, integrations, data flows, and production application components.
  • Hands-on familiarity with LLM application development, including prompt engineering, RAG pipelines, embeddings, vector or hybrid retrieval, tool calling, structured outputs, and evaluation methods.
  • Familiarity with agentic systems and orchestration patterns, including multi-step workflows, state management, memory, routing, decision logic, human-in-the-loop controls, and emerging frameworks such as LangGraph, LangChain, LlamaIndex, CrewAI, Google ADK, OpenAI Agents SDK, or comparable tools.
  • Understanding of cloud and production engineering practices, including containers, CI/CD, monitoring, logging, deployment, security, privacy, latency, cost, reliability, and operational support.
  • Ability to work with data and integration patterns, including APIs, event flows, data products, enterprise systems, searchable knowledge sources, and scalable architectures.
  • Ability to test and evaluate AI-enabled systems, including response quality, groundedness, source attribution, task completion, hallucination risk, latency, cost, safety, and reliability.

Key Success Factors

• Proven ability to ship production software with real reliability, latency, maintainability, and operational support expectations.
• Hands-on experience building LLM-powered applications, agents, RAG workflows, tool integrations, evaluation harnesses, or other applied AI systems beyond simple demos or thin API wrappers.
• Strong backend engineering fundamentals, including Python, APIs, cloud services, data flows, testing, CI/CD, observability, and production debugging.
• Ability to reason through agent and AI system failures, including retrieval gaps, context problems, prompt failures, tool-call errors, hallucination risk, latency issues, cost challenges, and workflow breakdowns.
• Strong product orientation and ability to partner with product, design, clinical, operational, and business stakeholders to ensure AI capabilities solve meaningful problems.
• Ability to balance innovation speed with privacy, security, reliability, evaluation discipline, auditability, cost, and responsible AI expectations.
• Comfort operating in a regulated or high-trust environment where customer trust, safety, and operational reliability are essential.