Staff Applied AI Engineer, Product & Agent Performance

Responsibilities

• Design and iterate on agent behavior across healthcare workflows
• Architect retrieval and context to ensure data integrity for AI models
• Handle memory and state in multi-turn agent interactions
• Create prompt templates integrating structured formatting and reasoning
• Conduct evaluations to measure performance and identify regressions
• Author evaluation rubrics that weight failures by severity and cost
• Design escalation paths for agents based on confidence levels

Qualifications

• 8+ years of production software engineering experience
• 3+ years of experience in ML, LLM, or agentic systems
• Demonstrated expertise in diagnosing agent failures and implementing effective solutions
• Hands-on experience with RAG architecture and frameworks for evaluating automated systems
• Familiarity with AWS AI/ML services, particularly Bedrock and SageMaker
• Ability to make evidence-led judgments, with experience in experimental design and implementation

Benefits

• Ownership over agent performance and safety metrics for healthcare workflows
• Cross-functional role linking production evidence to AI enhancements
• Mission-driven work aimed at improving patient care delivery
• Flexible and remote-friendly work culture
• Access to employee-driven development programs
• Integration into a diverse and purpose-driven community