Senior Applied AI Engineer

About the role

LTS is seeking a highly skilled Senior Applied AI Engineer to focus on continuously improving the intelligence behind the Agentic AI platform. The platform helps engineers understand, analyze, and modernize a critical legacy software system, enabling them to ask questions in plain English and receive transparent, verifiable answers.

Responsibilities

• Advance Applied AI Capabilities
• Design, prototype, and implement production-ready AI capabilities that improve reasoning, accuracy, explainability, and developer productivity.
• Evaluate emerging LLMs, multimodal models, agent frameworks, and AI techniques to identify opportunities for platform advancement.
• Rapidly prototype new AI capabilities and transition successful experiments into production.
• Optimize Agent Performance
• Improve autonomous and multi-agent workflows through prompt engineering, reasoning optimization, memory strategies, tool selection, and context management.
• Continuously refine Retrieval-Augmented Generation (RAG) pipelines, retrieval strategies, embeddings, reranking, and grounding techniques.
• Improve AI response quality through experimentation, benchmarking, and iterative optimization.
• Evaluate AI Systems
• Develop evaluation frameworks that measure accuracy, groundedness, explainability, latency, and overall AI effectiveness.
• Create benchmark datasets, automated evaluation pipelines, and performance metrics for production AI systems.
• Analyze AI failures, hallucinations, retrieval gaps, and reasoning errors to drive continuous improvement.
• Knowledge Engineer
• Collaborate with software engineers to improve knowledge ingestion, document processing, semantic search, embeddings, and enterprise knowledge management.
• Design approaches that maximize retrieval quality across large technical documentation and source code repositories.
• Improve how AI agents discover, organize, and reason over enterprise knowledge.
• Collaborate Across Engineers
• Partner closely with AI architects, platform engineers, software engineers, and front-end engineers to improve the overall intelligence of the platform.
• Share research findings, experimental results, and engineering recommendations with cross-functional teams.
• Help establish best practices for experimentation, evaluation, and AI quality throughout the organization.

Requirements

• Bachelor's degree in Computer Science, Artificial Intelligence, Machine Learning, Software Engineering, Engineering, Data Science, or a related technical discipline (or equivalent professional experience).
• 5+ years of software engineering, applied AI, machine learning, or AI systems development experience.
• Demonstrated experience developing production AI applications powered by Large Language Models (LLMs).
• Experience designing and optimizing Retrieval-Augmented Generation (RAG) systems.
• Experience with prompt engineering, embeddings, semantic search, vector databases, and knowledge retrieval.
• Experience evaluating AI model performance and implementing experimentation frameworks.
• Strong programming skills in Python and experience with modern software engineering practices.
• Experience with AI frameworks such as LangGraph, LangChain, LlamaIndex, Semantic Kernel, CrewAI, AutoGen, or similar.
• Experience with using AI coding assistants as part of your daily workflow.
• Familiarity with REST APIs, cloud-native applications, and distributed software systems.
• Strong analytical, problem-solving, and communication skills.
• Intellect and curiosity for AI systems and how they behave.
• Deep passion for experimenting with new AI techniques.
• Background in evaluation, explainability, and continuous improvement.
• Proven success with ownership of difficult technical challenges and collaboration across disciplines.

Qualifications

• Nice to have:
• Experience optimizing autonomous or multi-agent AI systems.
• Experience implementing automated AI evaluation frameworks.
• Experience with OpenAI, Azure OpenAI, Anthropic Claude, Google Vertex AI, AWS Bedrock, or open-source LLMs.
• Experience with vector databases including Pinecone, Weaviate, Qdrant, Milvus, or Azure AI Search.
• Experience with Responsible AI, AI governance, safety, and explainability.
• Familiarity with software engineering tools, code intelligence platforms, or developer productivity solutions.
• Experience supporting healthcare, Federal Government, or other highly regulated environments.
• Experience using AI coding assistants and autonomous agents as part of daily software development.