AI & Decision Engineer

About the role

The Systems Performance Analysis Group (KBS) at the Johns Hopkins University Applied Physics Laboratory is seeking a Generative AI & Decision Engineer to design, build, and deploy AI-enabled analytical and decision-support applications for Naval and Air Force sponsors.

Responsibilities

• Architect, develop, and evolve a Generative AI-powered analytical applications that support human–LLM interactions and LLM-driven decision-making at scale.
• Design and implement LLM agents that can autonomously plan, reason, and act using tools such as RAG, analytical tools, and relevant databases.
• Apply prompt engineering techniques with state-of-the-art models to produce data-driven recommendations based operational and developmental data for a variety of systems.
• Design, implement, and manage RAG pipelines using vector databases and frameworks to integrate relevant documentation, prior analytical results, and large information repositories into LLM decision-making.
• Develop and maintain agents capable of: (1) Interpreting database schemas and generating SQL queries to answer user questions (SQL agents). (2) Translating natural language inputs into structured actions and game events stored in a database for downstream simulation and adjudication.
• Experiment with and evaluate prompting strategies to improve reasoning quality, robustness, and transparency of model outputs in high-consequence decision contexts.
• Integrate orchestration and observability tools (e.g., Prefect) to monitor LLM pipelines, track outputs, and provide real-time insight into system behavior during wargame execution.
• Fine-tune and adapt foundation models (e.g., Llama, Mistral) using AWS SageMaker, Hugging Face TRL, and synthetic data generation (e.g., GPT-4 series) to optimize performance on sponsor-specific tasks such as deductive coding, domain knowledge transfer, etc.
• Engage with internal leadership and external sponsors to demonstrate capabilities, collect requirements, and potentially transition systems to operational users.
• Document system architectures, experiments, evaluation results, and operational guidance; prepare and present technical briefings and reports to technical and non-technical stakeholders.
• Collaborate in multidisciplinary teams (AI/ML, software, human factors, operations analysts) to integrate AI capabilities into broader analytic and operational workflows.

Qualifications

• Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, Applied Mathematics, Data Science, or a closely related field.
• Experience building production or prototype applications involving large language models (LLMs) or other generative models.
• Proficiency in Python and at least one modern web/backend framework (e.g., FastAPI, Flask, Django).
• Experience with relational databases, including schema design and query development (e.g., PostgreSQL).
• Hands-on experience with at least one of: Retrieval-Augmented Generation (RAG) systems. LLM agent frameworks (tool use, ReAct, chain-of-thought–style prompting). Vector databases (e.g., Qdrant, ChromaDB).
• Demonstrated ability to comprehend and synthesize complex technical or scientific information and make timely, well-reasoned decisions.
• Strong written and oral communication skills, including experience preparing technical analyses and presenting findings to a range of audiences.
• Hold an active Secret security clearance and can ultimately obtain Top Secret level clearance.

Additional Information

This position may require occasional travel to sponsor sites and test events.

Candidates will be expected to work in multidisciplinary teams and contribute to both research and applied development efforts.

APL is committed to fostering an inclusive environment and encourages applications from diverse backgrounds.