Agentic AI, LLM Evaluation, and Trustworthy Systems Research Internship

About the role

Siemens Research & Predevelopment (RPD) is the central R&D department of Siemens and plays a key role in shaping the future of our products. Our global, diverse team supports the executive units of Siemens by acting as a strategic partner. The main research focus is on future technologies for industry, infrastructure, mobility, and healthcare.

Responsibilities

• Research and develop scalable intelligent systems using LLMs and semantic technologies.
• Contribute to groundbreaking research focused on implementing a Verification and Validation (V&V) framework for multi-agent systems.
• Research and prototype next-generation methods for LLM and multi-agent system evaluation, including benchmarks, guardrails, failure-mode analysis, runtime monitoring, formal methods, and testing technologies.
• Collaborate with researchers and engineers to define milestones, run experiments, analyze results, and translate research insights into scalable industrial software concepts.

Qualifications

• Currently enrolled in a PhD program in Computer Science, Artificial Intelligence, Machine Learning, Software Engineering, Formal Methods, or a closely related technical field.
• 3+ years of research or hands-on experience in AI, machine learning, generative AI, software engineering, formal methods, autonomous systems, or intelligent agent systems.
• Strong programming skills in Python and practical experience with modern ML or LLM tooling such as PyTorch, Hugging Face Transformers, LangChain, LangGraph, AutoGen, Semantic Kernel, CrewAI, or comparable frameworks.
• Hands-on experience building, evaluating, or testing LLM-powered applications, agentic workflows, multi-agent systems, or AI-enabled software engineering tools.
• Strong understanding of software architecture, software engineering principles, testing methodologies, experimentation, and empirical evaluation of complex systems.
• Demonstrated ability to conduct independent research, read and synthesize technical literature, analyze complex problems, prototype solutions, and communicate findings clearly.
• Proficient in English, both written and verbal.

Preferred Skills

• Research experience in formal verification, model checking, theorem proving, runtime verification, AI safety, robust AI, explainable AI (XAI), or trustworthy machine learning.
• Experience with evaluation of LLMs or agents, including hallucination analysis, benchmark design, tool-use evaluation, prompt-injection testing, red teaming, or reliability metrics.
• Familiarity with RAG architectures, vector databases, knowledge graphs, semantic technologies, ontologies, or graph-based reasoning.
• Understanding of reinforcement learning, planning, reward modeling, preference optimization, or post-training approaches for LLMs and autonomous agents.
• Experience with cloud-native or distributed systems concepts, microservice architectures, APIs, CI/CD, Git, Docker, Kubernetes, Azure, AWS, or comparable platforms.
• Experience with testing frameworks for complex software systems, including property-based testing, fuzz testing, simulation-based testing, static analysis, or execution-based evaluation.
• Track record of research publications, open-source contributions, academic projects, or demonstrable prototypes related to AI, software engineering, formal methods, or agentic systems.
• Excellent problem-solving skills, attention to detail, and ability to quickly learn and apply new technologies, tools, and research methods.
• Strong written and verbal communication skills, with the ability to articulate complex technical concepts to research and engineering audiences.