About Me
I am a researcher at the German Research Center for Artificial Intelligence (DFKI) and a Ph.D. student at Saarland University. I study AI agents that solve sequential decision-making tasks, such as, logistics, industrial manufacturing, and autonomous driving. My work spans three connected questions: how to learn such agents, how to evaluate them rigorously, and how to interpret their learned behavior.
Learning. I develop neuro-symbolic policies that generalize across problem sizes, meaning agents trained on small problems that reliably solve much larger ones. I have studied which architectures and training objectives make this possible, including Graph Neural Networks, Transformers, Supervised Learning, and Deep Reinforcement Learning. The resulting policies are faster and generalize better than standard policy approaches and state-of-the-art planning algorithms.
Evaluating. Training curves offer little insight into how an agent will behave once deployed. I therefore develop methods and tools that provide statistically sound assessments of agent behavior. These include statistical model checking of neural agents, the open-source tool PyDSMC, safety-aware training that concentrates learning on state-space regions where an agent performs poorly, and LLM-based pipelines that automatically generate the test instances required to assess generalization.
Interpreting. In ongoing work, I investigate methods for making learned agent behavior interpretable. One line of work automatically derives constraints that characterize the problem instances on which a black-box policy fails. Another uses Deep Reinforcement Learning to learn inherently interpretable policies for vision-based tasks.
My research sits at the intersection of Reinforcement Learning, Automated Planning, and Formal Methods, and appears at venues such as ICAPS, QEST+FORMATS, and ACM TOMACS.
Beyond my academic research, I apply my expertise in sequential decision-making to industry projects, where I design neuro-symbolic algorithms for optimizing manufacturing sequences.
Interests
- Sequential Decision-Making
- Reinforcement Learning
- Symbolic Planning
- Explainability
Education
PhD in Computer Science (Artificial Intelligence), ongoing
Supervised by Prof. Verena Wolf at Saarland University
Saarbrücken Graduate School of Computer Science PhD Preparatory Phase (Artificial Intelligence), 2024
Supervised by Prof. Verena Wolf at Saarland University
Grade: 1.2
Bachelor of Science Data Science and Artificial Intelligence (Artificial Intelligence), 2022
Supervised by Prof. Verena Wolf at Saarland University
Safe Reinforcement Learning Through Regret and State Restorations in Evaluation Stages
Grade: 1.0 (overall grade 1.6)
Publications (10)
2026
Per-Domain Generalizing Policies: On Learning Efficient and Robust Q-Value Functions
2026 — Proceedings of the Thirty-Sixth International Conference on Automated Planning and Scheduling. International Conference on Automated Planning and Scheduling (ICAPS-2026), June 27 - July 2, Dublin, Ireland
2026 — Workshop on Planning in the Era of LLMs at ICAPS-26
2025
Learning Per-Domain Generalizing Policies Using Offline Reinforcement Learning
2025 — Proceedings of the Ninth Workshop on Bridging the Gap Between AI Planning and (Reinforcement) Learning (PRL @ ICAPS 2025). Workshop on Bridging the Gap Between AI Planning and Reinforcement Learning (PRL-2025), located at ICAPS'25, November 10, Melbourne, Australia
PyDSMC: Statistical Model Checking for Neural Agents Using the Gymnasium Interface
2025 — Quantitative Evaluation of Systems and Formal Modeling and Analysis of Timed Systems - Second International Joint Conference, Proceedings. International Joint Conference on Quantitative Evaluation of Systems and Formal Modeling and Analysis of Timed Systems (QEST+FORMATS-2025), August 26-28, Aarhus, Denmark
Using Action-Policy Testing in RL to Reduce the Number of Bugs
2025 — Proceedings of the Eighteenth International Symposium on Combinatorial Search. International Symposium on Combinatorial Search (SoCS-2025), August 12-15, Scotland, United Kingdom
Per-Domain Generalizing Policies: On Validation Instances and Scaling Behavior
2025 — The 35th International Conference on Automated Planning and Scheduling. International Conference on Automated Planning and Scheduling (ICAPS-2025), November 9-14, Melbourne, Australia
2024
Comparing State-of-the-art Graph Neural Networks and Transformers for General Policy Learning
2024 — ICAPS Workshop on Planning and Reinforcement Learning (PRL)
Safe Reinforcement Learning Through Regret and State Restorations in Evaluation Stages
2024 — Workshop on Reliable Data-Driven Planning and Scheduling. International Conference on Automated Planning and Scheduling (ICAPS-2024)
2023
2023 — ACM Transactions on Modeling and Computer Simulation (TOMACS)
Neuro-explicit semantic segmentation of the diffusion cloud chamber
2023 — Review of Scientific Instruments
Teaching (2)
Neuro-symbolic General Policy Learning
Instructor (English), in Winter 2024/25
Seminar at Saarland UniversitySpecial Topics in Deep Reinforcement Learning
Assistant (English), in Summer 2024
Seminar at Saarland University
Supervised Students (8)
Naya Rudolph
Evaluating Per-Domain Generalizing Policies for Classical Planning
Research Assistant, ongoing
Ruslan Dontsov
Learning Inherently Interpretable Reinforcement Learning Policies
Bachelor's Thesis, 2026
Moritz Oster
Uniting Supervised and Reinforcement Learning for Per-domain Generalizing Policies
Master's Thesis, 2026
Magnus Cunow
Learning 4 Planning: Benchmarking Architectures and Using Heuristics for Multi-Target Learning
Master's Thesis, 2025
Dominik Grund
Symbolic, Neural, and Neuro-Symbolic Approaches for Optimizing Final Assembly Sequences for an Automotive Manufacturer
Master's Thesis, 2025
Pascal Held
Bridging the Gap between Fast Downward and PyTorch: Towards Efficient Search Using General Policies
Master's Thesis, 2025
Julius Gabelmann
Using Large Language Models to Enable Children to Search the Web
Research Assistant, 2025
Michel Scherer
Lifting the RARE Algorithm to Continuous State and Action Spaces
Bachelor's Thesis, 2025