Saarland Informatics Campus
Campus D3.2 (DFKI), Room 2.24
66123 Saarbrücken, Germany
Contact:
+49 681 857755375
Office hour:
Whenever my door is open.
Other:
ResearchGate
GoogleScholarGithub
About me
I am a Ph.D. student in the Modeling and Simulation group of Prof. Dr. Verena Wolf and in the Foundations of Artificial Intelligence group of Prof. Dr. Jörg Hoffmann. Also, I am a member of the Graduate School of Computer Science. Since December 2023, I work at the German Research Center for Artificial Intelligence (DFKI). Especially, I am involved in the Centre for European Research in Trusted AI (CERTAIN) and the department of Neuro-mechanistic Modeling (NMM).
Publications
-
Safe Reinforcement Learning Through Regret and State Restorations in Evaluation Stages
Principles of Verification: Cycling the Probabilistic Landscape
Joint work with Nicola J. Müller, Daniel Höller, and Verena Wolf. -
Motion Primitives as the Action Space of Deep Q-Learning for Planning in Autonomous Driving
IEEE Transactions on Intelligent Transportation Systems.
Joint work with Tristan Schneider, Matheus Pedrosa, Verena Wolf, and Kathrin Flaßkamp. -
Safe Reinforcement Learning Through Regret and State Restorations in Evaluation Stages
Workshop on Reliable Data-Driven Planning and Scheduling (RDDPS), at ICAPS’24.
Joint work with Nicola J. Müller, Daniel Höller, and Verena Wolf. -
Comparing State-of-the-art Graph Neural Networks and Transformers for General Policy Learning
Workshop on Bridging the Gap Between AI Planning and Reinforcement Learning (PRL), at ICAPS’24.
Joint work with Nicola J. Müller, Pablo Sánchez, Jörg Hoffmann, and Verena Wolf.
-
XAI Requirements in Smart Production Processes: A Case Study.
World Conference on Explainable Artificial Intelligence.
Joint work with Deborah Baum, Kevin Baum, and Verena Wolf. -
Bridging the Gap Between AI Planning and Reinforcement Learning
Workshop at the 32nd International Joint Conference on Artificial Intelligence (IJCAI’23).
Joint work with Cameron Allen, Michael Katz, Harsha Kokel, Hector Palacios, and Sarath Sreedharan. -
DSMC Evaluation Stages: Fostering Robust and Safe Behavior in Deep Reinforcement Learning – Extended Version.
Proceedings of the 18th International Conference on Quantitative Evaluation of SysTems (QEST’21).
Joint work with Daniel Höller, Jörg Hoffmann, Michaela Klauck, Hendrik Meerkamp, Nicola J. Müller, Lukas Schaller, and Verena Wolf. -
Analyzing Neural Network Behavior Through Deep Statistical Model Checking.
International Journal on Software Tools for Technology Transfer (STTT).
Joint work with Holger Hermanns, Jörg Hoffmann, Michaela Klauck, and Marcel Steinmetz.
-
MoGym: Using Formal Models for Training and Verifying Decision-making Agents.
Proceedings of the International Conference on Computer Aided Verification (CAV’22).
Joint work with Holger Hermanns, Jörg Hoffmann, Michaela Klauck, Maximilian A. Köhl, and Verena Wolf. -
Glyph-Based Visual Analysis of Q-Learning Based Action Policy Ensembles on Racetrack.
Proceedings of the International Conference on Information Visualisation (IV’22).
Joint work with David Groß, Michaela Klauck, Marcel Steinmetz, Jörg Hoffmann, and Stefan Gumhold.
Best Paper Award -
Debugging a Policy: Automatic Action-Policy Testing in AI Planning.
Proceedings of the International Conference on Automated Planning and Scheduling (ICAPS’22).
Joint work with Marcel Steinmetz, Daniel Fišer, Hasan Ferit Eniser, Patrick Ferber, Philippe Heim, Daniel Höller, Xandra Schuler, Valentin Wüstholz, Maria Christakis, and Jörg Hoffmann. -
Metamorphic Relations via Relaxations: An Approach to Obtain Oracles for Action-Policy Testing.
Proceedings of the International Symposium on Software Testing and Analysis (ISSTA’22).
Joint work with Hasan Ferit Eniser, Valentin Wüstholz, Jörg Hoffmann, and Maria Christakis.
-
DSMC Evaluation Stages: Fostering Robust and Safe Behavior in Deep Reinforcement Learning.
Proceedings of the 18th International Conference on Quantitative Evaluation of SysTems (QEST’21).
Joint work with Daniel Höller, Jörg Hoffmann, Michaela Klauck, Hendrik Meerkamp, and Verena Wolf. -
Debugging a Policy: A Framework for Automatic Action Policy Testing.
Proceedings of the Workshop on Bridging the Gap Between AI Planning and Reinforcement Learning (PRL), at ICAPS’21.
Joint work with Marcel Steinmetz, Philippe Heim, Daniel Höller, and Jörg Hoffmann.
-
Lab Conditions for Research on Explainable Automated Decisions (Position Paper)
Post-Proceedings of the Workshop on Foundations of Trustworthy AI - Integrating Learning, Optimization and Reasoning (TAILOR’20).
Joint work with Christel Baier, Maria Christakis, David Groß, Stefan Gumhold, Holger Hermanns, and Michaela Klauck. -
Real-time Decision Making For a Car Manufacturing Process Using Deep Reinforcement Learning.
Proceedings of the 2020 Winter Simulation Conference.
Joint work with Joschka Groß and Verena Wolf. -
Tracking the Race Between Deep Reinforcement Learning and Imitation Learning.
Proceedings of the 17th International Conference on Quantitative Evaluation of SysTems (QEST’20).
Joint work with Daniel Höller, Jörg Hoffmann, and Verena Wolf. -
Tracking the Race Between Deep Reinforcement Learning and Imitation Learning – Extended Version.
Joint work with Daniel Höller, Jörg Hoffmann, and Verena Wolf. -
TraceVis: Towards Visualization for Deep Statistical Model Checking.
Proceedings of the 9th International Symposium On Leveraging Applications of Formal Methods, Verification and Validation (ISoLA’20)
Joint work with David Groß, Stefan Gumhold, Jörg Hoffmann, Michaela Klauck, and Marcel Steinmetz. -
Deep Statistical Model Checking.
Proceedings of the 40th International Conference on Formal Techniques for Distributed Objects, Components, and Systems (FORTE’20), 2020. Joint work with Holger Hermanns, Jörg Hoffmann, Michaela Klauck, and Marcel Steinmetz.
- aMAZEing Programming – Providing SKILLs to Fellow Students.
SKILL 2019 - Studierendenkonferenz Informatik Joint work with Pascal L. Held, Pascal Lauer, Niklas O. Metzger, and Kallistos Weis.
Theses
- Markov Automata taken by Storm, scientific work for studying (high school) teaching & Bachelor thesis, January 2018
- Tracking the Race: Analyzing Racetrack Agents Trained with Imitation Learning and Deep Reinforcement Learning, May 2021
Co-supervised Students
- Philipp Sauer, Masters’s thesis, ongoing.
- Michel Scherer, Bachelor’s thesis, ongoing.
- Ivo Hoese, Bachelor’s thesis, ongoing.
- Dennis Heß, Bachelor’s thesis, ongoing.
- Jan Kautenburger, Bachelor’s thesis, 2024.
GNPGNorks - Generalized Neural Policies Using Graph Neural Networks. - Ivo Hoese, research assistant, since 2023.
- Nicola Müller, research assistant, since 2022 - 2024.
DSMC Evaluation Stages with Regret. - Jan-Paul Schulz, Master’s thesis, 2024.
Extending the Racetrack Environment - Alexander Kunz, Bachelor’s thesis, 2023.
Using Deep Reinforcement Learning in the Steel Industry: A Game of Iron and Steel - Lukas Schaller, Master’s thesis, 2023.
Improving Description-based Q-learning - Nicola Müller, Bachelor’s thesis, 2022.
DSMC Evaluation Stages with Restart - Guiliano Rasper, scientific work for studying high school teaching, 2022.
Towards Undiscounted Q-learning - Michael Kormann, Bachelor thesis, 2022.
Reinforcement Learning with Reward Shaping for Planning Domains in Jani - Lukas Schaller, research assistant, 2021-2023.
Using Deep Reinforcement Learning for Conjunctive Port Mappings
DSMC Evaluation Stages: Fostering Robust and Safe Behavior in Deep Reinforcement Learning - Ilka Kopping, Master thesis, 2022.
Performance Comparison of Deep Reinforcement Learning Algorithms for the ProGen Benchmark - Christian Closheim, Bachelor thesis, 2022.
Deep Reinforcement Learning with LTL Auxiliary Tasks - Tristan Schneider, Bachelor thesis, 2022.
Motion Primitives for Motion Planning with Reinforcement Learning - Joschka Groß, Master thesis, 2021.
Using Deep Reinforcement Learning to Optimize Assignment Problems - Luis Paulus, Bachelor thesis, 2021.
Mapping Instructions to Ports: A Reinforcement Curriculum Learning Approach - Hendrik Meerkamp, Master thesis, 2020.
Deep Statistical Model Refinement - Roman Joeres: Bachelor thesis, 2020.
Multiple Sequence Alignment using Deep Reinforcement Learning - Anurag Das: Research Immersion Lab, 2020.
Real-time Decision Making For a Car Manufacturing Process Using Hierarchical Reinforcement Learning - Jeremias Wagner: Bachelor thesis, 2020.
Deep Reinforcement Learning to Battle Snake - Paul Krieger: Bachelor thesis, 2020.
Explainable Classification for an Application System
Teaching
- Winter 13 : Tutor for Programming 1 at the Dependable Systems and Software group
- Winter 14 : Tutor for Mathematics for Engineers 1 at the Chair for Numerical Mathematics
- Summer 15 : Tutor for Programming 2 at the Compiler Design Lab
- Winter 15 : Tutor for Basics of Algorithms and Data structures at the chair for Computational Complexity
- Summer 16 : Supervision-tutor for Programming 2 at the Compiler Design Lab
- Summer 17 : Supervision-tutor for Programming 2 at the Compiler Design Lab
- Summer 18 : Assistant for Concurrent Programming at the group of Modeling and Simulation
- Summer 19 : Assistant for Probabilistic Models and Data Analysis
- Winter 19: Assistant for Rein4cement Learning
- Summer 20: Assistent for Statistics Lab
- Summer 21: Assistent for Concurrent Programming
- Summer 21: Assistent for Deep Reinforcement Learning
- Winter 22: Assistent for Deep Reinforcement Learning
- Summer 24: Assistent for Special Topics in Deep Reinforcement Learning
For students
Other Work
I have reviewed manuscripts for ACM SIGSIM Conference on Principles of Advanced Discrete Simulation (PADS) 2020, Quantitative Evaluation of SysTems (QEST) 2020 - 2024, the 7th IFAC Conference on Analysis and Design of Hybrid Systems (ADHS) 2021, the ACM International Conference on Hybrid Systems: Computation and Control (HSCC) 2022 - 2024, the Artificial Intelligence Journal (AIJ), and the International Conference on Autonomous Agents and Multiagent Systems (AAMAS) 2024.
I was a co-organizer of the Bridging the Gap Between AI Planning and Reinforcement Learning (PRL) workshop at ICAPS 2023, IJCAI 2023. I am a co-organizer of the Bridging the Gap Between AI Planning and Reinforcement Learning (PRL) workshop at ICAPS 2024 and of the Workshop on Reliable Data-Driven Planning and Scheduling at ICAPS 2024.