Nicola J. Müller 🐱

he/himNicola J. Müller

PhD Student

DFKI – Neuro-Mechanistic Modeling

Saarland Informatics Campus

Campus D3.2 (DFKI), Room 2.11

66123 Saarbrücken

Germany

About Me

I am a researcher at the German Research Center for Artificial Intelligence (DFKI) and a Ph.D. student at Saarland University. I study AI agents that solve sequential decision-making tasks, such as, logistics, industrial manufacturing, and autonomous driving. My work spans three connected questions: how to learn such agents, how to evaluate them rigorously, and how to interpret their learned behavior.

Learning. I develop neuro-symbolic policies that generalize across problem sizes, meaning agents trained on small problems that reliably solve much larger ones. I have studied which architectures and training objectives make this possible, including Graph Neural Networks, Transformers, Supervised Learning, and Deep Reinforcement Learning. The resulting policies are faster and generalize better than standard policy approaches and state-of-the-art planning algorithms.

Evaluating. Training curves offer little insight into how an agent will behave once deployed. I therefore develop methods and tools that provide statistically sound assessments of agent behavior. These include statistical model checking of neural agents, the open-source tool PyDSMC, safety-aware training that concentrates learning on state-space regions where an agent performs poorly, and LLM-based pipelines that automatically generate the test instances required to assess generalization.

Interpreting. In ongoing work, I investigate methods for making learned agent behavior interpretable. One line of work automatically derives constraints that characterize the problem instances on which a black-box policy fails. Another uses Deep Reinforcement Learning to learn inherently interpretable policies for vision-based tasks.

My research sits at the intersection of Reinforcement Learning, Automated Planning, and Formal Methods, and appears at venues such as ICAPS, QEST+FORMATS, and ACM TOMACS.

Beyond my academic research, I apply my expertise in sequential decision-making to industry projects, where I design neuro-symbolic algorithms for optimizing manufacturing sequences.

Interests

  • Sequential Decision-Making
  • Reinforcement Learning
  • Symbolic Planning
  • Explainability

Education

Publications (10)

2026

2 papers

Per-Domain Generalizing Policies: On Learning Efficient and Robust Q-Value Functions

2026 — Proceedings of the Thirty-Sixth International Conference on Automated Planning and Scheduling. International Conference on Automated Planning and Scheduling (ICAPS-2026), June 27 - July 2, Dublin, Ireland

@inproceedings{mullerPerDomainGeneralizingPoliciesOn2026,
 author = {Müller, Nicola J. and Oster, Moritz and Valera, Isabel and Hoffmann, Jörg and Gros, Timo P.},
 booktitle = {Proceedings of the Thirty-Sixth International Conference on Automated Planning and Scheduling. International Conference on Automated Planning and Scheduling (ICAPS-2026), June 27 - July 2, Dublin, Ireland},
 doi = {10.1609/icaps.v36i1.42889},
 isbn = {ISSN 2334-0843},
 month = {6},
 number = {1},
 pages = {703-707},
 publisher = {AAAI Press},
 title = {Per-Domain Generalizing Policies: On Learning Efficient and Robust Q-Value Functions},
 url = {https://ojs.aaai.org/index.php/ICAPS/article/view/42889},
 volume = {36},
 year = {2026}
}
DOI URL

Toward a General Framework for Evaluating Per-Domain Generalization Using LLMs, Theorem Provers, and Statistical Model Checking

2026 — Workshop on Planning in the Era of LLMs at ICAPS-26

@inproceedings{mullerTowardAGeneralFramework2026,
 author = {Müller, Nicola J. and Rudolph, Naya and Taitler, Ayal and Gros, Timo P.},
 booktitle = {Workshop on Planning in the Era of LLMs at ICAPS-26},
 title = {Toward a General Framework for Evaluating Per-Domain Generalization Using LLMs, Theorem Provers, and Statistical Model Checking},
 url = {https://openreview.net/pdf?id=MLnzKKBdT4},
 year = {2026}
}
URL

2025

4 papers

Learning Per-Domain Generalizing Policies Using Offline Reinforcement Learning

2025 — Proceedings of the Ninth Workshop on Bridging the Gap Between AI Planning and (Reinforcement) Learning (PRL @ ICAPS 2025). Workshop on Bridging the Gap Between AI Planning and Reinforcement Learning (PRL-2025), located at ICAPS'25, November 10, Melbourne, Australia

PDF
@inproceedings{mullerLearningPerDomainGeneralizingPolicies2025,
 author = {Nicola J. Müller and Moritz Oster and Timo P. Gros},
 booktitle = {Proceedings of the Ninth Workshop on Bridging the Gap Between AI Planning and (Reinforcement) Learning (PRL @ ICAPS 2025). Workshop on Bridging the Gap Between AI Planning and Reinforcement Learning (PRL-2025), located at ICAPS'25, November 10, Melbourne, Australia},
 month = {11},
 publisher = {AAAI Press},
 title = {Learning Per-Domain Generalizing Policies Using Offline Reinforcement Learning},
 url = {https://prl-theworkshop.github.io/prl2025-icaps/papers/11.pdf},
 year = {2025}
}

PyDSMC: Statistical Model Checking for Neural Agents Using the Gymnasium Interface

2025 — Quantitative Evaluation of Systems and Formal Modeling and Analysis of Timed Systems - Second International Joint Conference, Proceedings. International Joint Conference on Quantitative Evaluation of Systems and Formal Modeling and Analysis of Timed Systems (QEST+FORMATS-2025), August 26-28, Aarhus, Denmark

@inproceedings{grosPyDSMCStatisticalModelChecking2025,
 author = {Timo P. Gros and Arnd Hartmanns and Ivo Hoese and Joshua Meyer and Nicola J. Müller and Verena Wolf},
 booktitle = {Quantitative Evaluation of Systems and Formal Modeling and Analysis of Timed Systems - Second International Joint Conference, Proceedings. International Joint Conference on Quantitative Evaluation of Systems and Formal Modeling and Analysis of Timed Systems (QEST+FORMATS-2025), August 26-28, Aarhus, Denmark},
 doi = {10.1007/978-3-032-05792-1_8},
 editor = {Pavithra Prabhakar and Andrea Vandin},
 isbn = {978-3-032-05791-4},
 month = {10},
 pages = {134-156},
 publisher = {Springer Nature, Switzerland},
 series = {Lecture Notes in Computer Science, LNCS},
 title = {PyDSMC: Statistical Model Checking for Neural Agents Using the Gymnasium Interface},
 url = {https://link.springer.com/chapter/10.1007/978-3-032-05792-1_8},
 volume = {16143},
 year = {2025}
}
DOI URL

Using Action-Policy Testing in RL to Reduce the Number of Bugs

2025 — Proceedings of the Eighteenth International Symposium on Combinatorial Search. International Symposium on Combinatorial Search (SoCS-2025), August 12-15, Scotland, United Kingdom

@inproceedings{eniserUsingActionPolicyTestingIn2025,
 author = {Hasan Ferit Eniser and Songtuan Lin and Nicola J. Müller and Anastasia Isychev and Valentin Wüstholz and Isabel Valera and Jörg Hoffmann and Maria Christakis},
 booktitle = {Proceedings of the Eighteenth International Symposium on Combinatorial Search. International Symposium on Combinatorial Search (SoCS-2025), August 12-15, Scotland, United Kingdom},
 doi = {10.1609/socs.v18i1.35990},
 editor = {Maxim Likhachev and Hana Rudová and Enrico Scala},
 isbn = {13 978-1-57735-901-2},
 month = {8},
 pages = {181-185},
 publisher = {AAAI Press, Washington, DC, USA},
 title = {Using Action-Policy Testing in RL to Reduce the Number of Bugs},
 url = {https://ojs.aaai.org/index.php/SOCS/article/view/35990},
 volume = {18},
 year = {2025}
}
DOI URL

Per-Domain Generalizing Policies: On Validation Instances and Scaling Behavior

2025 — The 35th International Conference on Automated Planning and Scheduling. International Conference on Automated Planning and Scheduling (ICAPS-2025), November 9-14, Melbourne, Australia

PDF
@inproceedings{grosPerDomainGeneralizingPoliciesOn2025,
 author = {Timo P. Gros and Nicola J. Müller and Daniel Fiser and Isabel Valera and Verena Wolf and Jörg Hoffmann},
 booktitle = {The 35th International Conference on Automated Planning and Scheduling. International Conference on Automated Planning and Scheduling (ICAPS-2025), November 9-14, Melbourne, Australia},
 publisher = {AAAI Press},
 title = {Per-Domain Generalizing Policies: On Validation Instances and Scaling Behavior},
 url = {https://doi.org/10.5281/zenodo.15314878},
 year = {2025}
}
URL

2024

2 papers

Comparing State-of-the-art Graph Neural Networks and Transformers for General Policy Learning

2024 — ICAPS Workshop on Planning and Reinforcement Learning (PRL)

@misc{mullerComparingStateOfTheArtGraphNeural2024,
 author = {Nicola J. Müller and Pablo Sánchez and Jörg Hoffmann and Verena Wolf and Timo P. Gros},
 booktitle = {ICAPS Workshop on Planning and Reinforcement Learning (PRL)},
 title = {Comparing State-of-the-art Graph Neural Networks and Transformers for General Policy Learning},
 year = {2024}
}

Safe Reinforcement Learning Through Regret and State Restorations in Evaluation Stages

2024 — Workshop on Reliable Data-Driven Planning and Scheduling. International Conference on Automated Planning and Scheduling (ICAPS-2024)

@inproceedings{grosSafeReinforcementLearningThrough2024,
 author = {Timo P. Gros and Nicola J. Müller and Daniel Höller and Verena Wolf},
 booktitle = {Workshop on Reliable Data-Driven Planning and Scheduling. International Conference on Automated Planning and Scheduling (ICAPS-2024)},
 publisher = {Springer},
 title = {Safe Reinforcement Learning Through Regret and State Restorations in Evaluation Stages},
 year = {2024}
}

2023

2 papers

DSMC Evaluation Stages: Fostering Robust and Safe Behavior in Deep Reinforcement Learning--Extended Version

2023 — ACM Transactions on Modeling and Computer Simulation (TOMACS)

@article{grosDSMCEvaluationStagesFostering2023,
 author = {Timo P. Gros and Joschka Groß and Daniel Höller and Jörg Hoffmann and Michaela Klauck and Hendrik Meerkamp and Nicola J. Müller and Lukas Schaller and Verena Wolf},
 doi = {10.1145/3607198},
 journal = {ACM Transactions on Modeling and Computer Simulation (TOMACS)},
 month = {10},
 number = {4},
 pages = {1-28},
 publisher = {ACM, New York, NY, USA},
 title = {DSMC Evaluation Stages: Fostering Robust and Safe Behavior in Deep Reinforcement Learning--Extended Version},
 volume = {33},
 year = {2023}
}
DOI

Neuro-explicit semantic segmentation of the diffusion cloud chamber

2023 — Review of Scientific Instruments

PDF
@article{mullerNeuroExplicitSemanticSegmentationOf2023,
 author = {Nicola J. Müller and Daniel Porawski and Lukas Wilde and Dennis Fink and Guillaume Trap and Annika Engel and Georges P. Schmartz},
 doi = {10.1063/5.0109284},
 editor = {Robert P. Behringer},
 journal = {Review of Scientific Instruments},
 month = {6},
 number = {6},
 pages = {063304},
 publisher = {AIP Publishing},
 title = {Neuro-explicit semantic segmentation of the diffusion cloud chamber},
 url = {https://www.dfki.de/fileadmin/user_upload/import/16919_Neuro-explicit_semantic_segmentation_of_the_diffus.pdf},
 volume = {94},
 year = {2023}
}
DOI

Teaching (2)

Supervised Students (8)

  • Naya Rudolph

    Evaluating Per-Domain Generalizing Policies for Classical Planning

    Research Assistant, ongoing

  • Ruslan Dontsov

    Learning Inherently Interpretable Reinforcement Learning Policies

    Bachelor's Thesis, 2026

  • Moritz Oster

    Uniting Supervised and Reinforcement Learning for Per-domain Generalizing Policies

    Master's Thesis, 2026

  • Magnus Cunow

    Learning 4 Planning: Benchmarking Architectures and Using Heuristics for Multi-Target Learning

    Master's Thesis, 2025

  • Dominik Grund

    Symbolic, Neural, and Neuro-Symbolic Approaches for Optimizing Final Assembly Sequences for an Automotive Manufacturer

    Master's Thesis, 2025

  • Pascal Held

    Bridging the Gap between Fast Downward and PyTorch: Towards Efficient Search Using General Policies

    Master's Thesis, 2025

  • Julius Gabelmann

    Using Large Language Models to Enable Children to Search the Web

    Research Assistant, 2025

  • Michel Scherer

    Lifting the RARE Algorithm to Continuous State and Action Spaces

    Bachelor's Thesis, 2025