Learning Per-Domain Generalizing Policies Using Offline Reinforcement Learning

Abstract

Learned per-domain generalizing policies are gaining popularity in classical planning, as they can solve arbitrary instances of a specific domain. They are typically trained using supervised learning (SL), where we learn to generalize beyond a training set, or reinforcement learning (RL), where we learn from scratch through trial-and-error. We argue that SL and RL should not be seen as contrasting approaches, and propose a training framework where a policy is first trained offline using SL, and then finetuned online using RL. The key method enabling this framework is offline RL. Preliminary experiments show that offline RL can indeed learn perdomain generalizing policies effectively.

@inproceedings{mullerLearningPerDomainGeneralizingPolicies2025,
 author = {Nicola J. Müller and Moritz Oster and Timo P. Gros},
 booktitle = {Proceedings of the Ninth Workshop on Bridging the Gap Between AI Planning and (Reinforcement) Learning (PRL @ ICAPS 2025). Workshop on Bridging the Gap Between AI Planning and Reinforcement Learning (PRL-2025), located at ICAPS'25, November 10, Melbourne, Australia},
 month = {11},
 publisher = {AAAI Press},
 title = {Learning Per-Domain Generalizing Policies Using Offline Reinforcement Learning},
 url = {https://prl-theworkshop.github.io/prl2025-icaps/papers/11.pdf},
 year = {2025}
}