Using Action-Policy Testing in RL to Reduce the Number of Bugs
Abstract
Reinforcement learning is becoming ever more prominent in solving combinatorial search problems, in particular ones where states are images. Prior work has devised action-policy testing methodology, that identifies so-called bug states where policy performance is sub-optimal. Here we show how to leverage this methodology during the RL process, using action-policy testing to find bugs and injecting those as alternate start states for the training runs. Running experiments across six 2D games, we find that our testing-guided training often achieves similar expected reward while reducing the number of bugs.
Type
Publication
Proceedings of the Eighteenth International Symposium on Combinatorial Search. International Symposium on Combinatorial Search (SoCS-2025), August 12-15, Scotland, United Kingdom
Cite
@inproceedings{eniserUsingActionPolicyTestingIn2025,
author = {Hasan Ferit Eniser and Songtuan Lin and Nicola J. Müller and Anastasia Isychev and Valentin Wüstholz and Isabel Valera and Jörg Hoffmann and Maria Christakis},
booktitle = {Proceedings of the Eighteenth International Symposium on Combinatorial Search. International Symposium on Combinatorial Search (SoCS-2025), August 12-15, Scotland, United Kingdom},
doi = {10.1609/socs.v18i1.35990},
editor = {Maxim Likhachev and Hana Rudová and Enrico Scala},
isbn = {13 978-1-57735-901-2},
month = {8},
pages = {181-185},
publisher = {AAAI Press, Washington, DC, USA},
title = {Using Action-Policy Testing in RL to Reduce the Number of Bugs},
url = {https://ojs.aaai.org/index.php/SOCS/article/view/35990},
volume = {18},
year = {2025}
}