DSMC Evaluation Stages: Fostering Robust and Safe Behavior in Deep Reinforcement Learning--Extended Version
Abstract
Neural networks (NN) are gaining importance in sequential decision-making. Deep reinforcement learning (DRL), in particular, is extremely successful in learning action policies in complex and dynamic environments. Despite this success, however, DRL technology is not without its failures, especially in safety-critical applications: (i) the training objective maximizes average rewards, which may disregard rare but critical situations and hence lack local robustness; (ii) optimization objectives targeting safety typically yield degenerated reward structures, which, for DRL to work, must be replaced with proxy objectives. Here, we introduce a methodology that can help to address both deficiencies. We incorporate evaluation stages (ES) into DRL, leveraging recent work on deep statistical model checking (DSMC), which verifies NN policies in Markov decision processes. Our ES apply DSMC at regular intervals to determine …
Cite
@article{grosDSMCEvaluationStagesFostering2023,
author = {Timo P. Gros and Joschka Groß and Daniel Höller and Jörg Hoffmann and Michaela Klauck and Hendrik Meerkamp and Nicola J. Müller and Lukas Schaller and Verena Wolf},
doi = {10.1145/3607198},
journal = {ACM Transactions on Modeling and Computer Simulation (TOMACS)},
month = {10},
number = {4},
pages = {1-28},
publisher = {ACM, New York, NY, USA},
title = {DSMC Evaluation Stages: Fostering Robust and Safe Behavior in Deep Reinforcement Learning--Extended Version},
volume = {33},
year = {2023}
}