Document Type
Conference Paper
Disciplines
1.2 COMPUTER AND INFORMATION SCIENCE, Computer Sciences
Abstract
Reinforcement Learning (RL) has shown promise in optimizing complex control and decision-making processes but Deep Reinforcement Learning (DRL) lacks interpretability, limiting its adoption in regulated sectors like manufacturing, finance, and healthcare. Difficulties arise from DRL’s opaque decision-making, hindering efficiency and resource use, this issue is amplified with every advancement. While many seek to move from Experience Replay to A3C, the latter demands more resources. Despite efforts to improve Experience Replay selection strategies, there is a tendency to keep capacity high. This dissertation investigates training a Deep Convolutional Q-learning agent across 20 Atari games, in solving a control task, physics task, and simulating addition, while intentionally reducing Experience Replay capacity from 1×106 to 5×102 . It was found that over 40% in the reduction of Experience Replay size is allowed for 18 of 23 simulations tested, offering a practical path to resource-efficient DRL. To illuminate agent decisions and align them with game mechanics, a novel method is employed: visualizing Experience Replay via Deep SHAP Explainer. This approach fosters comprehension and transparent, interpretable explanations, though any capacity reduction must be cautious to avoid overfitting. This study demonstrates the feasibility of reducing Experience Replay and advocates for transparent, interpretable decision explanations using the Deep SHAP Explainer to promote enhancing resource efficiency in Experience Replay.
Recommended Citation
Sullivan, R.S. (2023). Explaining Deep Q-Learning Experience Replay with SHapley Additive exPlanations. [Technological University Dublin].
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Publication Details
A dissertation submitted in partial fulfillment of the requirements of Technological University Dublin for the degree of M.Sc. in Computer Science (Data Science) June 16, 2023.