On the Performance and Robustness of Linear Model U-Trees in Mimic Learning

Abstract

The Linear Model U-Tree (LMUT) has been used to increase the interpretability of Deep Reinforcement Learning (DRL) agents by mimicking behavior in terms of Q-value predictions and gameplay. In this paper, we consider two extensions to LMUT. First, we evaluate the impact of prepruning and bottomup postpruning on LMUT and find that while prepruning has a mixed to negligible impact on performance, postpruning brings its Q-value predictions closer in line with the DRL agent, increasing the effectiveness of its influence on DRL interpretability. Second, we find evidence that LMUT gameplay typically more closely matches that of the DRL agent it learns to mimic when the DRL agent policy is more robust to noise, even after controlling for the performance of the DRL agent on the underlying task. This indicates that LMUT efficacy is driven in part by the robustness of the DRL policy.