Timezone: »

What Did You Think Would Happen? Explaining Agent Behaviour through Intended Outcomes
Herman Yau · Chris Russell · Simon Hadfield

Wed Dec 09 09:00 AM -- 11:00 AM (PST) @ Poster Session 3 #969

We present a novel form of explanation for Reinforcement Learning, based around the notion of intended outcome. These explanations describe the outcome an agent is trying to achieve by its actions. We provide a simple proof that general methods for post-hoc explanations of this nature are impossible in traditional reinforcement learning. Rather, the information needed for the explanations must be collected in conjunction with training the agent. We derive approaches designed to extract local explanations based on intention for several variants of Q-function approximation and prove consistency between the explanations and the Q-values learned. We demonstrate our method on multiple reinforcement learning problems, and provide code to help researchers introspecting their RL environments and algorithms.

Author Information

Herman Yau (University of Surrey)
Chris Russell (Amazon Web Services)
Simon Hadfield (University of Surrey)

More from the Same Authors