Timezone: »

 
Poster
Explainability Via Causal Self-Talk
Nicholas Roy · Junkyung Kim · Neil Rabinowitz

Thu Dec 01 09:00 AM -- 11:00 AM (PST) @ Hall J #613
in Poster Session 5 »

Explaining the behavior of AI systems is an important problem that, in practice, is generally avoided. While the XAI community has been developing an abundance of techniques, most incur a set of costs that the wider deep learning community has been unwilling to pay in most situations. We take a pragmatic view of the issue, and define a set of desiderata that capture both the ambitions of XAI and the practical constraints of deep learning. We describe an effective way to satisfy all the desiderata: train the AI system to build a causal model of itself. We develop an instance of this solution for Deep RL agents: Causal Self-Talk. CST operates by training the agent to communicate with itself across time. We implement this method in a simulated 3D environment, and show how it enables agents to generate faithful and semantically-meaningful explanations of their own behavior. Beyond explanations, we also demonstrate that these learned models provide new ways of building semantic control interfaces to AI systems.

Keywords: Causality · Deep Learning · Reinforcement Learning · interpretability · explainability

Author Information

Nicholas Roy (DeepMind)
Junkyung Kim (DeepMind)
Neil Rabinowitz (DeepMind)

More from the Same Authors