Empowerment and Causal Learning

In this work, we are interested in bridging causal learning in humans and reinforcement learning (RL) in agents. Earlier work in cognitive science on causal learning has found that both adults and children are strongly motivated to discover causal structure in their environment. Meanwhile, research in RL has focused on learning to maximize rewards without explicitly attempting to discover causal structure. We hypothesize that the concept of “empowerment” in reinforcement learning can provide a bridge between reinforcement learning and causal learning. “Empowerment” is an intrinsic reward that involves maximizing the mutual information between an agent’s actions and outcomes in the world, and so maximizing the agent’s ability to control the environment, rather than maximizing particular external rewards. This ability to control the environment is also at the heart of “interventionist” accounts of causality and causal learning (e.g. Woodward, 2005; Pearl 2000, 2009). From the machine learning perspective Empowerment may thus be an especially promising intrinsic motivation for RL agents to discover causal structure. From the cognitive science perspective we will explore whether human causal learning can be explained by a drive to maximize empowerment, compared to other forms of novelty-seeking drives.