Alison Gopnik - Empowerment as causal learning, causal learning as empowerment: A bridge between Bayesian causal hypothesis testing and reinforcement learning

Learning about the causal structure of the world is a fundamental problem for human cognition, and causal knowledge is central to both intuitive and scientific theories. Cognitive scientists have applied advances in our formal understanding of causation in philosophy and computer science, particularly within the Causal Bayes Net formalism, to understand human causal learning. In parallel, in the very different tradition of reinforcement learning, researchers have developed the idea of an intrinsic reward signal called “empowerment”. An agent is rewarded for maximizing the mutual information between its actions and their outcomes, regardless of the external reward value of those outcomes. In other words, the agent is rewarded if variation in an action systematically leads to parallel variation in an outcome so that variation in the action predicts variation in the outcome. The result is an agent that has maximal control over its environment. This is very close to the conception of causal knowledge and learning in the prevailing “interventionist” accounts of causal knowledge that underpin the Bayes net formalism. I argue that “empowerment” may be an important bridge between classical Bayesian causal learning and reinforcement learning and may help to characterize causal learning in humans and enable it in machines. Empowerment may also explain distinctive empirical features of children’s causal learning, as well as providing a more tractable computational account of how that learning is possible. Finally I will present some empirical results exploring empowerment in children and adults.