Implementing Human Information-Seeking Behaviour with Action-Agnostic Bayesian Surprise

In this paper, we aim to establish a link between model learning and the mechanism of curiosity. The main hypothesis is that exploration bonuses, as proposed in the reinforcement learning literature, are connected to Bayesian estimation principles through the construction of a parametric model of causal relationships between actions and observations. The core idea is to interpret Bayesian surprise as an estimate of information transfer between the observed data (including observations and motor commands) and the model parameters. We outline the general principles of this approach and present results suggesting that action selection guided by information transfer can explain certain experimental, behavioral, and neurological data in humans.