Timezone: »

 
The emergence of visual simulation in task-optimized recurrent neural networks
Alekh Karkada Ashok · Lakshmi Narasimhan Govindarajan · Drew Linsley · David Sheinberg · Thomas Serre

@
in Shared Visual Representations in Human and Machine Intelligence (SVRHM) »
Event URL: https://openreview.net/forum?id=qYLp6nNU7m- »

Primates display remarkable prowess in making rapid visual inferences even when sensory inputs are impoverished. One hypothesis about how they accomplish this is through a process called visual simulation, in which they imagine future states of their environment using a constructed mental model. Though a growing body of behavioral findings, in both humans and non-human primates, provides credence to this hypothesis, the computational mechanisms underlying this ability remain poorly understood. In this study, we probe the capability of feedforward and recurrent neural network models to solve the Planko task, parameterized to systematically control task variability. We demonstrate that visual simulation emerges as the optimal computational strategy in deep neural networks only when task variability is high. Moreover, we provide some of the first evidence that information about imaginary future states can be decoded from the model latent representations, despite no explicit supervision. Taken together, our work suggests that the optimality of visual simulation is task-specific and provides a framework to test its mechanistic basis.

Author Information

Alekh Karkada Ashok (Brown University)
Lakshmi Narasimhan Govindarajan (Brown University)
Drew Linsley (Brown University)

We need artificial vision to create intelligent machines that can reason about the world, but existing artificial vision systems cannot solve many of the visual challenges that we encounter and routinely solve in our daily lives. I look to biological vision to inspire new solutions to challenges faced by artificial vision. I do this by testing complementary hypotheses that connect computational theory with systems- and cognitive-neuroscience level experimental research: 1. Computational challenges for artificial vision can be identified through systematic comparisons with biological vision, and solved with algorithms inspired by those of biological vision. 2. Improved algorithms for artificial vision will lead to better methods for gleaning insight from large-scale experimental data, and better models for understanding the relationship between neural computation and perception.

David Sheinberg (Brown University)
Thomas Serre (Brown University)

More from the Same Authors