Timezone: »

 
Poster
Learning long-range spatial dependencies with horizontal gated recurrent units
Drew Linsley · Junkyung Kim · Vijay Veerabadran · Charles Windolf · Thomas Serre

Tue Dec 04 07:45 AM -- 09:45 AM (PST) @ Room 210 #82
in Tue Poster Session A »

Progress in deep learning has spawned great successes in many engineering applications. As a prime example, convolutional neural networks, a type of feedforward neural networks, are now approaching -- and sometimes even surpassing -- human accuracy on a variety of visual recognition tasks. Here, however, we show that these neural networks and their recent extensions struggle in recognition tasks where co-dependent visual features must be detected over long spatial ranges. We introduce a visual challenge, Pathfinder, and describe a novel recurrent neural network architecture called the horizontal gated recurrent unit (hGRU) to learn intrinsic horizontal connections -- both within and across feature columns. We demonstrate that a single hGRU layer matches or outperforms all tested feedforward hierarchical baselines including state-of-the-art architectures with orders of magnitude more parameters.

Keywords: Recurrent Networks · Supervised Deep Networks · Deep Autoencoders · Computer Vision · Perception · Biologically Plausible Deep Networks · Neuroscience · Cognitive Science · Visual Perception · CNN Architectures
[ Paper] 

Author Information

Drew Linsley (Brown University)

We need artificial vision to create intelligent machines that can reason about the world, but existing artificial vision systems cannot solve many of the visual challenges that we encounter and routinely solve in our daily lives. I look to biological vision to inspire new solutions to challenges faced by artificial vision. I do this by testing complementary hypotheses that connect computational theory with systems- and cognitive-neuroscience level experimental research: - Computational challenges for artificial vision can be identified through systematic comparisons with biological vision, and solved with algorithms inspired by those of biological vision. - Improved algorithms for artificial vision will lead to better methods for gleaning insight from large-scale experimental data, and better models for understanding the relationship between neural computation and perception.

Junkyung Kim (Brown University)
Vijay Veerabadran (University of California, San Diego)

Ph.D. student at UC San Diego - Working on neurally plausible computer vision models and deep learning for medical applications.

Charles Windolf (Brown University)
Thomas Serre (Brown University)

More from the Same Authors