Timezone: »

Bio-inspired learnable divisive normalization for ANNs
Vijay Veerabadran · Ritik Raina · Virginia de Sa

Mon Dec 13 12:00 PM -- 01:00 PM (PST) @
Event URL: https://openreview.net/forum?id=-ZOjASLOsrV »

In this work we introduce DivNormEI, a novel bio-inspired convolutional network that performs divisive normalization, a canonical cortical computation, along with lateral inhibition and excitation that is tailored for integration into modern Artificial Neural Networks (ANNs). DivNormEI, an extension of prior computational models of divisive normalization in the primate primary visual cortex, is implemented as a modular layer that can be integrated in a straightforward manner into most commonly used modern ANNs. DivNormEI normalizes incoming activations via learned non-linear within-feature shunting inhibition along with across-feature linear lateral inhibition and excitation. In this work, we show how the integration of DivNormEI within a task-driven self-supervised encoder-decoder architecture encourages the emergence of the well-known contrast-invariant tuning property found to be exhibited by simple cells in the primate primary visual cortex. Additionally, the integration of DivNormEI into an ANN (VGG-9 network) trained to perform image classification on ImageNet-100 improves both sample efficiency and top-1 accuracy on a held-out validation set. We believe our findings from the bio-inspired DivNormEI model that simultaneously explains properties found in primate V1 neurons and outperforms the competing baseline architecture on large-scale object recognition will promote further investigation of this crucial cortical computation in the context of modern machine learning tasks and ANNs.

Author Information

Vijay Veerabadran (UC San Diego)

Ph.D. candidate in Cognitive Science with an interest in biological and artificial intelligence (particularly interested in vision). I work on developing recurrent neural networks for efficient learning of long-range spatial dependencies and on quantifying behavioral similarity between human and machine vision.

Ritik Raina (University of California, San Diego)
Ritik Raina

Hi! My name is Ritik Raina and I'm a pre-doctoral researcher at UC San Diego where I am advised by Dr. Virginia de Sa. My research interest are in developing cognitively-inspired neural architectures that can be applied to learn robust visual representations.

Virginia de Sa (University of California, San Diego)

More from the Same Authors

  • 2021 : Spectrally Adaptive Common Spatial Patterns »
    Mahta Mousavi · Eric Lybrand · Shuangquan Feng · Shuai Tang · Rayan Saab · Virginia de Sa
  • 2022 : Exploring Biases in Facial Expression Analysis »
    Ritik Raina · Miguel Monares · Mingze Xu · Sarah Fabi · Xiaojing Xu · Lehan Li · Will Sumerfield · Jin Gan · Virginia de Sa
  • 2019 : Poster Session »
    Ethan Harris · Tom White · Oh Hyeon Choung · Takashi Shinozaki · Dipan Pal · Katherine L. Hermann · Judy Borowski · Camilo Fosco · Chaz Firestone · Vijay Veerabadran · Benjamin Lahner · Chaitanya Ryali · Fenil Doshi · Pulkit Singh · Sharon Zhou · Michel Besserve · Michael Chang · Anelise Newman · Mahesan Niranjan · Jonathon Hare · Daniela Mihai · Marios Savvides · Simon Kornblith · Christina M Funke · Aude Oliva · Virginia de Sa · Dmitry Krotov · Colin Conwell · George Alvarez · Alex Kolchinski · Shengjia Zhao · Mitchell Gordon · Michael Bernstein · Stefano Ermon · Arash Mehrjou · Bernhard Schölkopf · John Co-Reyes · Michael Janner · Jiajun Wu · Josh Tenenbaum · Sergey Levine · Yalda Mohsenzadeh · Zhenglong Zhou