Visual object recognition (OR) is a central problem in systems neuroscience, human psychophysics, and computer vision. A recognition system must be robust to image variation produced by different “views” of each object-- the so-called “invariance problem.” My laboratory aims to understand and emulate the primate brain's solution to this problem.
We have previously shown that a part of the non-human primate ventral visual stream (inferior temporal cortex, IT) rapidly and automatically conveys neuronal population rate codes that qualitatively solve the invariance problem for vision. But are such codes quantitatively sufficient to explain behavioral OR performance? Our results show that these codes are a powerful object representation, in that low complexity decoding tools can be applied to them to perfectly predict human performance over a large range of OR tasks.
But how does the brain build this powerful representation? High-throughput computational methods can be used to explore a large family of biologically-constrained neural network architectures. Using this approach, we have recently discovered that functional optimization of this large family leads to specific algorithms that predict the response properties of IT dramatically better than all previous models. This suggests that these networks have captured key encoding mechanisms of human OR, and that today’s computer vision algorithms are very close to emulating the power of the primate OR system.
James J DiCarlo (Massachusetts Institute of Technology)
Prof. DiCarlo received his Ph.D. in biomedical engineering and his M.D. from Johns Hopkins in 1998, and did his postdoctoral training in primate visual neurophysiology at Baylor College of Medicine. He joined the MIT faculty in 2002. He is a Sloan Fellow, a Pew Scholar, and a McKnight Scholar. His lab’s research goal is a computational understanding of the brain mechanisms that underlie object recognition. They use large-scale neurophysiology, brain imaging, optogenetic methods, and high-throughput computational simulations to understand how the primate ventral visual stream is able to untangle object identity from other latent image variables such as object position, scale, and pose. They have shown that populations of neurons at the highest cortical visual processing stage (IT) rapidly convey explicit representations of object identity, and that this ability is reshaped by natural visual experience. They have also shown how visual recognition tests can be used to discover new, high-performing bio-inspired algorithms. This understanding may inspire new machine vision systems, new neural prosthetics, and a foundation for understanding how high-level visual representation is altered in conditions such as agnosia, autism and dyslexia.
More from the Same Authors
2020 Poster: Simulating a Primary Visual Cortex at the Front of CNNs Improves Robustness to Image Perturbations »
Joel Dapello · Tiago Marques · Martin Schrimpf · Franziska Geiger · David Cox · James J DiCarlo
2020 Spotlight: Simulating a Primary Visual Cortex at the Front of CNNs Improves Robustness to Image Perturbations »
Martin Schrimpf · James J DiCarlo · David Cox · Franziska Geiger · Tiago Marques · Joel Dapello
2019 Poster: Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs »
Jonas Kubilius · Martin Schrimpf · Kohitij Kar · Rishi Rajalingham · Ha Hong · Najib Majaj · Elias Issa · Pouya Bashivan · Jonathan Prescott-Roy · Kailyn Schmidt · Aran Nayebi · Daniel Bear · Daniel Yamins · James J DiCarlo
2019 Oral: Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs »
Jonas Kubilius · Martin Schrimpf · Ha Hong · Najib Majaj · Rishi Rajalingham · Elias Issa · Kohitij Kar · Pouya Bashivan · Jonathan Prescott-Roy · Kailyn Schmidt · Aran Nayebi · Daniel Bear · Daniel Yamins · James J DiCarlo
2018 Poster: Task-Driven Convolutional Recurrent Models of the Visual System »
Aran Nayebi · Daniel Bear · Jonas Kubilius · Kohitij Kar · Surya Ganguli · David Sussillo · James J DiCarlo · Daniel Yamins
2013 Poster: Hierarchical Modular Optimization of Convolutional Networks Achieves Representations Similar to Macaque IT and Human Ventral Stream »
Daniel L Yamins · Ha Hong · Charles Cadieu · James J DiCarlo