Timezone: »
Backpropagation (BP) has been the most successful algorithm used to train artificial neural networks. However, there are several gaps between BP and learning in biologically plausible neuronal networks of the brain (learning in the brain, or simply BL, for short), in particular, (1) it has been unclear to date, if BP can be implemented exactly via BL, (2) there is a lack of local plasticity in BP, i.e., weight updates require information that is not locally available, while BL utilizes only locally available information, and (3)~there is a lack of autonomy in BP, i.e., some external control over the neural network is required (e.g., switching between prediction and learning stages requires changes to dynamics and synaptic plasticity rules), while BL works fully autonomously. Bridging such gaps, i.e., understanding how BP can be approximated by BL, has been of major interest in both neuroscience and machine learning. Despite tremendous efforts, however, no previous model has bridged the gaps at a degree of demonstrating an equivalence to BP, instead, only approximations to BP have been shown. Here, we present for the first time a framework within BL that bridges the above crucial gaps. We propose a BL model that (1) produces \emph{exactly the same} updates of the neural weights as~BP, while (2)~employing local plasticity, i.e., all neurons perform only local computations, done simultaneously. We then modify it to an alternative BL model that (3) also works fully autonomously. Overall, our work provides important evidence for the debate on the long-disputed question whether the brain can perform~BP.
Author Information
Yuhang Song (University of Oxford)
Thomas Lukasiewicz (University of Oxford)
Zhenghua Xu (Hebei University of Technology)
Rafal Bogacz (University of Oxford)
More from the Same Authors
-
2021 : Few-Shot Out-of-Domain Transfer of Natural Language Explanations »
Yordan Yordanov · Vid Kocijan · Thomas Lukasiewicz · Oana-Maria Camburu -
2021 : Few-Shot Out-of-Domain Transfer of Natural Language Explanations »
Yordan Yordanov · Vid Kocijan · Thomas Lukasiewicz · Oana-Maria Camburu -
2022 : Associative memory via covariance-learning predictive coding networks »
Mufeng Tang · Tommaso Salvatori · Yuhang Song · Beren Millidge · Thomas Lukasiewicz · Rafal Bogacz -
2023 Poster: Sequential Memory with Temporal Predictive Coding »
Mufeng Tang · Helen Barron · Rafal Bogacz -
2022 Spotlight: Predictive Coding beyond Gaussian Distributions »
Luca Pinchetti · Tommaso Salvatori · Yordan Yordanov · Beren Millidge · Yuhang Song · Thomas Lukasiewicz -
2022 Spotlight: Lightning Talks 1B-1 »
Qitian Wu · Runlin Lei · Rongqin Chen · Luca Pinchetti · Yangze Zhou · Abhinav Kumar · Hans Hao-Hsun Hsu · Wentao Zhao · Chenhao Tan · Zhen Wang · Shenghui Zhang · Yuesong Shen · Tommaso Salvatori · Gitta Kutyniok · Zenan Li · Amit Sharma · Leong Hou U · Yordan Yordanov · Christian Tomani · Bruno Ribeiro · Yaliang Li · David P Wipf · Daniel Cremers · Bolin Ding · Beren Millidge · Ye Li · Yuhang Song · Junchi Yan · Zhewei Wei · Thomas Lukasiewicz -
2022 Poster: Learning on Arbitrary Graph Topologies via Predictive Coding »
Tommaso Salvatori · Luca Pinchetti · Beren Millidge · Yuhang Song · Tianyi Bao · Rafal Bogacz · Thomas Lukasiewicz -
2022 Poster: Predictive Coding beyond Gaussian Distributions »
Luca Pinchetti · Tommaso Salvatori · Yordan Yordanov · Beren Millidge · Yuhang Song · Thomas Lukasiewicz -
2021 Poster: Associative Memories via Predictive Coding »
Tommaso Salvatori · Yuhang Song · Yujian Hong · Lei Sha · Simon Frieder · Zhenghua Xu · Rafal Bogacz · Thomas Lukasiewicz -
2020 Poster: Lightweight Generative Adversarial Networks for Text-Guided Image Manipulation »
Bowen Li · Xiaojuan Qi · Philip Torr · Thomas Lukasiewicz -
2020 Poster: Coherent Hierarchical Multi-Label Classification Networks »
Eleonora Giunchiglia · Thomas Lukasiewicz -
2020 Poster: BoxE: A Box Embedding Model for Knowledge Base Completion »
Ralph Abboud · Ismail Ceylan · Thomas Lukasiewicz · Tommaso Salvatori -
2020 Spotlight: BoxE: A Box Embedding Model for Knowledge Base Completion »
Ralph Abboud · Ismail Ceylan · Thomas Lukasiewicz · Tommaso Salvatori -
2019 Poster: Controllable Text-to-Image Generation »
Bowen Li · Xiaojuan Qi · Thomas Lukasiewicz · Philip Torr -
2018 Poster: e-SNLI: Natural Language Inference with Natural Language Explanations »
Oana-Maria Camburu · Tim Rocktäschel · Thomas Lukasiewicz · Phil Blunsom