Timezone: »

 
Poster
Credit Assignment in Neural Networks through Deep Feedback Control
Alexander Meulemans · Matilde Tristany Farinha · Javier Garcia Ordonez · Pau Vilimelis Aceituno · João Sacramento · Benjamin F. Grewe

Wed Dec 08 12:30 AM -- 02:00 AM (PST) @ Virtual
in Poster Session 3 »

The success of deep learning sparked interest in whether the brain learns by using similar techniques for assigning credit to each synaptic weight for its contribution to the network output. However, the majority of current attempts at biologically-plausible learning methods are either non-local in time, require highly specific connectivity motifs, or have no clear link to any known mathematical optimization method. Here, we introduce Deep Feedback Control (DFC), a new learning method that uses a feedback controller to drive a deep neural network to match a desired output target and whose control signal can be used for credit assignment. The resulting learning rule is fully local in space and time and approximates Gauss-Newton optimization for a wide range of feedback connectivity patterns. To further underline its biological plausibility, we relate DFC to a multi-compartment model of cortical pyramidal neurons with a local voltage-dependent synaptic plasticity rule, consistent with recent theories of dendritic processing. By combining dynamical system theory with mathematical optimization theory, we provide a strong theoretical foundation for DFC that we corroborate with detailed results on toy experiments and standard computer-vision benchmarks.

Keywords: Theory · Deep Learning · Optimization · Neuroscience · Vision

Author Information

Alexander Meulemans (ETH Zürich | University of Zürich | Institute of Neuroinformatics)
Matilde Tristany Farinha (Swiss Federal Institute of Technology)
Javier Garcia Ordonez (Swiss Federal Institute of Technology)
Pau Vilimelis Aceituno (Insititute of Neuroinformatics, University of Zurich and ETH Zurich, Swiss Federal Institute of Technology)
João Sacramento (ETH Zurich)
Benjamin F. Grewe (ETH Zurich)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors