Timezone: »
Workshop
Beyond BackPropagation: Novel Ideas for Training Neural Architectures
Mateusz Malinowski · Grzegorz Swirszcz · Viorica Patraucean · Marco Gori · Yanping Huang · Sindy Löwe · Anna Choromanska
Event URL: https://beyondbackprop.github.io/ »
Is backpropagation the ultimate tool on the path to achieving synthetic intelligence as its success and widespread adoption would suggest?
Many have questioned the biological plausibility of backpropagation as a learning mechanism since its discovery. The weight transport and timing problems are the most disputable. The same properties of backpropagation training also have practical consequences. For instance, backpropagation training is a global and coupled procedure that limits the amount of possible parallelism and yields high latency.
These limitations have motivated us to discuss possible alternative directions. In this workshop, we want to promote such discussions by bringing together researchers from various but related disciplines, and to discuss possible solutions from engineering, machine learning and neuroscientific perspectives.
|
Sat 6:00 a.m. - 6:15 a.m.
|
Live Intro
(
Talk
)
|
Mateusz Malinowski · Viorica Patraucean · Grzegorz Swirszcz · Sindy Löwe · Anna Choromanska · Marco Gori · Yanping Huang 🔗 |
|
Sat 6:15 a.m. - 6:17 a.m.
|
Introduction: Bastiaan Veeling
(
Introduction
)
|
Mateusz Malinowski 🔗 |
|
Sat 6:17 a.m. - 6:45 a.m.
|
Invited Talk Bastiaan Veeling
(
Talk
)
SlidesLive Video » |
Bas Veeling 🔗 |
|
Sat 6:45 a.m. - 6:47 a.m.
|
Introduction: Olivier Teytaud
(
Introduction
)
|
Sindy Löwe 🔗 |
|
Sat 6:47 a.m. - 7:15 a.m.
|
Invited Talk Olivier Teytaud
(
Talk
)
SlidesLive Video » |
Olivier Teytaud 🔗 |
|
Sat 7:15 a.m. - 8:30 a.m.
|
Poster Session: Morning ( Parallel poster session ) link » | 🔗 |
|
Sat 8:30 a.m. - 8:32 a.m.
|
Introduction: Karl Friston
(
Introduction
)
|
Mateusz Malinowski 🔗 |
|
Sat 8:32 a.m. - 9:00 a.m.
|
Invited Talk Karl Friston
(
Talk
)
SlidesLive Video » |
Karl Friston 🔗 |
|
Sat 9:00 a.m. - 9:45 a.m.
|
Panel discussion 1
(
Discussion
)
|
Bas Veeling · Olivier Teytaud · Karl Friston · Sindy Löwe · Mateusz Malinowski 🔗 |
|
Sat 9:45 a.m. - 11:00 a.m.
|
Long Break
|
🔗 |
|
Sat 11:00 a.m. - 11:02 a.m.
|
Introduction: Yoshua Bengio
(
Introduction
)
|
Anna Choromanska 🔗 |
|
Sat 11:02 a.m. - 11:32 a.m.
|
Invited Talk Yoshua Bengio
(
Talk
)
SlidesLive Video » |
Yoshua Bengio 🔗 |
|
Sat 11:32 a.m. - 11:34 a.m.
|
Introduction: Danielle Bassett
(
Introduction
)
|
Anna Choromanska 🔗 |
|
Sat 11:34 a.m. - 12:08 p.m.
|
Invited Talk Danielle Bassett
(
Talk
)
|
Danielle S Bassett 🔗 |
|
Sat 12:09 p.m. - 12:10 p.m.
|
Introduction: Oral 1.1 and Oral 1.2
(
Introduction
)
|
Viorica Patraucean 🔗 |
|
Sat 12:10 p.m. - 12:22 p.m.
|
Orals 1.1: Randomized Automatic Differentiation
(
Contributed Talks
)
SlidesLive Video » |
Deniz Oktay · Nick McGreivy · Alex Beatson · Ryan Adams 🔗 |
|
Sat 12:22 p.m. - 12:35 p.m.
|
Orals 1.2: ZORB: A Derivative-Free Backpropagation Algorithm for Neural Networks
(
Contributed Talks
)
SlidesLive Video » |
Varun Ranganathan · Alex Lewandowski 🔗 |
|
Sat 12:35 p.m. - 12:40 p.m.
|
Oral 1.1 and 1.2 Q&A
(
Q&A
)
|
🔗 |
|
Sat 12:40 p.m. - 12:45 p.m.
|
Short Break 2
|
🔗 |
|
Sat 12:45 p.m. - 12:46 p.m.
|
Introduction: Oral 2.1 and Oral 2.2
(
Introduction
)
|
Viorica Patraucean 🔗 |
|
Sat 12:46 p.m. - 12:59 p.m.
|
Orals 2.1: Policy Manifold Search for Improving Diversity-based Neuroevolution
(
Contributed Talks
)
SlidesLive Video » |
Nemanja Rakicevic 🔗 |
|
Sat 12:59 p.m. - 1:12 p.m.
|
Orals 2.2: Hardware Beyond Backpropagation: a Photonic Co-Processor for Direct Feedback Alignmen
(
Contributed Talks
)
SlidesLive Video » |
Julien Launay · Iacopo Poli · Laurent Daudet · Florent Krzakala 🔗 |
|
Sat 1:12 p.m. - 1:15 p.m.
|
Oral 2.1 and Oral 2.2 Q&A
(
Q&A
)
|
🔗 |
|
Sat 1:15 p.m. - 1:17 p.m.
|
Introduction: David Duvenaud
(
Introduction
)
|
Yanping Huang 🔗 |
|
Sat 1:17 p.m. - 1:45 p.m.
|
Invited Talk David Duvenaud
(
Talk
)
SlidesLive Video » |
David Duvenaud 🔗 |
|
Sat 1:45 p.m. - 1:47 p.m.
|
Introduction: Cristina Savin
(
Introduction
)
|
Yanping Huang 🔗 |
|
Sat 1:47 p.m. - 2:15 p.m.
|
Invited Talk Cristina Savin
(
Talk
)
SlidesLive Video » |
Cristina Savin 🔗 |
|
Sat 2:15 p.m. - 3:00 p.m.
|
Panel discussion 2
(
Discussion
)
|
Danielle S Bassett · Yoshua Bengio · Cristina Savin · David Duvenaud · Anna Choromanska · Yanping Huang 🔗 |
|
Sat 3:00 p.m. - 4:30 p.m.
|
Poster Session: Evening ( Posters ) link » | 🔗 |
Author Information
Mateusz Malinowski (DeepMind)
Mateusz Malinowski is a research scientist at DeepMind, where he works at the intersection of computer vision, natural language understanding, and deep learning. He was granted PhD (Dr.-Ing.) with the highest honor (summa cum laude) at Max Planck Institute for Informatics in 2017 in computer vision for his pioneering work on visual question answering, where he proposed the task and developed methods that answer questions about the content of images. Prior to this, he graduated with honors from Saarland University in computer science. Before that, he studied computer science at Wroclaw University in Poland.
Grzegorz Swirszcz (DeepMind)
Viorica Patraucean (DeepMind)
Marco Gori (University of Siena)
Yanping Huang (Google Brain)
Sindy Löwe (University of Amsterdam)
Anna Choromanska (NYU Tandon School of Engineering)
More from the Same Authors
-
2021 : LiRo: Benchmark and leaderboard for Romanian language tasks »
Stefan Dumitrescu · Petru Rebeja · Beata Lorincz · Mihaela Gaman · Andrei Avram · Mihai Ilie · Andrei Pruteanu · Adriana Stan · Lorena Rosia · Cristina Iacobescu · Luciana Morogan · George Dima · Gabriel Marchidan · Traian Rebedea · Madalina Chitez · Dani Yogatama · Sebastian Ruder · Radu Tudor Ionescu · Razvan Pascanu · Viorica Patraucean -
2021 : Amortized Causal Discovery: Learning to Infer Causal Graphs from Time-Series Data »
Sindy Löwe · David Madras · Richard Zemel · Max Welling -
2022 : Active Acquisition for Multimodal Temporal Data: A Challenging Decision-Making Task »
Jannik Kossen · Cătălina Cangea · Eszter Vértes · Andrew Jaegle · Viorica Patraucean · Ira Ktena · Nenad Tomasev · Danielle Belgrave -
2022 : Knowledge-driven Active Learning »
Gabriele Ciravegna · Frederic Precioso · Marco Gori -
2022 Poster: Mixture-of-Experts with Expert Choice Routing »
Yanqi Zhou · Tao Lei · Hanxiao Liu · Nan Du · Yanping Huang · Vincent Zhao · Andrew Dai · zhifeng Chen · Quoc V Le · James Laudon -
2022 Poster: Neural Payoff Machines: Predicting Fair and Stable Payoff Allocations Among Team Members »
Daphne Cornelisse · Thomas Rood · Yoram Bachrach · Mateusz Malinowski · Tal Kachman -
2020 : Panel discussion 2 »
Danielle S Bassett · Yoshua Bengio · Cristina Savin · David Duvenaud · Anna Choromanska · Yanping Huang -
2020 : Introduction: Cristina Savin »
Yanping Huang -
2020 : Introduction: David Duvenaud »
Yanping Huang -
2020 : Introduction: Oral 2.1 and Oral 2.2 »
Viorica Patraucean -
2020 : Introduction: Oral 1.1 and Oral 1.2 »
Viorica Patraucean -
2020 : Keynote 2: Marco Gori »
Marco Gori -
2020 : Panel discussion 1 »
Bas Veeling · Olivier Teytaud · Karl Friston · Sindy Löwe · Mateusz Malinowski -
2020 : Introduction: Karl Friston »
Mateusz Malinowski -
2020 : Introduction: Olivier Teytaud »
Sindy Löwe -
2020 : Introduction: Bastiaan Veeling »
Mateusz Malinowski -
2020 : Live Intro »
Mateusz Malinowski · Viorica Patraucean · Grzegorz Swirszcz · Sindy Löwe · Anna Choromanska · Marco Gori · Yanping Huang -
2020 Poster: Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout »
Zhao Chen · Jiquan Ngiam · Yanping Huang · Thang Luong · Henrik Kretzschmar · Yuning Chai · Dragomir Anguelov -
2019 Poster: Putting An End to End-to-End: Gradient-Isolated Learning of Representations »
Sindy Löwe · Peter O'Connor · Bas Veeling -
2019 Oral: Putting An End to End-to-End: Gradient-Isolated Learning of Representations »
Sindy Löwe · Peter O'Connor · Bas Veeling -
2019 Poster: GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism »
Yanping Huang · Youlong Cheng · Ankur Bapna · Orhan Firat · Dehao Chen · Mia Chen · HyoukJoong Lee · Jiquan Ngiam · Quoc V Le · Yonghui Wu · zhifeng Chen -
2019 Poster: Learning dynamic polynomial proofs »
Alhussein Fawzi · Mateusz Malinowski · Hamza Fawzi · Omar Fawzi -
2019 Spotlight: Learning dynamic polynomial proofs »
Alhussein Fawzi · Mateusz Malinowski · Hamza Fawzi · Omar Fawzi -
2018 Workshop: Visually grounded interaction and language »
Florian Strub · Harm de Vries · Erik Wijmans · Samyak Datta · Ethan Perez · Mateusz Malinowski · Stefan Lee · Peter Anderson · Aaron Courville · Jeremie MARY · Dhruv Batra · Devi Parikh · Olivier Pietquin · Chiori HORI · Tim Marks · Anoop Cherian -
2018 Poster: Learning to Navigate in Cities Without a Map »
Piotr Mirowski · Matt Grimes · Mateusz Malinowski · Karl Moritz Hermann · Keith Anderson · Denis Teplyashin · Karen Simonyan · koray kavukcuoglu · Andrew Zisserman · Raia Hadsell -
2017 Workshop: Visually grounded interaction and language »
Florian Strub · Harm de Vries · Abhishek Das · Satwik Kottur · Stefan Lee · Mateusz Malinowski · Olivier Pietquin · Devi Parikh · Dhruv Batra · Aaron Courville · Jeremie Mary -
2017 Poster: A simple neural network module for relational reasoning »
Adam Santoro · David Raposo · David Barrett · Mateusz Malinowski · Razvan Pascanu · Peter Battaglia · Timothy Lillicrap -
2017 Spotlight: A simple neural network module for relational reasoning »
Adam Santoro · David Raposo · David Barrett · Mateusz Malinowski · Razvan Pascanu · Peter Battaglia · Timothy Lillicrap -
2017 Poster: Sobolev Training for Neural Networks »
Wojciech Czarnecki · Simon Osindero · Max Jaderberg · Grzegorz Swirszcz · Razvan Pascanu -
2014 Poster: A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Input »
Mateusz Malinowski · Mario Fritz