Timezone: »

Beyond BackPropagation: Novel Ideas for Training Neural Architectures
Mateusz Malinowski · Grzegorz Swirszcz · Viorica Patraucean · Marco Gori · Yanping Huang · Sindy Löwe · Anna Choromanska

Sat Dec 12 06:00 AM -- 04:30 PM (PST) @ None
Event URL: https://beyondbackprop.github.io/ »

Is backpropagation the ultimate tool on the path to achieving synthetic intelligence as its success and widespread adoption would suggest?

Many have questioned the biological plausibility of backpropagation as a learning mechanism since its discovery. The weight transport and timing problems are the most disputable. The same properties of backpropagation training also have practical consequences. For instance, backpropagation training is a global and coupled procedure that limits the amount of possible parallelism and yields high latency.

These limitations have motivated us to discuss possible alternative directions. In this workshop, we want to promote such discussions by bringing together researchers from various but related disciplines, and to discuss possible solutions from engineering, machine learning and neuroscientific perspectives.

Sat 6:00 a.m. - 6:15 a.m.
Live Intro (Talk)
Mateusz Malinowski, Viorica Patraucean, Grzegorz Swirszcz, Sindy Löwe, Anna Choromanska, Marco Gori, Yanping Huang
Sat 6:15 a.m. - 6:17 a.m.
Introduction: Bastiaan Veeling (Introduction)
Mateusz Malinowski
Sat 6:17 a.m. - 6:45 a.m.
Invited Talk Bastiaan Veeling (Talk)   
Bas Veeling
Sat 6:45 a.m. - 6:47 a.m.
Introduction: Olivier Teytaud (Introduction)
Sindy Löwe
Sat 6:47 a.m. - 7:15 a.m.
Invited Talk Olivier Teytaud (Talk)   
Olivier Teytaud
Sat 7:15 a.m. - 8:30 a.m.
Poster Session: Morning (Parallel poster session)  link »
Sat 8:30 a.m. - 8:32 a.m.
Introduction: Karl Friston (Introduction)
Mateusz Malinowski
Sat 8:32 a.m. - 9:00 a.m.
Invited Talk Karl Friston (Talk)   
Karl Friston
Sat 9:00 a.m. - 9:45 a.m.
Panel discussion 1 (Discussion)
Bas Veeling, Olivier Teytaud, Karl Friston, Sindy Löwe, Mateusz Malinowski
Sat 9:45 a.m. - 11:00 a.m.
Long Break (Break)
Sat 11:00 a.m. - 11:02 a.m.
Introduction: Yoshua Bengio (Introduction)
Anna Choromanska
Sat 11:02 a.m. - 11:32 a.m.
Invited Talk Yoshua Bengio (Talk)   
Yoshua Bengio
Sat 11:32 a.m. - 11:34 a.m.
Introduction: Danielle Bassett (Introduction)
Anna Choromanska
Sat 11:34 a.m. - 12:08 p.m.
Invited Talk Danielle Bassett (Talk)
Dani S Bassett
Sat 12:09 p.m. - 12:10 p.m.
Introduction: Oral 1.1 and Oral 1.2 (Introduction)
Viorica Patraucean
Sat 12:10 p.m. - 12:22 p.m.
Orals 1.1: Randomized Automatic Differentiation (Contributed Talks)   
Deniz Oktay, Nick McGreivy, Alex Beatson, Ryan Adams
Sat 12:22 p.m. - 12:35 p.m.
Orals 1.2: ZORB: A Derivative-Free Backpropagation Algorithm for Neural Networks (Contributed Talks)   
Varun Ranganathan, Alex Lewandowski
Sat 12:35 p.m. - 12:40 p.m.
Oral 1.1 and 1.2 Q&A (Q&A)
Sat 12:40 p.m. - 12:45 p.m.
Short Break 2 (Break)
Sat 12:45 p.m. - 12:46 p.m.
Introduction: Oral 2.1 and Oral 2.2 (Introduction)
Viorica Patraucean
Sat 12:46 p.m. - 12:59 p.m.
Orals 2.1: Policy Manifold Search for Improving Diversity-based Neuroevolution (Contributed Talks)   
Nemanja Rakicevic
Sat 12:59 p.m. - 1:12 p.m.
Orals 2.2: Hardware Beyond Backpropagation: a Photonic Co-Processor for Direct Feedback Alignmen (Contributed Talks)   
Julien Launay, Iacopo Poli, Laurent Daudet, Florent Krzakala
Sat 1:12 p.m. - 1:15 p.m.
Oral 2.1 and Oral 2.2 Q&A (Q&A)
Sat 1:15 p.m. - 1:17 p.m.
Introduction: David Duvenaud (Introduction)
Yanping Huang
Sat 1:17 p.m. - 1:45 p.m.
Invited Talk David Duvenaud (Talk)   
David Duvenaud
Sat 1:45 p.m. - 1:47 p.m.
Introduction: Cristina Savin (Introduction)
Yanping Huang
Sat 1:47 p.m. - 2:15 p.m.
Invited Talk Cristina Savin (Talk)   
Cristina Savin
Sat 2:15 p.m. - 3:00 p.m.
Panel discussion 2 (Discussion)
Dani S Bassett, Yoshua Bengio, Cristina Savin, David Duvenaud, Anna Choromanska, Yanping Huang
Sat 3:00 p.m. - 4:30 p.m.
Poster Session: Evening (Posters)  link »

Author Information

Mateusz Malinowski (DeepMind)

Mateusz Malinowski is a research scientist at DeepMind, where he works at the intersection of computer vision, natural language understanding, and deep learning. He was granted PhD (Dr.-Ing.) with the highest honor (summa cum laude) at Max Planck Institute for Informatics in 2017 in computer vision for his pioneering work on visual question answering, where he proposed the task and developed methods that answer questions about the content of images. Prior to this, he graduated with honors from Saarland University in computer science. Before that, he studied computer science at Wroclaw University in Poland.

Grzegorz Swirszcz (DeepMind)
Viorica Patraucean (DeepMind)
Marco Gori (University of Siena)
Yanping Huang (Google Brain)
Sindy Löwe (University of Amsterdam)
Anna Choromanska (NYU Tandon School of Engineering)

More from the Same Authors