Timezone: »
Deep learning can still be a complex mix of art and engineering despite its tremendous success in recent years, and there is still progress to be made before it has fully evolved into a mature scientific discipline. The interdependence of architecture, data, and optimization gives rise to an enormous landscape of design and performance intricacies that are not well-understood. The evolution from engineering towards science in deep learning can be achieved by pushing the disciplinary boundaries. Unlike in the natural and physical sciences -- where experimental capabilities can hamper progress, i.e. limitations in what quantities can be probed and measured in physical systems, how much and how often -- in deep learning the vast majority of relevant quantities that we wish to measure can be tracked in some way. As such, a greater limiting factor towards scientific understanding and principled design in deep learning is how to insightfully harness the tremendous collective experimental capability of the field. As a community, some primary aims would be to (i) identify obstacles to better models and algorithms, (ii) identify the general trends that are potentially important which we wish to understand scientifically and potentially theoretically and; (iii) careful design of scientific experiments whose purpose is to clearly resolve and pinpoint the origin of mysteries (so-called 'smoking-gun' experiments).
Sat 8:00 a.m. - 8:15 a.m.
|
Welcoming remarks and introduction
(
Intro
)
|
Levent Sagun · Caglar Gulcehre · Adriana Romero Soriano · Negar Rostamzadeh · Nando de Freitas 🔗 |
Sat 8:15 a.m. - 8:35 a.m.
|
Surya Ganguli - An analytic theory of generalization dynamics and transfer learning in deep linear networks
(
Talk
)
|
Surya Ganguli 🔗 |
Sat 8:35 a.m. - 8:55 a.m.
|
Yasaman Bahri - Tractable limits for deep networks: an overview of the large width regime
(
Talk
)
|
Yasaman Bahri 🔗 |
Sat 8:55 a.m. - 9:15 a.m.
|
Florent Krzakala - Learning with "realistic" synthetic data
(
Talk
)
|
Florent Krzakala 🔗 |
Sat 9:15 a.m. - 9:45 a.m.
|
Surya Ganguli, Yasaman Bahri, Florent Krzakala moderated by Lenka Zdeborova
(
Mini-panel
)
The mini panel with Florent Krzakala, Yasaman Bahri, Surya Ganguli will be moderated by Lenka Zdeborova. Session advisors are Joan Bruna and Adji Bousso Dieng. |
Florent Krzakala · Yasaman Bahri · Surya Ganguli · Lenka Zdeborová · Adji Bousso Dieng · Joan Bruna 🔗 |
Sat 9:45 a.m. - 10:30 a.m.
|
Coffee and posters
|
🔗 |
Sat 10:30 a.m. - 10:50 a.m.
|
Carl Doersch - On Self-Supervised Learning for Vision
(
Mini-panel
)
|
Carl Doersch 🔗 |
Sat 10:50 a.m. - 11:10 a.m.
|
Raquel Urtasun - Science and Engineering for Self-driving
(
Talk
)
|
Raquel Urtasun 🔗 |
Sat 11:10 a.m. - 11:30 a.m.
|
Sanja Fidler - TBA
(
Talk
)
|
Sanja Fidler 🔗 |
Sat 11:30 a.m. - 12:00 p.m.
|
Carl Doersch, Raquel Urtasun, Sanja Fidler moderated by Natalia Neverova
(
Mini-panel
)
The mini-panel with Carl Doersch, Raquel Urtasun, Sanja Fidler will be moderated by Natalia Neverova. Session advisors are Alp Guler and Ilija Radosavovic. |
Raquel Urtasun · Sanja Fidler · Natalia Neverova · Ilija Radosavovic · Carl Doersch 🔗 |
Sat 12:00 p.m. - 2:00 p.m.
|
Lunch Break and Posters
(
Break and Posters
)
Since we are a small workshop, we will hold the poster sessions during the day, including all the breaks as the authors wish. |
Xingyou Song · Elad Hoffer · Wei-Cheng Chang · Jeremy Cohen · Jyoti Islam · Yaniv Blumenfeld · Andreas Madsen · Jonathan Frankle · Sebastian Goldt · Satrajit Chatterjee · Abhishek Panigrahi · Alex Renda · Brian Bartoldson · Israel Birhane · Aristide Baratin · Niladri Chatterji · Roman Novak · Jessica Forde · YiDing Jiang · Yilun Du · Linara Adilova · Michael Kamp · Berry Weinstein · Itay Hubara · Tal Ben-Nun · Torsten Hoefler · Daniel Soudry · Hsiang-Fu Yu · Kai Zhong · Yiming Yang · Inderjit Dhillon · Jaime Carbonell · Yanqing Zhang · Dar Gilboa · Johannes Brandstetter · Alexander R Johansen · Gintare Karolina Dziugaite · Raghav Somani · Ari Morcos · Freddie Kalaitzis · Hanie Sedghi · Lechao Xiao · John Zech · Muqiao Yang · Simran Kaur · Qianli Ma · Yao-Hung Hubert Tsai · Ruslan Salakhutdinov · Sho Yaida · Zachary Lipton · Daniel Roy · Michael Carbin · Florent Krzakala · Lenka Zdeborová · Guy Gur-Ari · Ethan Dyer · Dilip Krishnan · Hossein Mobahi · Samy Bengio · Behnam Neyshabur · Praneeth Netrapalli · Kris Sankaran · Julien Cornebise · Yoshua Bengio · Vincent Michalski · Samira Ebrahimi Kahou · Md Rifat Arefin · Jiri Hron · Jaehoon Lee · Jascha Sohl-Dickstein · Samuel Schoenholz · David Schwab · Dongyu Li · Sang Keun Choe · Henning Petzka · Ashish Verma · Zhichao Lin · Cristian Sminchisescu
|
Sat 2:00 p.m. - 2:20 p.m.
|
Douwe Kiela - Benchmarking Progress in AI: A New Benchmark for Natural Language Understanding
(
Talk
)
|
Douwe Kiela 🔗 |
Sat 2:20 p.m. - 2:40 p.m.
|
Audrey Durand - Trading off theory and practice: A bandit perspective
(
Talk
)
|
Audrey Durand 🔗 |
Sat 2:40 p.m. - 3:00 p.m.
|
Kamalika Chaudhuri - A Three Sample Test to Detect Data Copying in Generative Models
(
Talk
)
|
Kamalika Chaudhuri 🔗 |
Sat 3:00 p.m. - 3:30 p.m.
|
Audrey Durand, Douwe Kiela, Kamalika Chaudhuri moderated by Yann Dauphin
(
Talks and mini-panel
)
The mini-panel with Audrey Durand, Douwe Kiela, Kamalika Chaudhuri will be moderated by Yann Dauphin. Session advisors are Orhan First and Dilan Gorur. |
Audrey Durand · Kamalika Chaudhuri · Yann Dauphin · Orhan Firat · Dilan Gorur · Douwe Kiela 🔗 |
Sat 3:30 p.m. - 4:15 p.m.
|
Coffee and posters
|
🔗 |
Sat 4:15 p.m. - 5:10 p.m.
|
Panel - The Role of Communication at Large: Aparna Lakshmiratan, Jason Yosinski, Been Kim, Surya Ganguli, Finale Doshi-Velez
(
Panel
)
The panel with Aparna Lakshmiratan, Jason Yosinski, Been Kim, Surya Ganguli, Finale Doshi-Velez will be moderated by Zack Lipton. |
Aparna Lakshmiratan · Finale Doshi-Velez · Surya Ganguli · Zachary Lipton · Michela Paganini · Anima Anandkumar · Jason Yosinski 🔗 |
Sat 5:10 p.m. - 6:00 p.m.
|
Contributed Session - Spotlight Talks
(
Short talks
)
|
Jonathan Frankle · David Schwab · Ari Morcos · Qianli Ma · Yao-Hung Hubert Tsai · Ruslan Salakhutdinov · YiDing Jiang · Dilip Krishnan · Hossein Mobahi · Samy Bengio · Sho Yaida · Muqiao Yang
|
Author Information
Levent Sagun (Facebook AI Research)
Caglar Gulcehre (Deepmind)
Adriana Romero Soriano (FAIR)
Negar Rostamzadeh (Element AI)
Bio: Negar Rostamzadeh is a Senior Research Scientist at Google.
Nando de Freitas (DeepMind)
More from the Same Authors
-
2021 : Benchmarking Bias Mitigation Algorithms in Representation Learning through Fairness Metrics »
Charan Reddy · Deepak Sharma · Soroush Mehri · Adriana Romero Soriano · Samira Shabanian · Sina Honari -
2021 Spotlight: Instance-Conditioned GAN »
Arantxa Casanova · Marlene Careil · Jakob Verbeek · Michal Drozdzal · Adriana Romero Soriano -
2021 : Artsheets for Art Datasets »
Ramya Srinivasan · Remi Denton · Jordan Famularo · Negar Rostamzadeh · Fernando Diaz · Beth Coleman -
2021 : Thinking Beyond Distributions in Testing Machine Learned Models »
Negar Rostamzadeh · Ben Hutchinson · Vinodkumar Prabhakaran -
2021 : StarCraft II Unplugged: Large Scale Offline Reinforcement Learning »
Michael Mathieu · Sherjil Ozair · Srivatsan Srinivasan · Caglar Gulcehre · Shangtong Zhang · Ray Jiang · Tom Paine · Konrad Żołna · Julian Schrittwieser · David Choi · Petko I Georgiev · Daniel Toyama · Roman Ring · Igor Babuschkin · Timo Ewalds · · Aaron van den Oord · Wojciech Czarnecki · Nando de Freitas · Oriol Vinyals -
2022 : Multi-step Planning for Automated Hyperparameter Optimization with OptFormer »
Lucio M Dery · Abram Friesen · Nando de Freitas · Marc'Aurelio Ranzato · Yutian Chen -
2022 : Group Excess Risk Bound of Overparameterized Linear Regression with Constant-Stepsize SGD »
Arjun Subramonian · Levent Sagun · Kai-Wei Chang · Yizhou Sun -
2022 : From plane crashes to algorithmic harm: applicability of safety engineering frameworks for responsible ML »
Shalaleh Rismani · Renee Shelby · Andrew Smart · Edgar Jatho · Joshua Kroll · AJung Moon · Negar Rostamzadeh -
2022 : Q & A »
Cheng-Zhi Anna Huang · Negar Rostamzadeh · Mark Riedl -
2022 Tutorial: Creative Culture and Machine Learning »
Negar Rostamzadeh · Cheng-Zhi Anna Huang · Mark Riedl -
2022 : Tutorial part 1 »
Negar Rostamzadeh · Mark Riedl · Cheng-Zhi Anna Huang -
2022 : Ethics Roundtable »
Negar Rostamzadeh · Sina Fazelpour · Nyalleng Moorosi -
2022 Poster: Towards Learning Universal Hyperparameter Optimizers with Transformers »
Yutian Chen · Xingyou Song · Chansoo Lee · Zi Wang · Richard Zhang · David Dohan · Kazuya Kawakami · Greg Kochanski · Arnaud Doucet · Marc'Aurelio Ranzato · Sagi Perel · Nando de Freitas -
2021 : Retrospective Panel »
Sergey Levine · Nando de Freitas · Emma Brunskill · Finale Doshi-Velez · Nan Jiang · Rishabh Agarwal -
2021 Poster: Instance-Conditioned GAN »
Arantxa Casanova · Marlene Careil · Jakob Verbeek · Michal Drozdzal · Adriana Romero Soriano -
2021 Poster: Active 3D Shape Reconstruction from Vision and Touch »
Edward Smith · David Meger · Luis Pineda · Roberto Calandra · Jitendra Malik · Adriana Romero Soriano · Michal Drozdzal -
2021 Poster: Parameter Prediction for Unseen Deep Architectures »
Boris Knyazev · Michal Drozdzal · Graham Taylor · Adriana Romero Soriano -
2021 Poster: Active Offline Policy Selection »
Ksenia Konyushova · Yutian Chen · Thomas Paine · Caglar Gulcehre · Cosmin Paduraru · Daniel Mankowitz · Misha Denil · Nando de Freitas -
2020 : Panel »
Emma Brunskill · Nan Jiang · Nando de Freitas · Finale Doshi-Velez · Sergey Levine · John Langford · Lihong Li · George Tucker · Rishabh Agarwal · Aviral Kumar -
2020 : Offline RL »
Nando de Freitas -
2020 Poster: Critic Regularized Regression »
Ziyu Wang · Alexander Novikov · Konrad Zolna · Josh Merel · Jost Tobias Springenberg · Scott Reed · Bobak Shahriari · Noah Siegel · Caglar Gulcehre · Nicolas Heess · Nando de Freitas -
2020 Poster: Modular Meta-Learning with Shrinkage »
Yutian Chen · Abram Friesen · Feryal Behbahani · Arnaud Doucet · David Budden · Matthew Hoffman · Nando de Freitas -
2020 Spotlight: Modular Meta-Learning with Shrinkage »
Yutian Chen · Abram Friesen · Feryal Behbahani · Arnaud Doucet · David Budden · Matthew Hoffman · Nando de Freitas -
2020 Poster: RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learning »
Caglar Gulcehre · Ziyu Wang · Alexander Novikov · Thomas Paine · Sergio Gómez · Konrad Zolna · Rishabh Agarwal · Josh Merel · Daniel Mankowitz · Cosmin Paduraru · Gabriel Dulac-Arnold · Jerry Li · Mohammad Norouzi · Matthew Hoffman · Nicolas Heess · Nando de Freitas -
2019 : Welcoming remarks and introduction »
Levent Sagun · Caglar Gulcehre · Adriana Romero Soriano · Negar Rostamzadeh · Nando de Freitas -
2019 : Poster Session »
Jonathan Scarlett · Piotr Indyk · Ali Vakilian · Adrian Weller · Partha P Mitra · Benjamin Aubin · Bruno Loureiro · Florent Krzakala · Lenka Zdeborová · Kristina Monakhova · Joshua Yurtsever · Laura Waller · Hendrik Sommerhoff · Michael Moeller · Rushil Anirudh · Shuang Qiu · Xiaohan Wei · Zhuoran Yang · Jayaraman Thiagarajan · Salman Asif · Michael Gillhofer · Johannes Brandstetter · Sepp Hochreiter · Felix Petersen · Dhruv Patel · Assad Oberai · Akshay Kamath · Sushrut Karmalkar · Eric Price · Ali Ahmed · Zahra Kadkhodaie · Sreyas Mohan · Eero Simoncelli · Carlos Fernandez-Granda · Oscar Leong · Wesam Sakla · Rebecca Willett · Stephan Hoyer · Jascha Sohl-Dickstein · Sam Greydanus · Gauri Jagatap · Chinmay Hegde · Michael Kellman · Jonathan Tamir · Nouamane Laanait · Ousmane Dia · Mirco Ravanelli · Jonathan Binas · Negar Rostamzadeh · Shirin Jalali · Tiantian Fang · Alex Schwing · Sébastien Lachapelle · Philippe Brouillard · Tristan Deleu · Simon Lacoste-Julien · Stella Yu · Arya Mazumdar · Ankit Singh Rawat · Yue Zhao · Jianshu Chen · Xiaoyang Li · Hubert Ramsauer · Gabrio Rizzuti · Nikolaos Mitsakos · Dingzhou Cao · Thomas Strohmer · Yang Li · Pei Peng · Gregory Ongie -
2019 Poster: Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias »
Stéphane d'Ascoli · Levent Sagun · Giulio Biroli · Joan Bruna -
2019 Poster: Adaptive Cross-Modal Few-shot Learning »
Chen Xing · Negar Rostamzadeh · Boris Oreshkin · Pedro O. Pinheiro -
2019 Poster: Neural Multisensory Scene Inference »
Jae Hyun Lim · Pedro O. Pinheiro · Negar Rostamzadeh · Chris Pal · Sungjin Ahn -
2019 Poster: Learning Compositional Neural Programs with Recursive Tree Search and Planning »
Thomas PIERROT · Guillaume Ligner · Scott Reed · Olivier Sigaud · Nicolas Perrin · Alexandre Laterre · David Kas · Karim Beguir · Nando de Freitas -
2019 Spotlight: Learning Compositional Neural Programs with Recursive Tree Search and Planning »
Thomas PIERROT · Guillaume Ligner · Scott Reed · Olivier Sigaud · Nicolas Perrin · Alexandre Laterre · David Kas · Karim Beguir · Nando de Freitas -
2018 : TBA 5 »
Nando de Freitas -
2018 : Invited Talk 5: Nando de Freitas »
Nando de Freitas -
2018 : Coffee Break and Poster Session I »
Pim de Haan · Bin Wang · Dequan Wang · Aadil Hayat · Ibrahim Sobh · Muhammad Asif Rana · Thibault Buhet · Nicholas Rhinehart · Arjun Sharma · Alex Bewley · Michael Kelly · Lionel Blondé · Ozgur S. Oguz · Vaibhav Viswanathan · Jeroen Vanbaar · Konrad Żołna · Negar Rostamzadeh · Rowan McAllister · Sanjay Thakur · Alexandros Kalousis · Chelsea Sidrane · Sujoy Paul · Daphne Chen · Michal Garmulewicz · Henryk Michalewski · Coline Devin · Hongyu Ren · Jiaming Song · Wen Sun · Hanzhang Hu · Wulong Liu · Emilie Wirbel -
2018 Poster: Playing hard exploration games by watching YouTube »
Yusuf Aytar · Tobias Pfaff · David Budden · Thomas Paine · Ziyu Wang · Nando de Freitas -
2018 Spotlight: Playing hard exploration games by watching YouTube »
Yusuf Aytar · Tobias Pfaff · David Budden · Thomas Paine · Ziyu Wang · Nando de Freitas -
2017 Poster: Robust Imitation of Diverse Behaviors »
Ziyu Wang · Josh Merel · Scott Reed · Nando de Freitas · Gregory Wayne · Nicolas Heess -
2017 Poster: Plan, Attend, Generate: Planning for Sequence-to-Sequence Models »
Caglar Gulcehre · Francis Dutil · Adam Trischler · Yoshua Bengio -
2017 Tutorial: Deep Learning: Practice and Trends »
Nando de Freitas · Scott Reed · Oriol Vinyals -
2016 Workshop: Neural Abstract Machines & Program Induction »
Matko Bošnjak · Nando de Freitas · Tejas Kulkarni · Arvind Neelakantan · Scott E Reed · Sebastian Riedel · Tim Rocktäschel -
2016 : Nando De Freitas »
Nando de Freitas -
2016 : Learning To Optimize »
Nando de Freitas -
2016 Poster: Learning to learn by gradient descent by gradient descent »
Marcin Andrychowicz · Misha Denil · Sergio Gómez · Matthew Hoffman · David Pfau · Tom Schaul · Nando de Freitas -
2015 Workshop: Bayesian Optimization: Scalability and Flexibility »
Bobak Shahriari · Ryan Adams · Nando de Freitas · Amar Shah · Roberto Calandra