Timezone: »

Deep Learning and Unsupervised Feature Learning
Yoshua Bengio · Adam Coates · Yann LeCun · Nicolas Le Roux · Andrew Y Ng

Thu Dec 15 10:30 PM -- 11:00 AM (PST) @ Telecabina: Movie Theater
Event URL: http://deeplearningworkshopnips2011.wordpress.com/ »

In recent years, there has been a lot of interest in algorithms that learn feature hierarchies from unlabeled data. Deep learning methods such as deep belief networks, sparse coding-based methods, convolutional networks, and deep Boltzmann machines, have shown promise and have already been successfully applied to a variety of tasks in computer vision, audio processing, natural language processing, information retrieval, and robotics. In this workshop, we will bring together researchers who are interested in deep learning and unsupervised feature learning, review the recent technical progress, discuss the challenges, and identify promising future research directions.

Through invited talks, panels and discussions (see program schedule), we will attempt to address some of the more controversial topics in deep learning today, such as whether hierarchical systems are more powerful, the issues of scalability of deep learning, and what principles should guide the design of objective functions used to train these models.

The workshop will also invite paper submissions on the development of unsupervised feature learning and deep learning algorithms, theoretical foundations, inference and optimization, semi-supervised and transfer learning, and applications of deep learning and unsupervised feature learning to real-world tasks. Papers will be presented as oral or poster presentations (with a short spotlight presentation).

The workshop will also have two panel discussion sessions. The main topics of discussion will include:

* Whether/why hierarchical systems are really needed
* How to build hierarchical systems: advantages and disadvantages of bottom-up vs. top-down paradigm.
* Principles underlying learning of hierarchical systems: sparsity, reconstruction, (if supervised) what kind of supervision, how to learn invariances, etc.
* Issues of scalability of unsupervised feature learning and deep learning systems
* Major milestones and goals for the next 5 or 10 years
* Critiques of deep learning
* Real-world applications: what are challenging tasks and datasets?
* Relation to neuroscience: Can or should we design models that are more closely inspired by biological systems? Can we explain neural coding?

Panel discussions will be led by the members of the organizing committee as well as by prominent researchers from related fields.

The goal of this workshop is two-fold. First, we want to identify the next big challenges and propose research directions for the deep learning community. Second, we want to bridge the gap between researchers working on different (but related) fields, to leverage their expertise, and to encourage the exchange of ideas with all the other members of the NIPS community.

The proposed workshop builds on and extends the very successful Deep Learning and Unsupervised Feature Learning workshop held at NIPS 2010, which had over 150 attendees and received 30 research paper submissions.

The tentative timeline is (might be revised depending on the timing of notification of workshop acceptance):

August 30: Call for papers released
October 21: Paper submissions due
October 21 - November 7: Reviewing period
November 11: Notification of acceptance or rejection
December 1: Final version of papers due (for online proceedings)
December 16 or 17: Workshop*

* If possible, we'd prefer a Friday workshop date, which would allow us to organize a dinner for the attendees; but either day is fine.

Author Information

Yoshua Bengio (University of Montreal)

Yoshua Bengio (PhD'1991 in Computer Science, McGill University). After two post-doctoral years, one at MIT with Michael Jordan and one at AT&T Bell Laboratories with Yann LeCun, he became professor at the department of computer science and operations research at Université de Montréal. Author of two books (a third is in preparation) and more than 200 publications, he is among the most cited Canadian computer scientists and is or has been associate editor of the top journals in machine learning and neural networks. Since '2000 he holds a Canada Research Chair in Statistical Learning Algorithms, since '2006 an NSERC Chair, since '2005 his is a Senior Fellow of the Canadian Institute for Advanced Research and since 2014 he co-directs its program focused on deep learning. He is on the board of the NIPS foundation and has been program chair and general chair for NIPS. He has co-organized the Learning Workshop for 14 years and co-created the International Conference on Learning Representations. His interests are centered around a quest for AI through machine learning, and include fundamental questions on deep learning, representation learning, the geometry of generalization in high-dimensional spaces, manifold learning and biologically inspired learning algorithms.

Adam Coates (Baidu Research)
Yann LeCun (Facebook AI Research and New York University)

Yann LeCun is VP & Chief AI Scientist at Meta and Silver Professor at NYU affiliated with the Courant Institute of Mathematical Sciences & the Center for Data Science. He was the founding Director of FAIR (Meta's AI Research group) and of the NYU Center for Data Science. He received an Engineering Diploma from ESIEE (Paris) and a PhD from Sorbonne Université. After a postdoc in Toronto he joined AT&T Bell Labs in 1988, and AT&T Labs in 1996 as Head of Image Processing Research. He joined NYU as a professor in 2003 and Facebook in 2013. His interests include AI machine learning, computer perception, robotics and computational neuroscience. He is the recipient of the 2018 ACM Turing Award (with Geoffrey Hinton and Yoshua Bengio) for "conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing", a member of the National Academy of Sciences, the National Academy of Engineering and a Chevalier de la Légion d’Honneur.

Nicolas Le Roux (Google Brain)
Andrew Y Ng (Baidu Research)

More from the Same Authors