This is the public, feature-limited version of the conference webpage. After Registration and login please visit the full version.

Workshop: OPT2020: Optimization for Machine Learning

Courtney Paquette, Mark Schmidt, Sebastian Stich, Quanquan Gu, Martin Takac

Fri, Dec 11th @ 11:15 GMT – Sat, Dec 12th @ 00:30 GMT
Abstract: Optimization lies at the heart of many machine learning algorithms and enjoys great interest in our community. Indeed, this intimate relation of optimization with ML is the key motivation for the OPT series of workshops.

Looking back over the past decade, a strong trend is apparent: The intersection of OPT and ML has grown to the point that now cutting-edge advances in optimization often arise from the ML community. The distinctive feature of optimization within ML is its departure from textbook approaches, in particular, its focus on a different set of goals driven by "big-data, nonconvexity, and high-dimensions," where both theory and implementation are crucial.

We wish to use OPT 2020 as a platform to foster discussion, discovery, and dissemination of the state-of-the-art in optimization as relevant to machine learning. And well beyond that: as a platform to identify new directions and challenges that will drive future research, and continue to build the OPT+ML joint research community.

**Invited Speakers**
Volkan Cevher (EPFL)
Michael Friedlander (UBC)
Donald Goldfarb (Columbia)
Andreas Krause (ETH, Zurich)
Suvrit Sra (MIT)
Rachel Ward (UT Austin)
Ashia Wilson (MSR)
Tong Zhang (HKUST)

Please join us in gather.town for all breaks and poster sessions (for link see any abstract for a break or poster session, opens on December 11).

Chat

To ask questions please use rocketchat, available only upon registration and login.

Schedule

11:15 – 11:50 GMT
Welcome event (gather.town)
Quanquan Gu, Courtney Paquette, Mark Schmidt, Sebastian Stich, Martin Takac
11:50 – 12:00 GMT
Welcome remarks to Session 1
Sebastian Stich
12:00 – 12:20 GMT
Invited speaker: The Convexity of Learning Infinite-width Deep Neural Networks, Tong Zhang
Tong Zhang
12:20 – 12:30 GMT
Live Q&A with Tong Zhang (Zoom)
Sebastian Stich
12:30 – 12:50 GMT
Invited speaker: Adaptation and universality in first-order methods, Volkan Cevher
Volkan Cevher
12:50 – 13:00 GMT
Live Q&A with Volkan Cevher (Zoom)
Sebastian Stich
13:00 – 13:30 GMT
Contributed Video: Can We Find Near-Approximately-Stationary Points of Nonsmooth Nonconvex Functions?, Ohad Shamir
Ohad Shamir
13:00 – 13:30 GMT
Contributed Video: Distributed Proximal Splitting Algorithms with Rates and Acceleration, Laurent Condat
Laurent Condat
13:00 – 13:30 GMT
Contributed Video: Employing No Regret Learners for Pure Exploration in Linear Bandits, Mohammadi Zaki
Mohammadi Zaki
13:00 – 13:30 GMT
Contributed Video: Constraint-Based Regularization of Neural Networks, Tiffany Vlaar
Tiffany Vlaar
13:00 – 13:30 GMT
Contributed Video: PAGE: A Simple and Optimal Probabilistic Gradient Estimator for Nonconvex Optimization, Zhize Li
Zhize Li
13:00 – 13:30 GMT
Contributed talks in Session 1 (Zoom)
Sebastian Stich, Laurent Condat, Zhize Li, Ohad Shamir, Tiffany Vlaar, Mohammadi Zaki
13:30 – 14:00 GMT
Break (gather.town)
14:00 – 14:50 GMT
Poster Session 1 (gather.town)
Laurent Condat, Tiffany Vlaar, Ohad Shamir, Mohammadi Zaki, Zhize Li, Guan-Horng Liu, Samuel Horváth, Mher Safaryan, Yoni Choukroun, kumar Shridhar, Nabil Kahale, Jikai Jin, Pratik Kumar Jawanpuria, Gaurav Kumar Yadav, Kazuki Koyama, Junyoung Kim, Xiao Li, Saugata Purkayastha, Adil Salim, Dighanchal Banerjee, Peter Richtarik, Lakshman Mahto, Tian Ye, Bamdev Mishra
14:50 – 15:00 GMT
Welcome remarks to Session 2
Martin Takac
15:00 – 15:20 GMT
Invited speaker: Adaptive Sampling for Stochastic Risk-Averse Learning, Andreas Krause
Andreas Krause
15:20 – 15:30 GMT
Live Q&A with Andreas Krause (Zoom)
Martin Takac
15:30 – 15:50 GMT
Invited speaker: Practical Kronecker-factored BFGS and L-BFGS methods for training deep neural networks, Donald Goldfarb
Donald Goldfarb
15:50 – 16:00 GMT
Live Q&A with Donald Goldfarb (Zoom)
Martin Takac
16:00 – 16:30 GMT
Contributed Video: Adaptive Gradient Methods Converge Faster with Over-Parameterization (and you can do a line-search), Sharan Vaswani
Sharan Vaswani
16:00 – 16:30 GMT
Contributed Video: How to make your optimizer generalize better, Sharan Vaswani
Sharan Vaswani
16:00 – 16:30 GMT
Contributed talks in Session 2 (Zoom)
Martin Takac, Samuel Horváth, Guan-Horng Liu, Nicolas Loizou, Sharan Vaswani
16:00 – 16:30 GMT
Contributed Video: Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence, Nicolas Loizou
Nicolas Loizou
16:00 – 16:30 GMT
Contributed Video: DDPNOpt: Differential Dynamic Programming Neural Optimizer, Guan-Horng Liu
Guan-Horng Liu
16:00 – 16:30 GMT
Contributed Video: Adaptivity of Stochastic Gradient Methods for Nonconvex Optimization, Samuel Horvath
Samuel Horváth
16:30 – 17:00 GMT
Break (gather.town)
17:00 – 17:05 GMT
Intro to Invited Speaker 5
Martin Takac
17:00 – 17:20 GMT
Invited speaker: SGD without replacement: optimal rate analysis and more, Suvrit Sra
Suvrit Sra
17:20 – 17:30 GMT
Live Q&A with Suvrit Sra (Zoom)
Martin Takac
17:45 – 18:50 GMT
Poster Session 2 (gather.town)
Sharan Vaswani, Nicolas Loizou, Wenjie Li, Preetum Nakkiran, Zhan Gao, Sina Baghal, Jingfeng Wu, Roozbeh Yousefzadeh, Jinyi Wang, Jing Wang, Cong Xie, Anastasia Borovykh, Stanislaw Jastrzebski, Soham Dan, Yiliang Zhang, Mark Tuddenham, Sarath Pattathil, Ievgen Redko, Jeremy Cohen, Yasaman Esfandiari, Zhanhong Jiang, Mostafa ElAraby, Chulhee Yun, Jia-Jie Zhu, Michael Psenka, Robert Gower, Xiaoyu Wang
18:50 – 19:00 GMT
Welcome remarks to Session 3
Mark Schmidt
19:00 – 19:20 GMT
Invited speaker: Stochastic Geodesic Optimization, Ashia Wilson
Ashia Wilson
19:20 – 19:30 GMT
Live Q&A with Ashia Wilson (Zoom)
Mark Schmidt
19:30 – 19:50 GMT
Invited speaker: Concentration for matrix products, and convergence of Oja’s algorithm for streaming PCA, Rachel Ward
Rachel Ward
19:50 – 20:00 GMT
Live Q&A with Rachel Ward (Zoom)
Mark Schmidt
20:00 – 20:30 GMT
Contributed Video: Learning Rate Annealing Can Provably Help Generalization, Even for Convex Problems, Preetum Nakkiran
Preetum Nakkiran
20:00 – 20:30 GMT
Contributed Video: TenIPS: Inverse Propensity Sampling for Tensor Completion, Chengrun Yang
Chengrun Yang
20:00 – 20:30 GMT
Contributed Video: Variance Reduction on Adaptive Stochastic Mirror Descent, Wenjie Li
Wenjie Li
20:00 – 20:30 GMT
Contributed Video: Incremental Greedy BFGS: An Incremental Quasi-Newton Method with Explicit Superlinear Rate, Zhan Gao
Zhan Gao
20:00 – 20:30 GMT
Contributed talks in Session 3 (Zoom)
Mark Schmidt, Zhan Gao, Wenjie Li, Preetum Nakkiran, Denny Wu, Chengrun Yang
20:00 – 20:30 GMT
Contributed Video: When Does Preconditioning Help or Hurt Generalization?, Denny Wu
Denny Wu
20:30 – 21:30 GMT
Break (gather.town)
21:30 – 21:50 GMT
Invited speaker: Fast convergence of stochastic subgradient method under interpolation, Michael Friedlander
Michael Friedlander
21:30 – 21:35 GMT
Intro to Invited Speaker 8
Mark Schmidt
21:50 – 22:00 GMT
Live Q&A with Michael Friedlander (Zoom)
Mark Schmidt
22:00 – 22:50 GMT
Poster Session 3 (gather.town)
Denny Wu, Chengrun Yang, Tolga Ergen, sanae lotfi, Charles Guille-Escuret, Boris Ginsburg, Hanbake Lyu, Cong Xie, David Newton, Debraj Basu, Yewen Wang, James Lucas, MAOJIA LI, Lijun Ding, Jose Javier Gonzalez Ortiz, Reyhane Askari Hemmat, Zhiqi Bu, Neal Lawton, Kiran Thekumparampil, Jiaming Liang, Lindon Roberts, Jingyi Zhu
22:50 – 23:00 GMT
Welcome remarks to Session 4
Quanquan Gu
23:00 – 23:20 GMT
Invited speaker: Online nonnegative matrix factorization for Markovian and other real data, Deanna Needell and Hanbaek Lyu
Hanbake Lyu, Deanna Needell
23:20 – 23:30 GMT
Live Q&A with Deanna Needell and Hanbake Lyu (Zoom)
Quanquan Gu
Fri, Dec 11th @ 23:30 GMT – Sat, Dec 12th @ 00:00 GMT
Contributed Video: A Study of Condition Numbers for First-Order Optimization, Charles Guille-Escuret
Charles Guille-Escuret
Fri, Dec 11th @ 23:30 GMT – Sat, Dec 12th @ 00:00 GMT
Contributed talks in Session 4 (Zoom)
Quanquan Gu, sanae lotfi, Charles Guille-Escuret, Tolga Ergen, Dongruo Zhou
Fri, Dec 11th @ 23:30 GMT – Sat, Dec 12th @ 00:00 GMT
Contributed Video: Stochastic Damped L-BFGS with controlled norm of the Hessian approximation, Sanae Lotfi
sanae lotfi
Fri, Dec 11th @ 23:30 GMT – Sat, Dec 12th @ 00:00 GMT
Contributed Video: On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization, Dongruo Zhou
Dongruo Zhou
Fri, Dec 11th @ 23:30 GMT – Sat, Dec 12th @ 00:00 GMT
Contributed Video: Convex Programs for Global Optimization of Convolutional Neural Networks in Polynomial-Time, Tolga Ergen
Tolga Ergen
Fri, Dec 11th @ 23:30 GMT – Sat, Dec 12th @ 00:00 GMT
Contributed Video: Affine-Invariant Analysis of Frank-Wolfe on Strongly Convex Sets, Lewis Liu
00:00 – 00:05 GMT
Closing remarks
Quanquan Gu, Courtney Paquette, Mark Schmidt, Sebastian Stich, Martin Takac