Optimization lies at the heart of many machine learning algorithms and enjoys great interest in our community. Indeed, this intimate relation of optimization with ML is the key motivation for the OPT series of workshops.
Looking back over the past decade, a strong trend is apparent: The intersection of OPT and ML has grown to the point that now cutting-edge advances in optimization often arise from the ML community. The distinctive feature of optimization within ML is its departure from textbook approaches, in particular, its focus on a different set of goals driven by "big-data, nonconvexity, and high-dimensions," where both theory and implementation are crucial.
We wish to use OPT 2020 as a platform to foster discussion, discovery, and dissemination of the state-of-the-art in optimization as relevant to machine learning. And well beyond that: as a platform to identify new directions and challenges that will drive future research, and continue to build the OPT+ML joint research community.
Invited Speakers
Volkan Cevher (EPFL)
Michael Friedlander (UBC)
Donald Goldfarb (Columbia)
Andreas Krause (ETH, Zurich)
Suvrit Sra (MIT)
Rachel Ward (UT Austin)
Ashia Wilson (MSR)
Tong Zhang (HKUST)
Instructions
Please join us in gather.town for all breaks and poster sessions (Click "Open Link" on any break or poster session).
To see all submitted paper and posters, go to the "opt-ml website" at the top of the page.
Use RocketChat or Zoom link (top of page) if you want to ask the speaker a direct question during the Live Q&A and Contributed Talks.
| Welcome event (gather.town) (Social event/Break) | |
| Welcome remarks to Session 1 (Opening remarks) | |
| Invited speaker: The Convexity of Learning Infinite-width Deep Neural Networks, Tong Zhang (Talk) | |
| Live Q&A with Tong Zhang (Zoom) (Q&A) | |
| Invited speaker: Adaptation and universality in first-order methods, Volkan Cevher (Talk) | |
| Contributed Video: Constraint-Based Regularization of Neural Networks, Tiffany Vlaar (Talk) | |
| Contributed Video: Can We Find Near-Approximately-Stationary Points of Nonsmooth Nonconvex Functions?, Ohad Shamir (Talk) | |
| Contributed talks in Session 1 (Zoom) (Multiple talks) | |
| Contributed Video: Employing No Regret Learners for Pure Exploration in Linear Bandits, Mohammadi Zaki (Talk) | |
| Contributed Video: Distributed Proximal Splitting Algorithms with Rates and Acceleration, Laurent Condat (Talk) | |
| Contributed Video: PAGE: A Simple and Optimal Probabilistic Gradient Estimator for Nonconvex Optimization, Zhize Li (Talk) | |
| Poster Session 1 (gather.town) (Poster session) | |
| Welcome remarks to Session 2 (Opening remarks) | |
| Invited speaker: Adaptive Sampling for Stochastic Risk-Averse Learning, Andreas Krause (Talk) | |
| Live Q&A with Andreas Krause (Zoom) (Q&A) | |
| Invited speaker: Practical Kronecker-factored BFGS and L-BFGS methods for training deep neural networks, Donald Goldfarb (Talk) | |
| Contributed talks in Session 2 (Zoom) (Multiple talks) | |
| Contributed Video: Adaptive Gradient Methods Converge Faster with Over-Parameterization (and you can do a line-search), Sharan Vaswani (Talk) | |
| Contributed Video: Adaptivity of Stochastic Gradient Methods for Nonconvex Optimization, Samuel Horvath (Talk) | |
| Contributed Video: How to make your optimizer generalize better, Sharan Vaswani (Talk) | |
| Contributed Video: DDPNOpt: Differential Dynamic Programming Neural Optimizer, Guan-Horng Liu (Talk) | |
| Contributed Video: Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence, Nicolas Loizou (Talk) | |
| Break (gather.town) (Break) | |
| Invited speaker: SGD without replacement: optimal rate analysis and more, Suvrit Sra (Talk) | |
| Live Q&A with Suvrit Sra (Zoom) (Q&A) | |
| Poster Session 2 (gather.town) (Poster session) | |
| Welcome remarks to Session 3 (Opening remarks) | |
| Invited speaker: Stochastic Geodesic Optimization, Ashia Wilson (Talk) | |
| Live Q&A with Ashia Wilson (Zoom) (Q&A) | |
| Invited speaker: Concentration for matrix products, and convergence of Oja’s algorithm for streaming PCA, Rachel Ward (Talk) | |
| Live Q&A with Rachel Ward (Zoom) (Q&A) | |
| Contributed talks in Session 3 (Zoom) (Multiple talks) | |
| Contributed Video: Variance Reduction on Adaptive Stochastic Mirror Descent, Wenjie Li (Talk) | |
| Contributed Video: When Does Preconditioning Help or Hurt Generalization?, Denny Wu (Talk) | |
| Contributed Video: Incremental Greedy BFGS: An Incremental Quasi-Newton Method with Explicit Superlinear Rate, Zhan Gao (Talk) | |
| Contributed Video: TenIPS: Inverse Propensity Sampling for Tensor Completion, Chengrun Yang (Talk) | |
| Contributed Video: Learning Rate Annealing Can Provably Help Generalization, Even for Convex Problems, Preetum Nakkiran (Talk) | |
| Break (gather.town) (Break) | |
| Intro to Invited Speaker 8 (Organizer intro) | |
| Invited speaker: Fast convergence of stochastic subgradient method under interpolation, Michael Friedlander (Talk) | |
| Live Q&A with Michael Friedlander (Zoom) (Q&A) | |
| Poster Session 3 (gather.town) (Poster session) | |
| Welcome remarks to Session 4 (Opening remarks) | |
| Invited speaker: Online nonnegative matrix factorization for Markovian and other real data, Deanna Needell and Hanbaek Lyu (Talk) | |
| Live Q&A with Deanna Needell and Hanbake Lyu (Zoom) (Q&A) | |
| Contributed Video: A Study of Condition Numbers for First-Order Optimization, Charles Guille-Escuret (Talk) | |
| Contributed Video: Affine-Invariant Analysis of Frank-Wolfe on Strongly Convex Sets, Lewis Liu (Talk) | |
| Contributed Video: Stochastic Damped L-BFGS with controlled norm of the Hessian approximation, Sanae Lotfi (Talk) | |
| Contributed Video: Convex Programs for Global Optimization of Convolutional Neural Networks in Polynomial-Time, Tolga Ergen (Talk) | |
| Contributed Video: On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization, Dongruo Zhou (Talk) | |
| Contributed talks in Session 4 (Zoom) (Multiple talks) | |
| Closing remarks | |