Timezone: »
The Lipschitz constant of a network plays an important role in many applications of deep learning, such as robustness certification and Wasserstein Generative Adversarial Network. We introduce a semidefinite programming hierarchy to estimate the global and local Lipschitz constant of a multiple layer deep neural network. The novelty is to combine a polynomial lifting for ReLU functions derivatives with a weak generalization of Putinar's positivity certificate. This idea could also apply to other, nearly sparse, polynomial optimization problems in machine learning. We empirically demonstrate that our method provides a trade-off with respect to state of the art linear programming approach, and in some cases we obtain better bounds in less time.
Author Information
Tong Chen (LAAS-CNRS)
Jean Lasserre (lasserre@laas.fr)
Victor Magron (LAAS-CNRS)
Edouard Pauwels (IRIT)
More from the Same Authors
-
2022 Poster: Automatic differentiation of nonsmooth iterative algorithms »
Jerome Bolte · Edouard Pauwels · Samuel Vaiter -
2021 Poster: Semialgebraic Representation of Monotone Deep Equilibrium Models and Applications to Certification »
Tong Chen · Jean Lasserre · Victor Magron · Edouard Pauwels -
2021 Poster: Nonsmooth Implicit Differentiation for Machine-Learning and Optimization »
Jérôme Bolte · Tam Le · Edouard Pauwels · Tony Silveti-Falls -
2021 Poster: Numerical influence of ReLU’(0) on backpropagation »
David Bertoin · Jérôme Bolte · Sébastien Gerchinovitz · Edouard Pauwels -
2020 Poster: A mathematical model for automatic differentiation in machine learning »
Jérôme Bolte · Edouard Pauwels -
2020 Spotlight: A mathematical model for automatic differentiation in machine learning »
Jérôme Bolte · Edouard Pauwels -
2018 Poster: Relating Leverage Scores and Density using Regularized Christoffel Functions »
Edouard Pauwels · Francis Bach · Jean-Philippe Vert -
2016 : The moment-LP and moment-SOS approaches in optimization and some related applications »
Jean Lasserre -
2016 Poster: Sorting out typicality with the inverse moment matrix SOS polynomial »
Edouard Pauwels · Jean Lasserre -
2014 Workshop: OPT2014: Optimization for Machine Learning »
Zaid Harchaoui · Suvrit Sra · Alekh Agarwal · Martin Jaggi · Miro Dudik · Aaditya Ramdas · Jean Lasserre · Yoshua Bengio · Amir Beck