Timezone: »
Poster
Semialgebraic Representation of Monotone Deep Equilibrium Models and Applications to Certification
Tong Chen · Jean Lasserre · Victor Magron · Edouard Pauwels
Deep equilibrium models are based on implicitly defined functional relations and have shown competitive performance compared with the traditional deep networks. Monotone operator equilibrium networks (monDEQ) retain interesting performance with additional theoretical guaranties. Existing certification tools for classical deep networks cannot directly be applied to monDEQs for which much fewer tools exist. We introduce a semialgebraic representation for ReLU based monDEQs which allow to approximate the corresponding input output relation by semidefinite programs (SDP). We present several applications to network certification and obtain SDP models for the following problems : robustness certification, Lipschitz constant estimation, ellipsoidal uncertainty propagation. We use these models to certify robustness of monDEQs with respect to a general $L_p$ norm. Experimental results show that the proposed models outperform existing approaches for monDEQ certification. Furthermore, our investigations suggest that monDEQs are much more robust to $L_2$ perturbations than $L_{\infty}$ perturbations.
Author Information
Tong Chen (LAAS-CNRS)
Jean Lasserre (lasserre@laas.fr)
Victor Magron (LAAS-CNRS)
Edouard Pauwels (IRIT)
More from the Same Authors
-
2022 Poster: Automatic differentiation of nonsmooth iterative algorithms »
Jerome Bolte · Edouard Pauwels · Samuel Vaiter -
2021 Poster: Nonsmooth Implicit Differentiation for Machine-Learning and Optimization »
Jérôme Bolte · Tam Le · Edouard Pauwels · Tony Silveti-Falls -
2021 Poster: Numerical influence of ReLU’(0) on backpropagation »
David Bertoin · Jérôme Bolte · Sébastien Gerchinovitz · Edouard Pauwels -
2020 Poster: Semialgebraic Optimization for Lipschitz Constants of ReLU Networks »
Tong Chen · Jean Lasserre · Victor Magron · Edouard Pauwels -
2016 : The moment-LP and moment-SOS approaches in optimization and some related applications »
Jean Lasserre -
2016 Poster: Sorting out typicality with the inverse moment matrix SOS polynomial »
Edouard Pauwels · Jean Lasserre -
2014 Workshop: OPT2014: Optimization for Machine Learning »
Zaid Harchaoui · Suvrit Sra · Alekh Agarwal · Martin Jaggi · Miro Dudik · Aaditya Ramdas · Jean Lasserre · Yoshua Bengio · Amir Beck