Timezone: »
Poster
ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies
Bao Wang · Zuoqiang Shi · Stanley Osher
Tue Dec 10 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #14
We unify the theory of optimal control of transport equations with the practice of training and testing of ResNets. Based on this unified viewpoint, we propose a simple yet effective ResNets ensemble algorithm to boost the accuracy of the robustly trained model on both clean and adversarial images. The proposed algorithm consists of two components: First, we modify the base ResNets by injecting a variance specified Gaussian noise to the output of each residual mapping. Second, we average over the production of multiple jointly trained modified ResNets to get the final prediction. These two steps give an approximation to the Feynman-Kac formula for representing the solution of a convection-diffusion equation. For the CIFAR10 benchmark, this simple algorithm leads to a robust model with a natural accuracy of {\bf 85.62}\% on clean images and a robust accuracy of ${\bf 57.94 \%}$ under the 20 iterations of the IFGSM attack, which outperforms the current state-of-the-art in defending against IFGSM attack on the CIFAR10.
Author Information
Bao Wang (UCLA)
Zuoqiang Shi (zqshi@mail.tsinghua.edu.cn)
Stanley Osher (UCLA)
More from the Same Authors
-
2022 Poster: M2N: Mesh Movement Networks for PDE Solvers »
Wenbin Song · Mingrui Zhang · Joseph G Wallwork · Junpeng Gao · Zheng Tian · Fanglei Sun · Matthew Piggott · Junqing Chen · Zuoqiang Shi · Xiang Chen · Jun Wang -
2022 Poster: Finite-Time Analysis of Adaptive Temporal Difference Learning with Deep Neural Networks »
Tao Sun · Dongsheng Li · Bao Wang -
2022 Poster: Improving Neural Ordinary Differential Equations with Nesterov's Accelerated Gradient Method »
Ho Huu Nghia Nguyen · Tan Nguyen · Huyen Vo · Stanley Osher · Thieu Vo -
2022 Poster: FourierFormer: Transformer Meets Generalized Fourier Integral Theorem »
Tan Nguyen · Minh Pham · Tam Nguyen · Khai Nguyen · Stanley Osher · Nhat Ho -
2022 Poster: Improving Transformer with an Admixture of Attention Heads »
Tan Nguyen · Tam Nguyen · Hai Do · Khai Nguyen · Vishwanath Saragadam · Minh Pham · Khuong Duy Nguyen · Nhat Ho · Stanley Osher -
2021 : Performance-Guaranteed ODE Solvers with Complexity-Informed Neural Networks »
Feng Zhao · Xiang Chen · Jun Wang · Zuoqiang Shi · Shao-Lun Huang -
2021 : Stan Osher Talk »
Stanley Osher -
2021 Poster: FMMformer: Efficient and Flexible Transformer via Decomposed Near-field and Far-field Attention »
Tan Nguyen · Vai Suliafu · Stanley Osher · Long Chen · Bao Wang -
2021 Poster: Heavy Ball Neural Ordinary Differential Equations »
Hedi Xia · Vai Suliafu · Hangjie Ji · Tan Nguyen · Andrea Bertozzi · Stanley Osher · Bao Wang -
2020 Poster: MomentumRNN: Integrating Momentum into Recurrent Neural Networks »
Tan Nguyen · Richard Baraniuk · Andrea Bertozzi · Stanley Osher · Bao Wang -
2018 Poster: Deep Neural Nets with Interpolating Function as Output Activation »
Bao Wang · Xiyang Luo · Zhen Li · Wei Zhu · Zuoqiang Shi · Stanley Osher