Timezone: »
Adversarial robustness has received increasing attention along with the study of adversarial examples. So far, existing works show that robust models not only obtain robustness against various adversarial attacks but also boost the performance in some downstream tasks. However, the underlying mechanism of adversarial robustness is still not clear. In this paper, we interpret adversarial robustness from the perspective of linear components, and find that there exist some statistical properties for comprehensively robust models. Specifically, robust models show obvious hierarchical clustering effect on their linearized sub-networks, when removing or replacing all non-linear components (e.g., batch normalization, maximum pooling, or activation layers). Based on these observations, we propose a novel understanding of adversarial robustness and apply it on more tasks including domain adaption and robustness boosting. Experimental evaluations demonstrate the rationality and superiority of our proposed clustering strategy. Our code is available at https://github.com/bymavis/AdvWeightNeurIPS2021.
Author Information
Yang Bai (Tsinghua University)
Xin Yan (None)
Yong Jiang (Tsinghua)
Shu-Tao Xia (Tsinghua University)
Yisen Wang (Peking University)
Related Events (a corresponding poster, oral, or spotlight)
-
2021 Spotlight: Clustering Effect of Adversarial Robust Models »
Dates n/a. Room
More from the Same Authors
-
2021 Spotlight: Training Feedback Spiking Neural Networks by Implicit Differentiation on the Equilibrium State »
Mingqing Xiao · Qingyan Meng · Zongpeng Zhang · Yisen Wang · Zhouchen Lin -
2022 Poster: Improving Out-of-Distribution Generalization by Adversarial Training with Structured Priors »
Qixun Wang · Yifei Wang · Hong Zhu · Yisen Wang -
2022 Poster: When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecture »
Yichuan Mo · Dongxian Wu · Yifei Wang · Yiwen Guo · Yisen Wang -
2022 : BAAT: Towards Sample-specific Backdoor Attack with Clean Labels »
Yiming Li · Mingyan Zhu · Chengxiao Luo · Haiqing Weng · Yong Jiang · Tao Wei · Shu-Tao Xia -
2023 Poster: Interpreting Unsupervised Anomaly Detection in Security via Rule Extraction »
Ruoyu Li · Qing Li · Yu Zhang · Dan Zhao · Yong Jiang · Yong Yang -
2023 Poster: Balance, Imbalance, and Rebalance: Understanding Robust Overfitting from a Minimax Game Perspective »
Yifei Wang · Liangchen Li · Jiansheng Yang · Zhouchen Lin · Yisen Wang -
2023 Poster: Adversarial Examples Are Not Real Features »
Ang Li · Yifei Wang · Yiwen Guo · Yisen Wang -
2023 Poster: GEQ: Gaussian Kernel Inspired Equilibrium Models »
Mingjie Li · Yisen Wang · Zhouchen Lin -
2023 Poster: Domain Watermark: Effective and Harmless Dataset Copyright Protection is Closed at Hand »
Junfeng Guo · Yiming Li · Lixu Wang · Shu-Tao Xia · Heng Huang · Cong Liu · Bo Li -
2023 Poster: Architecture Matters: Uncovering Implicit Mechanisms in Graph Contrastive Learning »
Xiaojun Guo · Yifei Wang · Zeming Wei · Yisen Wang -
2023 Poster: Identifiable Contrastive Learning with Automatic Feature Importance Discovery »
Qi Zhang · Yifei Wang · Yisen Wang -
2023 Poster: Laplacian Canonization: A Minimalist Approach to Sign and Basis Invariant Spectral Embedding »
George Ma · Yifei Wang · Yisen Wang -
2023 Poster: Metis: Understanding and Enhancing In-Network Regular Expressions »
Zhengxin Zhang · Yucheng Huang · Guanglin Duan · Qing Li · Dan Zhao · Yong Jiang · Lianbo Ma · Xi Xiao · Hengyang Xu -
2022 Spotlight: Lightning Talks 6A-2 »
Yichuan Mo · Botao Yu · Gang Li · Zezhong Xu · Haoran Wei · Arsene Fansi Tchango · Raef Bassily · Haoyu Lu · Qi Zhang · Songming Liu · Mingyu Ding · Peiling Lu · Yifei Wang · Xiang Li · Dongxian Wu · Ping Guo · Wen Zhang · Hao Zhongkai · Mehryar Mohri · Rishab Goel · Yisen Wang · Yifei Wang · Yangguang Zhu · Zhi Wen · Ananda Theertha Suresh · Chengyang Ying · Yujie Wang · Peng Ye · Rui Wang · Nanyi Fei · Hui Chen · Yiwen Guo · Wei Hu · Chenglong Liu · Julien Martel · Yuqi Huo · Wu Yichao · Hang Su · Yisen Wang · Peng Wang · Huajun Chen · Xu Tan · Jun Zhu · Ding Liang · Zhiwu Lu · Joumana Ghosn · Shanshan Zhang · Wei Ye · Ze Cheng · Shikun Zhang · Tao Qin · Tie-Yan Liu -
2022 Spotlight: How Mask Matters: Towards Theoretical Understandings of Masked Autoencoders »
Qi Zhang · Yifei Wang · Yisen Wang -
2022 Spotlight: When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecture »
Yichuan Mo · Dongxian Wu · Yifei Wang · Yiwen Guo · Yisen Wang -
2022 Spotlight: Lightning Talks 1B-3 »
Chaofei Wang · Qixun Wang · Jing Xu · Long-Kai Huang · Xi Weng · Fei Ye · Harsh Rangwani · shrinivas ramasubramanian · Yifei Wang · Qisen Yang · Xu Luo · Lei Huang · Adrian G. Bors · Ying Wei · Xinglin Pan · Sho Takemori · Hong Zhu · Rui Huang · Lei Zhao · Yisen Wang · Kato Takashi · Shiji Song · Yanan Li · Rao Anwer · Yuhei Umeda · Salman Khan · Gao Huang · Wenjie Pei · Fahad Shahbaz Khan · Venkatesh Babu R · Zenglin Xu -
2022 Spotlight: Improving Out-of-Distribution Generalization by Adversarial Training with Structured Priors »
Qixun Wang · Yifei Wang · Hong Zhu · Yisen Wang -
2022 Poster: Untargeted Backdoor Watermark: Towards Harmless and Stealthy Dataset Copyright Protection »
Yiming Li · Yang Bai · Yong Jiang · Yong Yang · Shu-Tao Xia · Bo Li -
2022 Poster: How Mask Matters: Towards Theoretical Understandings of Masked Autoencoders »
Qi Zhang · Yifei Wang · Yisen Wang -
2021 Poster: On Training Implicit Models »
Zhengyang Geng · Xin-Yu Zhang · Shaojie Bai · Yisen Wang · Zhouchen Lin -
2021 Poster: Dissecting the Diffusion Process in Linear Graph Convolutional Networks »
Yifei Wang · Yisen Wang · Jiansheng Yang · Zhouchen Lin -
2021 Poster: Adversarial Neuron Pruning Purifies Backdoored Deep Models »
Dongxian Wu · Yisen Wang -
2021 Poster: Gauge Equivariant Transformer »
Lingshen He · Yiming Dong · Yisen Wang · Dacheng Tao · Zhouchen Lin -
2021 Poster: Training Feedback Spiking Neural Networks by Implicit Differentiation on the Equilibrium State »
Mingqing Xiao · Qingyan Meng · Zongpeng Zhang · Yisen Wang · Zhouchen Lin -
2021 Poster: Efficient Equivariant Network »
Lingshen He · Yuxuan Chen · zhengyang shen · Yiming Dong · Yisen Wang · Zhouchen Lin -
2021 Poster: Towards a Unified Game-Theoretic View of Adversarial Perturbations and Robustness »
Jie Ren · Die Zhang · Yisen Wang · Lu Chen · Zhanpeng Zhou · Yiting Chen · Xu Cheng · Xin Wang · Meng Zhou · Jie Shi · Quanshi Zhang -
2021 Poster: Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks »
Hanxun Huang · Yisen Wang · Sarah Erfani · Quanquan Gu · James Bailey · Xingjun Ma -
2021 Poster: Finding Optimal Tangent Points for Reducing Distortions of Hard-label Attacks »
Chen Ma · Xiangyu Guo · Li Chen · Jun-Hai Yong · Yisen Wang -
2021 Poster: Residual Relaxation for Multi-view Representation Learning »
Yifei Wang · Zhengyang Geng · Feng Jiang · Chuming Li · Yisen Wang · Jiansheng Yang · Zhouchen Lin -
2021 Poster: MoriĆ© Attack (MA): A New Potential Risk of Screen Photos »
Dantong Niu · Ruohao Guo · Yisen Wang -
2020 Poster: Variance Reduction via Accelerated Dual Averaging for Finite-Sum Optimization »
Chaobing Song · Yong Jiang · Yi Ma -
2020 Poster: Optimistic Dual Extrapolation for Coherent Non-monotone Variational Inequalities »
Chaobing Song · Zhengyuan Zhou · Yichao Zhou · Yong Jiang · Yi Ma -
2020 Poster: Adversarial Weight Perturbation Helps Robust Generalization »
Dongxian Wu · Shu-Tao Xia · Yisen Wang -
2020 Poster: Stochastic Deep Gaussian Processes over Graphs »
Naiqi Li · Wenjie Li · Jifeng Sun · Yinghua Gao · Yong Jiang · Shu-Tao Xia -
2018 Poster: BML: A High-performance, Low-cost Gradient Synchronization Algorithm for DML Training »
Songtao Wang · Dan Li · Yang Cheng · Jinkun Geng · Yanshu Wang · Shuai Wang · Shu-Tao Xia · Jianping Wu -
2017 Poster: Accelerated Stochastic Greedy Coordinate Descent by Soft Thresholding Projection onto Simplex »
Chaobing Song · Shaobo Cui · Yong Jiang · Shu-Tao Xia -
2017 Spotlight: Accelerated Stochastic Greedy Coordinate Descent by Soft Thresholding Projection onto Simplex »
Chaobing Song · Shaobo Cui · Yong Jiang · Shu-Tao Xia