Timezone: »

 
Poster
Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks
Hanxun Huang · Yisen Wang · Sarah Erfani · Quanquan Gu · James Bailey · Xingjun Ma

Tue Dec 07 04:30 PM -- 06:00 PM (PST) @
in Poster Session 2 »

Deep neural networks (DNNs) are known to be vulnerable to adversarial attacks. A range of defense methods have been proposed to train adversarially robust DNNs, among which adversarial training has demonstrated promising results. However, despite preliminary understandings developed for adversarial training, it is still not clear, from the architectural perspective, what configurations can lead to more robust DNNs. In this paper, we address this gap via a comprehensive investigation on the impact of network width and depth on the robustness of adversarially trained DNNs. Specifically, we make the following key observations: 1) more parameters (higher model capacity) does not necessarily help adversarial robustness; 2) reducing capacity at the last stage (the last group of blocks) of the network can actually improve adversarial robustness; and 3) under the same parameter budget, there exists an optimal architectural configuration for adversarial robustness. We also provide a theoretical analysis explaning why such network configuration can help robustness. These architectural insights can help design adversarially robust DNNs.

Keywords: Deep Learning · Robustness · Adversarial Robustness and Security

Author Information

Hanxun Huang (University of Melbourne)
Yisen Wang (Peking University)
Sarah Erfani (University of Melbourne)
Quanquan Gu (UCLA)
James Bailey (THE UNIVERSITY OF MELBOURNE)
Xingjun Ma (Deakin University)

More from the Same Authors