Timezone: »

 
Poster
MixACM: Mixup-Based Robustness Transfer via Distillation of Activated Channel Maps
Awais Muhammad · Fengwei Zhou · Chuanlong Xie · Jiawei Li · Sung-Ho Bae · Zhenguo Li

Wed Dec 08 12:30 AM -- 02:00 AM (PST) @ Virtual
in Poster Session 3 »

Deep neural networks are susceptible to adversarially crafted, small, and imperceptible changes in the natural inputs. The most effective defense mechanism against these examples is adversarial training which constructs adversarial examples during training by iterative maximization of loss. The model is then trained to minimize the loss on these constructed examples. This min-max optimization requires more data, larger capacity models, and additional computing resources. It also degrades the standard generalization performance of a model. Can we achieve robustness more efficiently? In this work, we explore this question from the perspective of knowledge transfer. First, we theoretically show the transferability of robustness from an adversarially trained teacher model to a student model with the help of mixup augmentation. Second, we propose a novel robustness transfer method called Mixup-Based Activated Channel Maps (MixACM) Transfer. MixACM transfers robustness from a robust teacher to a student by matching activated channel maps generated without expensive adversarial perturbations. Finally, extensive experiments on multiple datasets and different learning scenarios show our method can transfer robustness while also improving generalization on natural images.

Keywords: Deep Learning · Optimization · Robustness · Adversarial Robustness and Security

Author Information

Awais Muhammad (AI Theory Group, Noah's Ark Lab, Huawei Technologies Ltd, Hong Kong)
Fengwei Zhou (Huawei Technologies Ltd.)
Chuanlong Xie (Huawei Noah's Ark Lab)
Jiawei Li (Huawei Technologies Ltd.)
Sung-Ho Bae (Kyung Hee University)
Zhenguo Li (Noah's Ark Lab, Huawei Tech Investment Co Ltd)

More from the Same Authors