Timezone: »

MORA: Improving Ensemble Robustness Evaluation with Model Reweighing Attack
yunrui yu · Xitong Gao · Cheng-Zhong Xu

Thu Dec 01 09:00 AM -- 11:00 AM (PST) @ Hall J #708
Adversarial attacks can deceive neural networks by adding tiny perturbations to their input data. Ensemble defenses, which are trained to minimize attack transferability among sub-models, offer a promising research direction to improve robustness against such attacks while maintaining a high accuracy on natural inputs. We discover, however, that recent state-of-the-art (SOTA) adversarial attack strategies cannot reliably evaluate ensemble defenses, sizeably overestimating their robustness. This paper identifies the two factors that contribute to this behavior. First, these defenses form ensembles that are notably difficult for existing gradient-based method to attack, due to gradient obfuscation. Second, ensemble defenses diversify sub-model gradients, presenting a challenge to defeat all sub-models simultaneously, simply summing their contributions may counteract the overall attack objective; yet, we observe that ensemble may still be fooled despite most sub-models being correct. We therefore introduce MORA, a model-reweighing attack to steer adversarial example synthesis by reweighing the importance of sub-model gradients. MORA finds that recent ensemble defenses all exhibit varying degrees of overestimated robustness. Comparing it against recent SOTA white-box attacks, it can converge orders of magnitude faster while achieving higher attack success rates across all ensemble models examined with three different ensemble modes (i.e, ensembling by either softmax, voting or logits). In particular, most ensemble defenses exhibit near or exactly $0\%$ robustness against MORA with $\ell^\infty$ perturbation within $0.02$ on CIFAR-10, and $0.01$ on CIFAR-100. We make MORA open source with reproducible results and pre-trained models; and provide a leaderboard of ensemble defenses under various attack strategies.

Author Information

yunrui yu (Intelligent Transportation)
yunrui yu

Yunrui Yu received the BSc and MSc degrees in Space Engineering from Beihang University, in 2016 and 2019, respectively. He is currently working towards his PhD degree in Computer Science and Technology from the University of Macau. His research interests include Adversarial Attacks and Defenses in Deep Learning.

Xitong Gao (Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences)
Cheng-Zhong Xu (University of Macau)

More from the Same Authors