Timezone: »

 
Poster
Towards Efficient and Effective Adversarial Training
Gaurang Sriramanan · Sravanti Addepalli · Arya Baburaj · Venkatesh Babu R

Wed Dec 08 04:30 PM -- 06:00 PM (PST) @ Virtual
in Poster Session 4 »

The vulnerability of Deep Neural Networks to adversarial attacks has spurred immense interest towards improving their robustness. However, present state-of-the-art adversarial defenses involve the use of 10-step adversaries during training, which renders them computationally infeasible for application to large-scale datasets. While the recent single-step defenses show promising direction, their robustness is not on par with multi-step training methods. In this work, we bridge this performance gap by introducing a novel Nuclear-Norm regularizer on network predictions to enforce function smoothing in the vicinity of data samples. While prior works consider each data sample independently, the proposed regularizer uses the joint statistics of adversarial samples across a training minibatch to enhance optimization during both attack generation and training, obtaining state-of-the-art results amongst efficient defenses. We achieve further gains by incorporating exponential averaging of network weights over training iterations. We finally introduce a Hybrid training approach that combines the effectiveness of a two-step variant of the proposed defense with the efficiency of a single-step defense. We demonstrate superior results when compared to multi-step defenses such as TRADES and PGD-AT as well, at a significantly lower computational cost.

Keywords: Deep Learning · Optimization · Robustness · Adversarial Robustness and Security

Author Information

Gaurang Sriramanan (Indian Institute of Science, Bangalore)
Sravanti Addepalli (Indian Institute of Science)
Arya Baburaj (Indian Institute of Science)
Venkatesh Babu R (Indian Institute of Science)

More from the Same Authors