Timezone: »

 
Poster
When does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?
Lijie Fan · Sijia Liu · Pin-Yu Chen · Gaoyuan Zhang · Chuang Gan

Tue Dec 07 08:30 AM -- 10:00 AM (PST) @
in Poster Session 1 »

Contrastive learning (CL) can learn generalizable feature representations and achieve state-of-the-art performance of downstream tasks by finetuning a linear classifier on top of it. However, as adversarial robustness becomes vital in image classification, it remains unclear whether or not CL is able to preserve robustness to downstream tasks. The main challenge is that in the self-supervised pretraining + supervised finetuning paradigm, adversarial robustness is easily forgotten due to a learning task mismatch from pretraining to finetuning. We call such challenge 'cross-task robustness transferability'. To address the above problem, in this paper we revisit and advance CL principles through the lens of robustness enhancement. We show that (1) the design of contrastive views matters: High-frequency components of images are beneficial to improving model robustness; (2) Augmenting CL with pseudo-supervision stimulus (e.g., resorting to feature clustering) helps preserve robustness without forgetting. Equipped with our new designs, we propose AdvCL, a novel adversarial contrastive pretraining framework. We show that AdvCL is able to enhance cross-task robustness transferability without loss of model accuracy and finetuning efficiency. With a thorough experimental study, we demonstrate that AdvCL outperforms the state-of-the-art self-supervised robust learning methods across multiple datasets (CIFAR-10, CIFAR-100, and STL-10) and finetuning schemes (linear evaluation and full model finetuning).

Keywords: Machine Learning · Robustness · Adversarial Robustness and Security · Self-Supervised Learning · Vision · Contrastive Learning · Clustering

Author Information

Lijie Fan (MIT)
Sijia Liu (Michigan State University)
Pin-Yu Chen (IBM Research AI)
Gaoyuan Zhang (International Business Machines)
Chuang Gan (MIT-IBM Watson AI Lab)

More from the Same Authors