Timezone: »

 
Poster
Robust Contrastive Learning Using Negative Samples with Diminished Semantics
Songwei Ge · Shlok Mishra · Chun-Liang Li · Haohan Wang · David Jacobs

Fri Dec 10 08:30 AM -- 10:00 AM (PST) @
in Poster Session 8 »

Unsupervised learning has recently made exceptional progress because of the development of more effective contrastive learning methods. However, CNNs are prone to depend on low-level features that humans deem non-semantic. This dependency has been conjectured to induce a lack of robustness to image perturbations or domain shift. In this paper, we show that by generating carefully designed negative samples, contrastive learning can learn more robust representations with less dependence on such features. Contrastive learning utilizes positive pairs which preserve semantic information while perturbing superficial features in the training images. Similarly, we propose to generate negative samples in a reversed way, where only the superfluous instead of the semantic features are preserved. We develop two methods, texture-based and patch-based augmentations, to generate negative samples. These samples achieve better generalization, especially under out-of-domain settings. We also analyze our method and the generated texture-based samples, showing that texture features are indispensable in classifying particular ImageNet classes and especially finer classes. We also show that the model bias between texture and shape features favors them differently under different test settings.

Keywords: Robustness · Self-Supervised Learning · Contrastive Learning

Author Information

Songwei Ge (University of Maryland, College Park)
Shlok Mishra (University of Maryland, College Park)
Chun-Liang Li (Google)
Haohan Wang (Carnegie Mellon University)
David Jacobs (University of Maryland, USA)

More from the Same Authors