Timezone: »
With the rapid developments of deep neural networks, numerous deep cross-modal analysis methods have been presented and are being applied in widespread real-world applications, including healthcare and safety-critical environments. However, the recent studies on robustness and stability of deep neural networks show that a microscopic modification, known as adversarial sample, which is even imperceptible to humans, can easily fool a well-performed deep neural network and brings a new obstacle to deep cross-modal correlation exploring. In this paper, we propose a novel Cross-Modal correlation Learning with Adversarial samples, namely CMLA, which for the first time presents the existence of adversarial samples in cross-modal data. Moreover, we provide a simple yet effective adversarial sample learning method, where inter- and intra- modality similarity regularizations across different modalities are simultaneously integrated into the learning of adversarial samples. Finally, our proposed CMLA is demonstrated to be highly effective in cross-modal hashing based retrieval. Extensive experiments on two cross-modal benchmark datasets show that the adversarial examples produced by our CMLA are efficient in fooling a target deep cross-modal hashing network. On the other hand, such adversarial examples can significantly strengthen the robustness of the target network by conducting an adversarial training.
Author Information
CHAO LI (Xidian University)
Shangqian Gao (University of Pittsburgh)
Cheng Deng (Xidian University)
De Xie (XiDian University)
Wei Liu (Tencent AI Lab)
More from the Same Authors
-
2022 Poster: RSA: Reducing Semantic Shift from Aggressive Augmentations for Self-supervised Learning »
Yingbin Bai · Erkun Yang · Zhaoqing Wang · Yuxuan Du · Bo Han · Cheng Deng · Dadong Wang · Tongliang Liu -
2023 Poster: Punctuation-level Attack: Single-shot and Single Punctuation Can Fool Text Models »
wenqiang wang · Chongyang Du · Tao Wang · Kaihao Zhang · Wenhan Luo · Lin Ma · Wei Liu · Xiaochun Cao -
2022 Spotlight: Lightning Talks 6A-4 »
Xiu-Shen Wei · Konstantina Dritsa · Guillaume Huguet · ABHRA CHAUDHURI · Zhenbin Wang · Kevin Qinghong Lin · Yutong Chen · Jianan Zhou · Yongsen Mao · Junwei Liang · Jinpeng Wang · Mao Ye · Yiming Zhang · Aikaterini Thoma · H.-Y. Xu · Daniel Sumner Magruder · Enwei Zhang · Jianing Zhu · Ronglai Zuo · Massimiliano Mancini · Hanxiao Jiang · Jun Zhang · Fangyun Wei · Faen Zhang · Ioannis Pavlopoulos · Zeynep Akata · Xiatian Zhu · Jingfeng ZHANG · Alexander Tong · Mattia Soldan · Chunhua Shen · Yuxin Peng · Liuhan Peng · Michael Wray · Tongliang Liu · Anjan Dutta · Yu Wu · Oluwadamilola Fasina · Panos Louridas · Angel Chang · Manik Kuchroo · Manolis Savva · Shujie LIU · Wei Zhou · Rui Yan · Gang Niu · Liang Tian · Bo Han · Eric Z. XU · Guy Wolf · Yingying Zhu · Brian Mak · Difei Gao · Masashi Sugiyama · Smita Krishnaswamy · Rong-Cheng Tu · Wenzhe Zhao · Weijie Kong · Chengfei Cai · WANG HongFa · Dima Damen · Bernard Ghanem · Wei Liu · Mike Zheng Shou -
2022 Spotlight: Egocentric Video-Language Pretraining »
Kevin Qinghong Lin · Jinpeng Wang · Mattia Soldan · Michael Wray · Rui Yan · Eric Z. XU · Difei Gao · Rong-Cheng Tu · Wenzhe Zhao · Weijie Kong · Chengfei Cai · WANG HongFa · Dima Damen · Bernard Ghanem · Wei Liu · Mike Zheng Shou -
2022 Spotlight: RSA: Reducing Semantic Shift from Aggressive Augmentations for Self-supervised Learning »
Yingbin Bai · Erkun Yang · Zhaoqing Wang · Yuxuan Du · Bo Han · Cheng Deng · Dadong Wang · Tongliang Liu -
2022 Poster: Egocentric Video-Language Pretraining »
Kevin Qinghong Lin · Jinpeng Wang · Mattia Soldan · Michael Wray · Rui Yan · Eric Z. XU · Difei Gao · Rong-Cheng Tu · Wenzhe Zhao · Weijie Kong · Chengfei Cai · WANG HongFa · Dima Damen · Bernard Ghanem · Wei Liu · Mike Zheng Shou -
2022 Poster: MetricFormer: A Unified Perspective of Correlation Exploring in Similarity Learning »
Jiexi Yan · Erkun Yang · Cheng Deng · Heng Huang -
2022 Poster: Enhanced Bilevel Optimization via Bregman Distance »
Feihu Huang · Junyi Li · Shangqian Gao · Heng Huang -
2022 Poster: Accelerated Zeroth-Order and First-Order Momentum Methods from Mini to Minimax Optimization »
Feihu Huang · Shangqian Gao · Jian Pei · Heng Huang -
2021 Poster: Neural Routing by Memory »
Kaipeng Zhang · Zhenqiang Li · Zhifeng Li · Wei Liu · Yoichi Sato -
2021 Poster: Generalized and Discriminative Few-Shot Object Detection via SVD-Dictionary Enhancement »
Aming WU · Suqi Zhao · Cheng Deng · Wei Liu -
2020 Poster: Towards Playing Full MOBA Games with Deep Reinforcement Learning »
Deheng Ye · Guibin Chen · Wen Zhang · Sheng Chen · Bo Yuan · Bo Liu · Jia Chen · Zhao Liu · Fuhao Qiu · Hongsheng Yu · Yinyuting Yin · Bei Shi · Liang Wang · Tengfei Shi · Qiang Fu · Wei Yang · Lanxiao Huang · Wei Liu -
2020 Poster: Fewer is More: A Deep Graph Metric Learning Perspective Using Fewer Proxies »
Yuehua Zhu · Muli Yang · Cheng Deng · Wei Liu -
2020 Poster: Optimal Epoch Stochastic Gradient Descent Ascent Methods for Min-Max Optimization »
Yan Yan · Yi Xu · Qihang Lin · Wei Liu · Tianbao Yang -
2020 Spotlight: Fewer is More: A Deep Graph Metric Learning Perspective Using Fewer Proxies »
Yuehua Zhu · Muli Yang · Cheng Deng · Wei Liu -
2020 Poster: Adversarial Learning for Robust Deep Clustering »
Xu Yang · Cheng Deng · Kun Wei · Junchi Yan · Wei Liu -
2019 Poster: Semantic Conditioned Dynamic Modulation for Temporal Sentence Grounding in Videos »
Yitian Yuan · Lin Ma · Jingwen Wang · Wei Liu · Wenwu Zhu -
2019 Poster: Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation »
Qiming ZHANG · Jing Zhang · Wei Liu · Dacheng Tao -
2018 Poster: Nonlocal Neural Networks, Nonlocal Diffusion and Nonlocal Modeling »
Yunzhe Tao · Qi Sun · Qiang Du · Wei Liu -
2018 Poster: Generalizing Graph Matching beyond Quadratic Assignment Model »
Tianshu Yu · Junchi Yan · Yilin Wang · Wei Liu · baoxin Li -
2018 Poster: Deep Non-Blind Deconvolution via Generalized Low-Rank Approximation »
Wenqi Ren · Jiawei Zhang · Lin Ma · Jinshan Pan · Xiaochun Cao · Wangmeng Zuo · Wei Liu · Ming-Hsuan Yang -
2018 Poster: Bilevel Distance Metric Learning for Robust Image Recognition »
Jie Xu · Lei Luo · Cheng Deng · Heng Huang -
2018 Poster: Distilled Wasserstein Learning for Word Embedding and Topic Modeling »
Hongteng Xu · Wenlin Wang · Wei Liu · Lawrence Carin -
2018 Poster: Parsimonious Quantile Regression of Financial Asset Tail Dynamics via Sequential Learning »
Xing Yan · Weizhong Zhang · Lin Ma · Wei Liu · Qi Wu -
2017 Poster: Geometric Descent Method for Convex Composite Minimization »
Shixiang Chen · Shiqian Ma · Wei Liu -
2017 Poster: Mixture-Rank Matrix Approximation for Collaborative Filtering »
Dongsheng Li · Chao Chen · Wei Liu · Tun Lu · Ning Gu · Stephen Chu -
2014 Poster: Discrete Graph Hashing »
Wei Liu · Cun Mu · Sanjiv Kumar · Shih-Fu Chang -
2014 Spotlight: Discrete Graph Hashing »
Wei Liu · Cun Mu · Sanjiv Kumar · Shih-Fu Chang -
2014 Poster: Zeta Hull Pursuits: Learning Nonconvex Data Hulls »
Yuanjun Xiong · Wei Liu · Deli Zhao · Xiaoou Tang