Timezone: »
Collecting training data from untrusted sources exposes machine learning services to poisoning adversaries, who maliciously manipulate training data to degrade the model accuracy. When trained on offline datasets, poisoning adversaries have to inject the poisoned data in advance before training, and the order of feeding these poisoned batches into the model is stochastic. In contrast, practical systems are more usually trained/fine-tuned on sequentially captured real-time data, in which case poisoning adversaries could dynamically poison each data batch according to the current model state. In this paper, we focus on the real-time settings and propose a new attacking strategy, which affiliates an accumulative phase with poisoning attacks to secretly (i.e., without affecting accuracy) magnify the destructive effect of a (poisoned) trigger batch. By mimicking online learning and federated learning on MNIST and CIFAR-10, we show that model accuracy significantly drops by a single update step on the trigger batch after the accumulative phase. Our work validates that a well-designed but straightforward attacking strategy can dramatically amplify the poisoning effects, with no need to explore complex techniques.
Author Information
Tianyu Pang (Tsinghua University)
Xiao Yang (Tsinghua University)
Yinpeng Dong (Tsinghua University)
Hang Su (Tsinghua Univiersity)
Jun Zhu (Tsinghua University)
More from the Same Authors
-
2021 : Counter-Strike Deathmatch with Large-Scale Behavioural Cloning »
Tim Pearce · Jun Zhu -
2021 Poster: Stability and Generalization of Bilevel Programming in Hyperparameter Optimization »
Fan Bao · Guoqiang Wu · Chongxuan LI · Jun Zhu · Bo Zhang -
2021 Poster: On the Convergence of Prior-Guided Zeroth-Order Optimization Algorithms »
Shuyu Cheng · Guoqiang Wu · Jun Zhu -
2021 Poster: Scalable Quasi-Bayesian Inference for Instrumental Variable Regression »
Ziyu Wang · Yuhao Zhou · Tongzheng Ren · Jun Zhu -
2021 Poster: Rethinking and Reweighting the Univariate Losses for Multi-Label Ranking: Consistency and Generalization »
Guoqiang Wu · Chongxuan LI · Kun Xu · Jun Zhu -
2021 Poster: AFEC: Active Forgetting of Negative Transfer in Continual Learning »
Liyuan Wang · Mingtian Zhang · Zhongfan Jia · Qian Li · Chenglong Bao · Kaisheng Ma · Jun Zhu · Yi Zhong -
2020 Poster: Multi-label classification: do Hamming loss and subset accuracy really conflict with each other? »
Guoqiang Wu · Jun Zhu -
2020 Poster: Bi-level Score Matching for Learning Energy-based Latent Variable Models »
Fan Bao · Chongxuan LI · Kun Xu · Hang Su · Jun Zhu · Bo Zhang -
2020 Poster: Further Analysis of Outlier Detection with Deep Generative Models »
Ziyu Wang · Bin Dai · David P Wipf · Jun Zhu -
2020 Poster: Efficient Learning of Generative Models via Finite-Difference Score Matching »
Tianyu Pang · Kun Xu · Chongxuan LI · Yang Song · Stefano Ermon · Jun Zhu -
2020 Poster: Calibrated Reliable Regression using Maximum Mean Discrepancy »
Peng Cui · Wenbo Hu · Jun Zhu -
2020 Poster: Boosting Adversarial Training with Hypersphere Embedding »
Tianyu Pang · Xiao Yang · Yinpeng Dong · Kun Xu · Jun Zhu · Hang Su -
2020 Poster: Adversarial Distributional Training for Robust Deep Learning »
Yinpeng Dong · Zhijie Deng · Tianyu Pang · Jun Zhu · Hang Su -
2020 Poster: Understanding and Exploring the Network with Stochastic Architectures »
Zhijie Deng · Yinpeng Dong · Shifeng Zhang · Jun Zhu -
2019 Poster: Improving Black-box Adversarial Attacks with a Transfer-based Prior »
Shuyu Cheng · Yinpeng Dong · Tianyu Pang · Hang Su · Jun Zhu -
2019 Poster: Generative Well-intentioned Networks »
Justin Cosentino · Jun Zhu -
2019 Poster: Multi-objects Generation with Amortized Structural Regularization »
Kun Xu · Chongxuan LI · Jun Zhu · Bo Zhang -
2018 : Adversarial Vision Challenge: Black-box Attacks with Small Search Subspace and Covariance Matrix Adaptation of Perturbation Noise »
Shuyu Cheng · Yinpeng Dong -
2018 Poster: Towards Robust Detection of Adversarial Examples »
Tianyu Pang · Chao Du · Yinpeng Dong · Jun Zhu -
2018 Spotlight: Towards Robust Detection of Adversarial Examples »
Tianyu Pang · Chao Du · Yinpeng Dong · Jun Zhu -
2018 Poster: Graphical Generative Adversarial Networks »
Chongxuan LI · Max Welling · Jun Zhu · Bo Zhang -
2017 : Competition I: Adversarial Attacks and Defenses »
Alexey Kurakin · Ian Goodfellow · Samy Bengio · Yao Zhao · Yinpeng Dong · Tianyu Pang · Fangzhou Liao · Cihang Xie · Adithya Ganesh · Oguz Elibol -
2017 Poster: Triple Generative Adversarial Nets »
Chongxuan LI · Kun Xu · Jun Zhu · Bo Zhang -
2017 Poster: Population Matching Discrepancy and Applications in Deep Learning »
Jianfei Chen · Chongxuan LI · Yizhong Ru · Jun Zhu -
2016 Poster: Kernel Bayesian Inference with Posterior Regularization »
Yang Song · Jun Zhu · Yong Ren -
2016 Poster: Stochastic Gradient Geodesic MCMC Methods »
Chang Liu · Jun Zhu · Yang Song -
2016 Poster: Conditional Generative Moment-Matching Networks »
Yong Ren · Jun Zhu · Jialian Li · Yucen Luo -
2015 Poster: Max-Margin Majority Voting for Learning from Crowds »
TIAN TIAN · Jun Zhu -
2015 Poster: Max-Margin Deep Generative Models »
Chongxuan Li · Jun Zhu · Tim Shi · Bo Zhang -
2014 Poster: Distributed Bayesian Posterior Sampling via Moment Sharing »
Minjie Xu · Balaji Lakshminarayanan · Yee Whye Teh · Jun Zhu · Bo Zhang -
2014 Poster: Spectral Methods for Supervised Topic Models »
Yining Wang · Jun Zhu -
2014 Poster: Robust Bayesian Max-Margin Clustering »
Changyou Chen · Jun Zhu · Xinhua Zhang -
2013 Poster: Scalable Inference for Logistic-Normal Topic Models »
Jianfei Chen · Jun Zhu · Zi Wang · Xun Zheng · Bo Zhang -
2012 Poster: Monte Carlo Methods for Maximum Margin Supervised Topic Models »
Qixia Jiang · Jun Zhu · Maosong Sun · Eric Xing -
2012 Poster: Bayesian Nonparametric Maximum Margin Matrix Factorization for Collaborative Prediction »
Minjie Xu · Jun Zhu · Bo Zhang -
2011 Poster: Infinite Latent SVM for Classification and Multi-task Learning »
Jun Zhu · Ning Chen · Eric Xing -
2010 Poster: Large Margin Learning of Upstream Scene Understanding Models »
Jun Zhu · Li-Jia Li · Li Fei-Fei · Eric Xing -
2010 Poster: Predictive Subspace Learning for Multi-view Data: a Large Margin Approach »
Ning Chen · Jun Zhu · Eric Xing -
2010 Poster: Adaptive Multi-Task Lasso: with Application to eQTL Detection »
Seunghak Lee · Jun Zhu · Eric Xing -
2010 Poster: Efficient Relational Learning with Hidden Variable Detection »
Ni Lao · Jun Zhu · Liu Xinwang · Yandong Liu · William Cohen -
2008 Poster: Partially Observed Maximum Entropy Discrimination Markov Networks »
Jun Zhu · Eric Xing · Bo Zhang