Timezone: »
The recent, counter-intuitive discovery that deep generative models (DGMs) can frequently assign a higher likelihood to outliers has implications for both outlier detection applications as well as our overall understanding of generative modeling. In this work, we present a possible explanation for this phenomenon, starting from the observation that a model's typical set and high-density region may not conincide. From this vantage point we propose a novel outlier test, the empirical success of which suggests that the failure of existing likelihood-based outlier tests does not necessarily imply that the corresponding generative model is uncalibrated. We also conduct additional experiments to help disentangle the impact of low-level texture versus high-level semantics in differentiating outliers. In aggregate, these results suggest that modifications to the standard evaluation practices and benchmarks commonly applied in the literature are needed.
Author Information
Ziyu Wang (Tsinghua University)
Bin Dai (Samsung Research China - Beijing)
David P Wipf (AWS)
Jun Zhu (Tsinghua University)
More from the Same Authors
-
2021 Spotlight: On the Value of Infinite Gradients in Variational Autoencoder Models »
Bin Dai · Li Wenliang · David Wipf -
2021 : A Closer Look at Distribution Shifts and Out-of-Distribution Generalization on Graphs »
Mucong Ding · Kezhi Kong · Jiuhai Chen · John Kirchenbauer · Micah Goldblum · David P Wipf · Furong Huang · Tom Goldstein -
2021 : Quasi-Bayesian Dual Instrumental Variable Regression »
Ziyu Wang -
2021 : Counter-Strike Deathmatch with Large-Scale Behavioural Cloning »
Tim Pearce · Jun Zhu -
2021 : A Closer Look at Distribution Shifts and Out-of-Distribution Generalization on Graphs »
Mucong Ding · Kezhi Kong · Jiuhai Chen · John Kirchenbauer · Micah Goldblum · David P Wipf · Furong Huang · Tom Goldstein -
2021 Poster: Stability and Generalization of Bilevel Programming in Hyperparameter Optimization »
Fan Bao · Guoqiang Wu · Chongxuan LI · Jun Zhu · Bo Zhang -
2021 Poster: On the Convergence of Prior-Guided Zeroth-Order Optimization Algorithms »
Shuyu Cheng · Guoqiang Wu · Jun Zhu -
2021 Poster: Scalable Quasi-Bayesian Inference for Instrumental Variable Regression »
Ziyu Wang · Yuhao Zhou · Tongzheng Ren · Jun Zhu -
2021 Poster: Rethinking and Reweighting the Univariate Losses for Multi-Label Ranking: Consistency and Generalization »
Guoqiang Wu · Chongxuan LI · Kun Xu · Jun Zhu -
2021 Poster: AFEC: Active Forgetting of Negative Transfer in Continual Learning »
Liyuan Wang · Mingtian Zhang · Zhongfan Jia · Qian Li · Chenglong Bao · Kaisheng Ma · Jun Zhu · Yi Zhong -
2021 Poster: On the Value of Infinite Gradients in Variational Autoencoder Models »
Bin Dai · Li Wenliang · David Wipf -
2021 Poster: Accumulative Poisoning Attacks on Real-time Data »
Tianyu Pang · Xiao Yang · Yinpeng Dong · Hang Su · Jun Zhu -
2020 : Ziyu Wang---Further Analysis of Outlier Detection with Deep Generative Models »
Ziyu Wang -
2020 Poster: Multi-label classification: do Hamming loss and subset accuracy really conflict with each other? »
Guoqiang Wu · Jun Zhu -
2020 Poster: Bi-level Score Matching for Learning Energy-based Latent Variable Models »
Fan Bao · Chongxuan LI · Kun Xu · Hang Su · Jun Zhu · Bo Zhang -
2020 Poster: Efficient Learning of Generative Models via Finite-Difference Score Matching »
Tianyu Pang · Kun Xu · Chongxuan LI · Yang Song · Stefano Ermon · Jun Zhu -
2020 Poster: Calibrated Reliable Regression using Maximum Mean Discrepancy »
Peng Cui · Wenbo Hu · Jun Zhu -
2020 Poster: Boosting Adversarial Training with Hypersphere Embedding »
Tianyu Pang · Xiao Yang · Yinpeng Dong · Kun Xu · Jun Zhu · Hang Su -
2020 Poster: Adversarial Distributional Training for Robust Deep Learning »
Yinpeng Dong · Zhijie Deng · Tianyu Pang · Jun Zhu · Hang Su -
2020 Poster: Understanding and Exploring the Network with Stochastic Architectures »
Zhijie Deng · Yinpeng Dong · Shifeng Zhang · Jun Zhu -
2019 Poster: Improving Black-box Adversarial Attacks with a Transfer-based Prior »
Shuyu Cheng · Yinpeng Dong · Tianyu Pang · Hang Su · Jun Zhu -
2019 Poster: Generative Well-intentioned Networks »
Justin Cosentino · Jun Zhu -
2019 Poster: Multi-objects Generation with Amortized Structural Regularization »
Kun Xu · Chongxuan LI · Jun Zhu · Bo Zhang -
2018 Poster: Towards Robust Detection of Adversarial Examples »
Tianyu Pang · Chao Du · Yinpeng Dong · Jun Zhu -
2018 Spotlight: Towards Robust Detection of Adversarial Examples »
Tianyu Pang · Chao Du · Yinpeng Dong · Jun Zhu -
2018 Poster: Graphical Generative Adversarial Networks »
Chongxuan LI · Max Welling · Jun Zhu · Bo Zhang -
2017 Poster: Triple Generative Adversarial Nets »
Chongxuan LI · Kun Xu · Jun Zhu · Bo Zhang -
2017 Poster: Population Matching Discrepancy and Applications in Deep Learning »
Jianfei Chen · Chongxuan LI · Yizhong Ru · Jun Zhu -
2016 Poster: Kernel Bayesian Inference with Posterior Regularization »
Yang Song · Jun Zhu · Yong Ren -
2016 Poster: Stochastic Gradient Geodesic MCMC Methods »
Chang Liu · Jun Zhu · Yang Song -
2016 Poster: Conditional Generative Moment-Matching Networks »
Yong Ren · Jun Zhu · Jialian Li · Yucen Luo -
2015 Poster: Max-Margin Majority Voting for Learning from Crowds »
TIAN TIAN · Jun Zhu -
2015 Poster: Max-Margin Deep Generative Models »
Chongxuan Li · Jun Zhu · Tim Shi · Bo Zhang -
2014 Poster: Distributed Bayesian Posterior Sampling via Moment Sharing »
Minjie Xu · Balaji Lakshminarayanan · Yee Whye Teh · Jun Zhu · Bo Zhang -
2014 Poster: Spectral Methods for Supervised Topic Models »
Yining Wang · Jun Zhu -
2014 Poster: Robust Bayesian Max-Margin Clustering »
Changyou Chen · Jun Zhu · Xinhua Zhang -
2013 Poster: Scalable Inference for Logistic-Normal Topic Models »
Jianfei Chen · Jun Zhu · Zi Wang · Xun Zheng · Bo Zhang -
2012 Poster: Monte Carlo Methods for Maximum Margin Supervised Topic Models »
Qixia Jiang · Jun Zhu · Maosong Sun · Eric Xing -
2012 Poster: Dual-Space Analysis of the Sparse Linear Model »
David P Wipf -
2012 Poster: Bayesian Nonparametric Maximum Margin Matrix Factorization for Collaborative Prediction »
Minjie Xu · Jun Zhu · Bo Zhang -
2011 Poster: Sparse Estimation with Structured Dictionaries »
David P Wipf -
2011 Spotlight: Sparse Estimation with Structured Dictionaries »
David P Wipf -
2011 Poster: Infinite Latent SVM for Classification and Multi-task Learning »
Jun Zhu · Ning Chen · Eric Xing -
2010 Poster: Large Margin Learning of Upstream Scene Understanding Models »
Jun Zhu · Li-Jia Li · Li Fei-Fei · Eric Xing -
2010 Poster: Predictive Subspace Learning for Multi-view Data: a Large Margin Approach »
Ning Chen · Jun Zhu · Eric Xing -
2010 Poster: Adaptive Multi-Task Lasso: with Application to eQTL Detection »
Seunghak Lee · Jun Zhu · Eric Xing -
2010 Poster: Efficient Relational Learning with Hidden Variable Detection »
Ni Lao · Jun Zhu · Liu Xinwang · Yandong Liu · William Cohen -
2009 Poster: Sparse Estimation Using General Likelihoods and Non-Factorial Priors »
David P Wipf · Sri Nagarajan -
2008 Poster: Estimating the Location and Orientation of Complex, Correlated Neural Activity using MEG »
David P Wipf · Julia Owen · Hagai Attias · Kensuke Sekihara · Sri Nagarajan -
2008 Spotlight: Estimating the Location and Orientation of Complex, Correlated Neural Activity using MEG »
David P Wipf · Julia Owen · Hagai Attias · Kensuke Sekihara · Sri Nagarajan -
2008 Poster: Partially Observed Maximum Entropy Discrimination Markov Networks »
Jun Zhu · Eric Xing · Bo Zhang -
2007 Poster: A New View of Automatic Relevance Determination »
David P Wipf · Srikantan Nagarajan -
2006 Poster: Analysis of Empirical Bayesian Methods for Neuroelectromagnetic Source Localization »
David P Wipf · Rey R Ramirez · Jason A Palmer · Scott Makeig · Bhaskar Rao -
2006 Spotlight: Analysis of Empirical Bayesian Methods for Neuroelectromagnetic Source Localization »
David P Wipf · Rey R Ramirez · Jason A Palmer · Scott Makeig · Bhaskar Rao