Timezone: »
Discrete mixtures are used routinely in broad sweeping applications ranging from unsupervised settings to fully supervised multi-task learning. Indeed, finite mixtures and infinite mixtures, relying on Dirichlet processes and modifications, have become a standard tool. One important issue that arises in using discrete mixtures is low separation in the components; in particular, different components can be introduced that are very similar and hence redundant. Such redundancy leads to too many clusters that are too similar, degrading performance in unsupervised learning and leading to computational problems and an unnecessarily complex model in supervised settings. Redundancy can arise in the absence of a penalty on components placed close together even when a Bayesian approach is used to learn the number of components. To solve this problem, we propose a novel prior that generates components from a repulsive process, automatically penalizing redundant components. We characterize this repulsive prior theoretically and propose a Markov chain Monte Carlo sampling algorithm for posterior computation. The methods are illustrated using synthetic examples and an iris data set.
Author Information
FRANCESCA PETRALIA (Mt Sinai School of Medicine)
Vinayak Rao (Purdue University)
David B Dunson (Duke University)
More from the Same Authors
-
2021 : Privacy-Aware Rejection Sampling »
Jordan Awan · Vinayak Rao -
2022 Poster: Data Augmentation MCMC for Bayesian Inference from Privatized Data »
Nianqiao Ju · Jordan Awan · Ruobin Gong · Vinayak Rao -
2022 Spotlight: Lightning Talks 1A-4 »
Siwei Wang · Jing Liu · Nianqiao Ju · Shiqian Li · Eloïse Berthier · Muhammad Faaiz Taufiq · Arsene Fansi Tchango · Chen Liang · Chulin Xie · Jordan Awan · Jean-Francois Ton · Ziad Kobeissi · Wenguan Wang · Xinwang Liu · Kewen Wu · Rishab Goel · Jiaxu Miao · Suyuan Liu · Julien Martel · Ruobin Gong · Francis Bach · Chi Zhang · Rob Cornish · Sanmi Koyejo · Zhi Wen · Yee Whye Teh · Yi Yang · Jiaqi Jin · Bo Li · Yixin Zhu · Vinayak Rao · Wenxuan Tu · Gaetan Marceau Caron · Arnaud Doucet · Xinzhong Zhu · Joumana Ghosn · En Zhu -
2022 Spotlight: Data Augmentation MCMC for Bayesian Inference from Privatized Data »
Nianqiao Ju · Jordan Awan · Ruobin Gong · Vinayak Rao -
2021 : Privacy-Aware Rejection Sampling »
Jordan Awan · Vinayak Rao -
2017 Poster: Collapsed variational Bayes for Markov jump processes »
Boqian Zhang · Jiangwei Pan · Vinayak Rao -
2016 Poster: DECOrrelated feature space partitioning for distributed sparse regression »
Xiangyu Wang · David B Dunson · Chenlei Leng -
2015 Poster: Parallelizing MCMC with Random Partition Trees »
Xiangyu Wang · Fangjian Guo · Katherine Heller · David B Dunson -
2015 Poster: On the consistency theory of high dimensional variable screening »
Xiangyu Wang · Chenlei Leng · David B Dunson -
2015 Poster: Probabilistic Curve Learning: Coulomb Repulsion and the Electrostatic Gaussian Process »
Ye Wang · David B Dunson -
2014 Poster: Median Selection Subset Aggregation for Parallel Inference »
Xiangyu Wang · Peichao Peng · David B Dunson -
2014 Oral: Median Selection Subset Aggregation for Parallel Inference »
Xiangyu Wang · Peichao Peng · David B Dunson -
2013 Poster: Locally Adaptive Bayesian Multivariate Time Series »
Daniele Durante · Bruno Scarpa · David B Dunson -
2013 Poster: Multiscale Dictionary Learning for Estimating Conditional Distributions »
Francesca Petralia · Joshua T Vogelstein · David B Dunson -
2012 Poster: Multiresolution Gaussian Processes »
Emily Fox · David B Dunson -
2012 Poster: MCMC for continuous-time discrete-state systems »
Vinayak Rao · Yee Whye Teh -
2011 Poster: Generalized Beta Mixtures of Gaussians »
Artin Armagan · David B Dunson · Merlise Clyde -
2011 Poster: The Kernel Beta Process »
Lu Ren · Yingjian Wang · David B Dunson · Lawrence Carin -
2011 Spotlight: The Kernel Beta Process »
Lu Ren · Yingjian Wang · David B Dunson · Lawrence Carin -
2011 Poster: Gaussian process modulated renewal processes »
Vinayak Rao · Yee Whye Teh -
2011 Poster: Hierarchical Topic Modeling for Analysis of Time-Evolving Personal Choices »
XianXing Zhang · David B Dunson · Lawrence Carin -
2010 Poster: Joint Analysis of Time-Evolving Binary Matrices and Associated Documents »
Eric X Wang · Dehong Liu · Jorge G Silva · David B Dunson · Lawrence Carin -
2009 Workshop: Nonparametric Bayes »
Dilan Gorur · Francois Caron · Yee Whye Teh · David B Dunson · Zoubin Ghahramani · Michael Jordan -
2009 Poster: A Bayesian Model for Simultaneous Image Clustering, Annotation and Object Segmentation »
Lan Du · Lu Ren · David B Dunson · Lawrence Carin -
2009 Poster: Spatial Normalized Gamma Processes »
Vinayak Rao · Yee Whye Teh -
2009 Spotlight: Spatial Normalized Gamma Processes »
Vinayak Rao · Yee Whye Teh -
2007 Spotlight: Retrieved context and the discovery of semantic structure »
Vinayak Rao · Marc Howard -
2007 Poster: Retrieved context and the discovery of semantic structure »
Vinayak Rao · Marc Howard