Skip to yearly menu bar Skip to main content


Poster

On the Model Shrinkage Effect of Gamma Process Edge Partition Models

Iku Ohama · Issei Sato · Takuya Kida · Hiroki Arimura

Pacific Ballroom #187

Keywords: [ Network Analysis ] [ Matrix and Tensor Factorization ] [ Relational Learning ] [ Components Analysis (e.g., CCA, ICA, LDA, PCA) ] [ Latent Variable Models ] [ Hierarchical Models ] [ Unsupervised Learning ] [ Model Selection and Structure Learning ] [ MCMC ] [ Bayesian Nonparametrics ] [ Clustering ]


Abstract: The edge partition model (EPM) is a fundamental Bayesian nonparametric model for extracting an overlapping structure from binary matrix. The EPM adopts a gamma process ($\Gamma$P) prior to automatically shrink the number of active atoms. However, we empirically found that the model shrinkage of the EPM does not typically work appropriately and leads to an overfitted solution. An analysis of the expectation of the EPM's intensity function suggested that the gamma priors for the EPM hyperparameters disturb the model shrinkage effect of the internal $\Gamma$P. In order to ensure that the model shrinkage effect of the EPM works in an appropriate manner, we proposed two novel generative constructions of the EPM: CEPM incorporating constrained gamma priors, and DEPM incorporating Dirichlet priors instead of the gamma priors. Furthermore, all DEPM's model parameters including the infinite atoms of the $\Gamma$P prior could be marginalized out, and thus it was possible to derive a truly infinite DEPM (IDEPM) that can be efficiently inferred using a collapsed Gibbs sampler. We experimentally confirmed that the model shrinkage of the proposed models works well and that the IDEPM indicated state-of-the-art performance in generalization ability, link prediction accuracy, mixing efficiency, and convergence speed.

Live content is unavailable. Log in and register to view live content