Timezone: »
Abstract Statistical tasks such as density estimation and approximate Bayesian inference often involve densities with unknown normalising constants. Score-based methods, including score matching, are popular techniques as they are free of normalising constants. Although these methods enjoy theoretical guarantees, a little-known fact is that they suffer from practical failure modes when the unnormalised distribution of interest has isolated components --- they cannot discover isolated components or identify the correct mixing proportions between components. We demonstrate these findings using simple distributions and present heuristic attempts to address these issues. We hope to bring the attention of theoreticians and practitioners to these issues when developing new algorithms and applications.
Author Information
Li Kevin Wenliang (Independent)
Heishiro Kanagawa (Gatsby Unit, University College London)
More from the Same Authors
-
2021 Poster: Deep Proxy Causal Learning and its Application to Confounded Bandit Policy Evaluation »
Liyuan Xu · Heishiro Kanagawa · Arthur Gretton -
2020 Poster: COT-GAN: Generating Sequential Data via Causal Optimal Transport »
Tianlin Xu · Li Kevin Wenliang · Michael Munn · Beatrice Acciaio -
2019 Poster: A neurally plausible model for online recognition and postdiction in a dynamical environment »
Li Kevin Wenliang · Maneesh Sahani -
2018 Poster: Informative Features for Model Comparison »
Wittawat Jitkrittum · Heishiro Kanagawa · Patsorn Sangkloy · James Hays · Bernhard Schölkopf · Arthur Gretton -
2016 Poster: Minimax Optimal Alternating Minimization for Kernel Nonparametric Tensor Learning »
Taiji Suzuki · Heishiro Kanagawa · Hayato Kobayashi · Nobuyuki Shimizu · Yukihiro Tagami