Timezone: »
Poster
Quantile Propagation for Wasserstein-Approximate Gaussian Processes
Rui Zhang · Christian Walder · Edwin Bonilla · Marian-Andrei Rizoiu · Lexing Xie
Approximate inference techniques are the cornerstone of probabilistic methods based on Gaussian process priors. Despite this, most work approximately optimizes standard divergence measures such as the Kullback-Leibler (KL) divergence, which lack the basic desiderata for the task at hand, while chiefly offering merely technical convenience. We develop a new approximate inference method for Gaussian process models which overcomes the technical challenges arising from abandoning these convenient divergences. Our method---dubbed Quantile Propagation (QP)---is similar to expectation propagation (EP) but minimizes the $L_2$ Wasserstein distance (WD) instead of the KL divergence. The WD exhibits all the required properties of a distance metric, while respecting the geometry of the underlying sample space. We show that QP matches quantile functions rather than moments as in EP and has the same mean update but a smaller variance update than EP, thereby alleviating EP's tendency to over-estimate posterior variances. Crucially, despite the significant complexity of dealing with the WD, QP has the same favorable locality property as EP, and thereby admits an efficient algorithm. Experiments on classification and Poisson regression show that QP outperforms both EP and variational Bayes.
Author Information
Rui Zhang (The Australian National University)
Christian Walder (DATA61)
Edwin Bonilla (CSIRO's Data61)
Marian-Andrei Rizoiu (University of Technology Sydney)
Lexing Xie (Australian National University)
More from the Same Authors
-
2021 : Factorized Fourier Neural Operators »
Alasdair Tran · Alexander Mathews · Lexing Xie · Cheng Soon Ong -
2022 Poster: Fair Wrapping for Black-box Predictions »
Alexander Soen · Ibrahim Alabdulmohsin · Sanmi Koyejo · Yishay Mansour · Nyalleng Moorosi · Richard Nock · Ke Sun · Lexing Xie -
2021 Poster: Model Selection for Bayesian Autoencoders »
Ba-Hien Tran · Simone Rossi · Dimitrios Milios · Pietro Michiardi · Edwin Bonilla · Maurizio Filippone -
2021 Poster: TacticZero: Learning to Prove Theorems from Scratch with Deep Reinforcement Learning »
Minchao Wu · Michael Norrish · Christian Walder · Amir Dezfouli -
2020 : Bayesian optimization by density ratio estimation »
Louis Tiao · Aaron Klein · Cedric Archambeau · Edwin Bonilla · Matthias W Seeger · Fabio Ramos -
2020 Poster: All your loss are belong to Bayes »
Christian Walder · Richard Nock -
2020 Poster: Variational Inference for Graph Convolutional Networks in the Absence of Graph Data and Adversarial Settings »
Pantelis Elinas · Edwin Bonilla · Louis Tiao -
2020 Spotlight: Variational Inference for Graph Convolutional Networks in the Absence of Graph Data and Adversarial Settings »
Pantelis Elinas · Edwin Bonilla · Louis Tiao -
2019 : Outstanding Contribution Talk: Variational Graph Convolutional Networks »
Edwin Bonilla -
2019 Poster: Structured Variational Inference in Continuous Cox Process Models »
Virginia Aglietti · Edwin Bonilla · Theodoros Damoulas · Sally Cripps -
2016 : Chord2Vec: Learning Musical Chord Embeddings »
Christian Walder -
2015 Poster: Scalable Inference for Gaussian Process Models with Black-Box Likelihoods »
Amir Dezfouli · Edwin Bonilla -
2014 Poster: Extended and Unscented Gaussian Processes »
Daniel M Steinberg · Edwin Bonilla -
2014 Spotlight: Extended and Unscented Gaussian Processes »
Daniel M Steinberg · Edwin Bonilla -
2014 Poster: Automated Variational Inference for Gaussian Process Models »
Trung V Nguyen · Edwin Bonilla -
2013 Workshop: Machine Learning for Sustainability »
Edwin Bonilla · Thomas Dietterich · Theodoros Damoulas · Andreas Krause · Daniel Sheldon · Iadine Chades · J. Zico Kolter · Bistra Dilkina · Carla Gomes · Hugo P Simao -
2011 Poster: Improving Topic Coherence with Regularized Topic Models »
David Newman · Edwin Bonilla · Wray Buntine -
2010 Poster: Gaussian Process Preference Elicitation »
Edwin Bonilla · Shengbo Guo · Scott Sanner -
2007 Poster: Multi-task Gaussian Process Prediction »
Edwin Bonilla · Kian Ming A Chai · Chris Williams -
2007 Spotlight: Multi-task Gaussian Process Prediction »
Edwin Bonilla · Kian Ming A Chai · Chris Williams