Timezone: »
Approximate Bayesian inference estimates descriptors of an intractable target distribution - in essence, an optimization problem within a family of distributions. For example, Langevin dynamics (LD) extracts asymptotically exact samples from a diffusion process because the time evolution of its marginal distributions constitutes a curve that minimizes the KL-divergence via steepest descent in the Wasserstein space. Parallel to LD, Stein variational gradient descent (SVGD) similarly minimizes the KL, albeit endowed with a novel Stein-Wasserstein distance, by deterministically transporting a set of particle samples, thus de-randomizes the stochastic diffusion process. We propose de-randomized kernel-based particle samplers to all diffusion-based samplers known as MCMC dynamics. Following previous work in interpreting MCMC dynamics, we equip the Stein-Wasserstein space with a fiber-Riemannian Poisson structure, with the capacity of characterizing a fiber-gradient Hamiltonian flow that simulates MCMC dynamics. Such dynamics discretizes into generalized SVGD (GSVGD), a Stein-type deterministic particle sampler, with particle updates coinciding with applying the diffusion Stein operator to a kernel function. We demonstrate empirically that GSVGD can de-randomize complex MCMC dynamics, which combine the advantages of auxiliary momentum variables and Riemannian structure, while maintaining the high sample quality from an interacting particle system.
Author Information
Zheyang Shen (Aalto University)
Markus Heinonen (Aalto University)
Samuel Kaski (Aalto University and University of Manchester)
More from the Same Authors
-
2022 : Modular Flows: Differential Molecular Generation »
Yogesh Verma · Samuel Kaski · Markus Heinonen · Vikas Garg -
2022 : Targeted Causal Elicitation »
Nazaal Ibrahim · ST John · Zhigao Guo · Samuel Kaski -
2022 : More trustworthy Bayesian optimization of materials properties by adding human into the loop »
Armi Tiihonen · Louis Filstroff · Petrus Mikkola · Emma Lehto · Samuel Kaski · Milica Todorović · Patrick Rinke -
2022 : Provably expressive temporal graph networks »
Amauri Souza · Diego Mesquita · Samuel Kaski · Vikas Garg -
2022 : Modular Flows: Differential Molecular Generation »
Yogesh Verma · Samuel Kaski · Markus Heinonen · Vikas Garg -
2022 : Differentiable User Models »
Alex Hämäläinen · Mustafa Mert Çelikok · Samuel Kaski -
2023 Poster: Practical Equivariances via Relational Conditional Neural Processes »
Daolang Huang · Manuel Haussmann · Ulpu Remes · ST John · Grégoire Clarté · Kevin Sebastian Luck · Samuel Kaski · Luigi Acerbi -
2023 Poster: Compositional Sculpting of Iterative Generative Processes »
Timur Garipov · Sebastiaan De Peuter · Ge Yang · Vikas Garg · Samuel Kaski · Tommi Jaakkola -
2023 Poster: Continuous-Time Functional Diffusion Processes »
Giulio Franzese · Giulio Corallo · Simone Rossi · Markus Heinonen · Maurizio Filippone · Pietro Michiardi -
2023 Poster: Learning Robust Statistics for Simulation-based Inference under Model Misspecification »
Daolang Huang · Ayush Bharti · Amauri Souza · Luigi Acerbi · Samuel Kaski -
2023 Poster: Learning Space-Time Continuous Latent Neural PDEs from Partially Observed States »
Valerii Iakovlev · Markus Heinonen · Harri Lähdesmäki -
2022 : Panel Discussion »
Cynthia Rudin · Dan Bohus · Brenna Argall · Alison Gopnik · Igor Mordatch · Samuel Kaski -
2022 : Collaborative AI for assisting virtual laboratories »
Samuel Kaski -
2022 : Noise-Aware Statistical Inference with Differentially Private Synthetic Data »
Ossi Räisä · Joonas Jälkö · Antti Honkela · Samuel Kaski -
2022 : HAPNEST: An efficient tool for generating large-scale genetics datasets from limited training data »
Sophie Wharrie · Zhiyu Yang · Vishnu Raj · Remo Monti · Rahul Gupta · Ying Wang · Alicia Martin · Luke O'Connor · Samuel Kaski · Pekka Marttinen · Pier Palamara · Christoph Lippert · Andrea Ganna -
2022 Poster: Modular Flows: Differential Molecular Generation »
Yogesh Verma · Samuel Kaski · Markus Heinonen · Vikas Garg -
2022 Poster: Deconfounded Representation Similarity for Comparison of Neural Networks »
Tianyu Cui · Yogesh Kumar · Pekka Marttinen · Samuel Kaski -
2022 Poster: Provably expressive temporal graph networks »
Amauri Souza · Diego Mesquita · Samuel Kaski · Vikas Garg -
2020 Poster: Rethinking pooling in graph neural networks »
Diego Mesquita · Amauri Souza · Samuel Kaski -
2019 Poster: Machine Teaching of Active Sequential Learners »
Tomi Peltola · Mustafa Mert Çelikok · Pedram Daee · Samuel Kaski -
2019 Poster: ODE2VAE: Deep generative second order ODEs with Bayesian neural networks »
Cagatay Yildiz · Markus Heinonen · Harri Lähdesmäki -
2017 Poster: Non-Stationary Spectral Kernels »
Sami Remes · Markus Heinonen · Samuel Kaski -
2017 Poster: Differentially private Bayesian learning on distributed data »
Mikko Heikkilä · Eemil Lagerspetz · Samuel Kaski · Kana Shimizu · Sasu Tarkoma · Antti Honkela -
2014 Workshop: Machine Learning in Computational Biology »
Oliver Stegle · Sara Mostafavi · Anna Goldenberg · Su-In Lee · Michael Leung · Anshul Kundaje · Mark B Gerstein · Martin Renqiang Min · Hannes Bretschneider · Francesco Paolo Casale · Loïc Schwaller · Amit G Deshwar · Benjamin A Logsdon · Yuanyang Zhang · Ali Punjani · Derek C Aguiar · Samuel Kaski