Timezone: »

 
Poster
Posterior Refinement Improves Sample Efficiency in Bayesian Neural Networks
Agustinus Kristiadi · Runa Eschenhagen · Philipp Hennig

Wed Nov 30 09:00 AM -- 11:00 AM (PST) @ Hall J #717
in Poster Session 3 »

Monte Carlo (MC) integration is the de facto method for approximating the predictive distribution of Bayesian neural networks (BNNs). But, even with many MC samples, Gaussian-based BNNs could still yield bad predictive performance due to the posterior approximation's error. Meanwhile, alternatives to MC integration are expensive. In this work, we experimentally show that the key to good MC-approximated predictive distributions is the quality of the approximate posterior itself. However, previous methods for obtaining accurate posterior approximations are expensive and non-trivial to implement. We, therefore, propose to refine Gaussian approximate posteriors with normalizing flows. When applied to last-layer BNNs, it yields a simple, cost-efficient, post hoc method for improving pre-existing parametric approximations. We show that the resulting posterior approximation is competitive with even the gold-standard full-batch Hamiltonian Monte Carlo.

Keywords: Bayesian Neural Networks · normalizing flows · predictive calibration

Author Information

Agustinus Kristiadi (University of Tübingen)
Runa Eschenhagen (University of Tübingen)
Philipp Hennig (University of Tuebingen)

More from the Same Authors