Timezone: »

 
PAC^m-Bayes: Narrowing the Empirical Risk Gap in the Misspecified Bayesian Regime
Joshua Dillon · Warren Morningstar · Alexander Alemi

@
in Your Model is Wrong: Robustness and misspecification in probabilistic modeling »

The Bayesian posterior minimizes the "inferential risk" which itself bounds the "predictive risk." This bound is tight when the likelihood and prior are well-specified. However since misspecification induces a gap, the Bayesian posterior predictive distribution may have poor generalization performance. This work develops a multi-sample loss (PAC^m) which can close the gap by spanning a trade-off between the two risks. The loss is computationally favorable and offers PAC generalization guarantees. Empirical study demonstrates improvement to the predictive distribution.

Author Information

Joshua Dillon (Google Research)
Warren Morningstar (Google)

I am an AI resident at google, studying how to model uncertainty in Neural Networks. Before Google, I was an astrophysicist at the Kavli Institute for Particle Astrophysics and Cosmology at Stanford University, working on statistical modeling and machine learning applied to astronomical observations.

Alexander Alemi (Google)

More from the Same Authors