Timezone: »

 
Poster
PAC-Bayesian Theory Meets Bayesian Inference
Pascal Germain · Francis Bach · Alexandre Lacoste · Simon Lacoste-Julien

Mon Dec 05 09:00 AM -- 12:30 PM (PST) @ Area 5+6+7+8 #29
in Monday Posters »

We exhibit a strong link between frequentist PAC-Bayesian bounds and the Bayesian marginal likelihood. That is, for the negative log-likelihood loss function, we show that the minimization of PAC-Bayesian generalization bounds maximizes the Bayesian marginal likelihood. This provides an alternative explanation to the Bayesian Occam's razor criteria, under the assumption that the data is generated by an i.i.d. distribution. Moreover, as the negative log-likelihood is an unbounded loss function, we motivate and propose a PAC-Bayesian theorem tailored for the sub-gamma loss family, and we show that our approach is sound on classical Bayesian linear regression tasks.

Keywords: Learning Theory · (Other) Bayesian Inference

Author Information

Pascal Germain (Laval University)
Francis Bach (INRIA - Ecole Normale Superieure)
Alexandre Lacoste (Universite de Montreal)
Simon Lacoste-Julien (INRIA)

Simon Lacoste-Julien is an associate professor at Mila and DIRO from Université de Montréal, and Canada CIFAR AI Chair holder. He also heads part time the SAIT AI Lab Montreal from Samsung. His research interests are machine learning and applied math, with applications in related fields like computer vision and natural language processing. He obtained a B.Sc. in math., physics and computer science from McGill, a PhD in computer science from UC Berkeley and a post-doc from the University of Cambridge. He spent a few years as a research faculty at INRIA and École normale supérieure in Paris before coming back to his roots in Montreal in 2016 to answer the call from Yoshua Bengio in growing the Montreal AI ecosystem.

More from the Same Authors