Timezone: »

 
Stochastic Pruning: Fine-Tuning, and PAC-Bayes bound optimization
Soufiane Hayou · Bobby He · Gintare Karolina Dziugaite

@
in Bayesian Deep Learning »
Event URL: https://openreview.net/forum?id=at_dDR2u6K »

We introduce an algorithmic framework for stochastic fine-tuning of pruning masks, starting from masks produced by several baselines. We further show that by minimizing a PAC-Bayes bound with data-dependent priors, we obtain a self-bounded learning algorithm with numerically tight bounds. In the linear model, we show that a PAC-Bayes generalization error bound is controlled by the magnitude of the change in feature alignment between the prior'' andposterior'' data.

Author Information

Soufiane Hayou (National University of Singapore)

PTA Assistant Professor of Mathematics at NUS

Bobby He (University of Oxford)
Gintare Karolina Dziugaite (Google Research, Brain Team)

More from the Same Authors