Timezone: »

Stochastic Pruning: Fine-Tuning, and PAC-Bayes bound optimization
Soufiane Hayou · Bobby He · Gintare Karolina Dziugaite

We introduce an algorithmic framework for stochastic fine-tuning of pruning masks, starting from masks produced by several baselines. We further show that by minimizing a PAC-Bayes bound with data-dependent priors, we obtain a self-bounded learning algorithm with numerically tight bounds. In the linear model, we show that a PAC-Bayes generalization error bound is controlled by the magnitude of the change in feature alignment between the prior'' andposterior'' data.

#### Author Information

##### Soufiane Hayou (National University of Singapore)

PTA Assistant Professor of Mathematics at NUS