Timezone: »

 
Poster
Differentiable Meta-Learning of Bandit Policies
Craig Boutilier · Chih-wei Hsu · Branislav Kveton · Martin Mladenov · Csaba Szepesvari · Manzil Zaheer

Tue Dec 08 09:00 PM -- 11:00 PM (PST) @ Poster Session 2 #588
in Poster Session 3 »

Exploration policies in Bayesian bandits maximize the average reward over problem instances drawn from some distribution P. In this work, we learn such policies for an unknown distribution P using samples from P. Our approach is a form of meta-learning and exploits properties of P without making strong assumptions about its form. To do this, we parameterize our policies in a differentiable way and optimize them by policy gradients, an approach that is pleasantly general and easy to implement. We derive effective gradient estimators and propose novel variance reduction techniques. We also analyze and experiment with various bandit policy classes, including neural networks and a novel softmax policy. The latter has regret guarantees and is a natural starting point for our optimization. Our experiments show the versatility of our approach. We also observe that neural network policies can learn implicit biases expressed only through the sampled instances.

Author Information

Craig Boutilier (Google)
Chih-wei Hsu ( Google Research)
Branislav Kveton (Google Research)
Martin Mladenov (Google)
Csaba Szepesvari (DeepMind / University of Alberta)
Manzil Zaheer (Google)

More from the Same Authors