Timezone: »

 
Diffusion Prior for Online Decision Making: A Case Study of Thompson Sampling
Yu-Guan Hsieh · Shiva Kasiviswanathan · Branislav Kveton · Patrick Blöbaum

@
in NeurIPS 2022 Workshop on Score-Based Methods »
Event URL: https://openreview.net/forum?id=sgmDSzt-H2 »

In this work, we investigate the possibility of using denoising diffusion models to learn priors for online decision making problems. Our special focus is on the meta-learning for bandit framework, with the goal of learning a strategy that performs well across bandit tasks of a same class. To this end, we train a diffusion model that learns the underlying task distribution and combine Thompson sampling with the learned prior to deal with new task at test time. Our posterior sampling algorithm is designed to carefully balance between the learned prior and the noisy observations that come from the learner's interaction with the environment. Preliminary experiments clearly demonstrate the potential of the considered approach.

Author Information

Yu-Guan Hsieh (Université Grenoble Alpes / Inria)
Shiva Kasiviswanathan (Amazon)
Branislav Kveton (Amazon)
Patrick Blöbaum (Amazon)

More from the Same Authors