Timezone: »

 
Poster
Sequential Test for the Lowest Mean: From Thompson to Murphy Sampling
Emilie Kaufmann · Wouter Koolen · Aurélien Garivier

Thu Dec 06 02:00 PM -- 04:00 PM (PST) @ Room 517 AB #147
in Thu Poster Session B »

Learning the minimum/maximum mean among a finite set of distributions is a fundamental sub-problem in planning, game tree search and reinforcement learning. We formalize this learning task as the problem of sequentially testing how the minimum mean among a finite set of distributions compares to a given threshold. We develop refined non-asymptotic lower bounds, which show that optimality mandates very different sampling behavior for a low vs high true minimum. We show that Thompson Sampling and the intuitive Lower Confidence Bounds policy each nail only one of these cases. We develop a novel approach that we call Murphy Sampling. Even though it entertains exclusively low true minima, we prove that MS is optimal for both possibilities. We then design advanced self-normalized deviation inequalities, fueling more aggressive stopping rules. We complement our theoretical guarantees by experiments showing that MS works best in practice.

Keywords: Bandit Algorithms
[ Paper]  [ Poster] 

Author Information

Emilie Kaufmann (CNRS & CRIStAL (SequeL))
Wouter Koolen (Centrum Wiskunde & Informatica, Amsterdam)
Aurélien Garivier (ENS Lyon)

More from the Same Authors