Timezone: »

 
Poster
Maximizing and Satisficing in Multi-armed Bandits with Graph Information
Parth Thaker · Mohit Malu · Nikhil Rao · Gautam Dasarathy

Wed Nov 30 02:00 PM -- 04:00 PM (PST) @ Hall J #324
in Poster Session 4 »

Pure exploration in multi-armed bandits has emerged as an important framework for modeling decision making and search under uncertainty. In modern applications however, one is often faced with a tremendously large number of options and even obtaining one observation per option may be too costly rendering traditional pure exploration algorithms ineffective. Fortunately, one often has access to similarity relationships amongst the options that can be leveraged. In this paper, we consider the pure exploration problem in stochastic multi-armed bandits where the similarities between the arms is captured by a graph and the rewards may be represented as a smooth signal on this graph. In particular, we consider the problem of finding the arm with the maximum reward (i.e., the maximizing problem) or one that has sufficiently high reward (i.e., the satisficing problem) under this model. We propose novel algorithms GRUB (GRaph based UcB) and zeta-GRUB for these problems and provide theoretical characterization of their performance which specifically elicits the benefit of the graph side information. We also prove a lower bound on the data requirement that shows a large class of problems where these algorithms are near-optimal. We complement our theory with experimental results that show the benefit of capitalizing on such side information.

Keywords: Sample Complexity · Pure Exploration · Multi-armed Bandits · Graph Smoothness · Best Arm Identification

Author Information

Parth Thaker (Arizona State university)
Mohit Malu (Arizona State University)
Nikhil Rao (Microsoft)
Gautam Dasarathy (Arizona State University)

More from the Same Authors