Timezone: »

 
Poster
Combinatorial Multi-Armed Bandit with General Reward Functions
Wei Chen · Wei Hu · Fu Li · Jian Li · Yu Liu · Pinyan Lu

Tue Dec 06 09:00 AM -- 12:30 PM (PST) @ Area 5+6+7+8 #13
in Tuesday Posters »
In this paper, we study the stochastic combinatorial multi-armed bandit (CMAB) framework that allows a general nonlinear reward function, whose expected value may not depend only on the means of the input random variables but possibly on the entire distributions of these variables. Our framework enables a much larger class of reward functions such as the $\max()$ function and nonlinear utility functions. Existing techniques relying on accurate estimations of the means of random variables, such as the upper confidence bound (UCB) technique, do not work directly on these functions. We propose a new algorithm called stochastically dominant confidence bound (SDCB), which estimates the distributions of underlying random variables and their stochastically dominant confidence bounds. We prove that SDCB can achieve $O(\log T)$ distribution-dependent regret and $\tilde{O}(\sqrt{T})$ distribution-independent regret, where $T$ is the time horizon. We apply our results to the $K$-MAX problem and expected utility maximization problems. In particular, for $K$-MAX, we provide the first polynomial-time approximation scheme (PTAS) for its offline problem, and give the first $\tilde{O}(\sqrt T)$ bound on the $(1-\epsilon)$-approximation regret of its online problem, for any $\epsilon>0$.
Keywords: Bandit Algorithms · Online Learning · Combinatorial Optimization
[ PDF] 

Author Information

Wei Chen (Microsoft Research)
Wei Hu (Princeton University)
Fu Li (The University of Texas at Austin)
Jian Li (Tsinghua University)
Yu Liu (Tsinghua University)
Pinyan Lu (Shanghai University of Finance and Economics)

More from the Same Authors