Timezone: »

 
Poster
Multi-armed Bandits with Compensation
Siwei Wang · Longbo Huang

Thu Dec 06 02:00 PM -- 04:00 PM (PST) @ Room 517 AB #155
in Thu Poster Session B »
We propose and study the known-compensation multi-arm bandit (KCMAB) problem, where a system controller offers a set of arms to many short-term players for $T$ steps. In each step, one short-term player arrives to the system. Upon arrival, the player greedily selects an arm with the current best average reward and receives a stochastic reward associated with the arm. In order to incentivize players to explore other arms, the controller provides proper payment compensation to players. The objective of the controller is to maximize the total reward collected by players while minimizing the compensation. We first give a compensation lower bound $\Theta(\sum_i {\Delta_i\log T\over KL_i})$, where $\Delta_i$ and $KL_i$ are the expected reward gap and Kullback-Leibler (KL) divergence between distributions of arm $i$ and the best arm, respectively. We then analyze three algorithms to solve the KCMAB problem, and obtain their regrets and compensations. We show that the algorithms all achieve $O(\log T)$ regret and $O(\log T)$ compensation that match the theoretical lower bound. Finally, we use experiments to show the behaviors of those algorithms.
Keywords: Bandit Algorithms · Online Learning · Reinforcement Learning

Author Information

Siwei Wang (IIIS, Tsinghua University)
Longbo Huang (IIIS, Tsinghua Univeristy)

More from the Same Authors