Timezone: »

Stochastic Multi-Armed Bandits with Control Variates
Arun Verma · Manjesh Kumar Hanawal

Thu Dec 09 12:30 AM -- 02:00 AM (PST) @ None #None

This paper studies a new variant of the stochastic multi-armed bandits problem where auxiliary information about the arm rewards is available in the form of control variates. In many applications like queuing and wireless networks, the arm rewards are functions of some exogenous variables. The mean values of these variables are known a priori from historical data and can be used as control variates. Leveraging the theory of control variates, we obtain mean estimates with smaller variance and tighter confidence bounds. We develop an improved upper confidence bound based algorithm named UCB-CV and characterize the regret bounds in terms of the correlation between rewards and control variates when they follow a multivariate normal distribution. We also extend UCB-CV to other distributions using resampling methods like Jackknifing and Splitting. Experiments on synthetic problem instances validate performance guarantees of the proposed algorithms.

Author Information

Arun Verma (National University of Singapore)

Postdoctoral Research Fellow at National University of Singapore

Manjesh Kumar Hanawal (IIT Bombay)

Manjesh K. Hanawal received the M.S. degree in ECE from the Indian Institute of Science, Bengaluru, India, in 2009, and the Ph.D. degree from INRIA, Sophia Antipolis, France, and the University of Avignon, Avignon, France, in 2013. He was a Scientist-B with the Center for Artificial Intelligence and Robotics, DRDO, Bengaluru, India. He was a Post-Doctoral Fellow with Boston University for two years. He is currently an Assistant Professor in industrial engineering and operations research with the Indian Institute of Technology Bombay, Mumbai, India. His research interests include performance evaluation, machine learning, and network economics. He is a recipient of the Inspire Faculty Award from DST and the Early Career Research Award from SERB, Govt. of India.

More from the Same Authors