Timezone: »

 
Poster
Incentivizing Combinatorial Bandit Exploration
Xinyan Hu · Dung Ngo · Aleksandrs Slivkins · Steven Wu

Tue Nov 29 02:00 PM -- 04:00 PM (PST) @ Hall J #326
in Poster Session 2 »

Consider a bandit algorithm that recommends actions to self-interested users in a recommendation system. The users are free to choose other actions and need to be incentivized to follow the algorithm's recommendations. While the users prefer to exploit, the algorithm can incentivize them to explore by leveraging the information collected from the previous users. All published work on this problem, known as incentivized exploration, focuses on small, unstructured action sets and mainly targets the case when the users' beliefs are independent across actions. However, realistic exploration problems often feature large, structured action sets and highly correlated beliefs. We focus on a paradigmatic exploration problem with structure: combinatorial semi-bandits. We prove that Thompson Sampling, when applied to combinatorial semi-bandits, is incentive-compatible when initialized with a sufficient number of samples of each arm (where this number is determined in advance by the Bayesian prior). Moreover, we design incentive-compatible algorithms for collecting the initial samples.

Keywords: Multi-armed Bandits · thompson sampling · incentivized exploration · Bayesian incentive-compatibility · combinatorial semi-bandits
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Xinyan Hu (UC Berkeley)
Dung Ngo (University of Minnesota)
Aleksandrs Slivkins (Microsoft Research NYC)
Steven Wu (Carnegie Mellon University)
Steven Wu

I am an Assistant Professor in the School of Computer Science at Carnegie Mellon University. My broad research interests are in algorithms and machine learning. These days I am excited about: - Foundations of responsible AI, with emphasis on privacy and fairness considerations. - Interactive learning, including contextual bandits and reinforcement learning, and its interactions with causal inference and econometrics. - Economic aspects of machine learning, with a focus on learning in the presence of strategic agents.

More from the Same Authors