Timezone: »

 
Poster
Pessimism for Offline Linear Contextual Bandits using $\ell_p$ Confidence Sets
Gene Li · Cong Ma · Nati Srebro

Wed Nov 30 09:00 AM -- 11:00 AM (PST) @ Hall J #823
in Poster Session 3 »
We present a family $\{\widehat{\pi}_p\}_{p\ge 1}$ of pessimistic learning rules for offline learning of linear contextual bandits, relying on confidence sets with respect to different $\ell_p$ norms, where $\widehat{\pi}_2$ corresponds to Bellman-consistent pessimism (BCP), while $\widehat{\pi}_\infty$ is a novel generalization of lower confidence bound (LCB) to the linear setting. We show that the novel $\widehat{\pi}_\infty$ learning rule is, in a sense, adaptively optimal, as it achieves the minimax performance (up to log factors) against all $\ell_q$-constrained problems, and as such it strictly dominates all other predictors in the family, including $\widehat{\pi}_2$.
Keywords: offline reinforcement learning · pessimism · linear contextual bandits
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Gene Li (Toyota Technological Institute at Chicago)
Cong Ma (University of California Berkeley)
Nati Srebro (TTI-Chicago)

More from the Same Authors