Timezone: »

 
Poster
Breaking the Moments Condition Barrier: No-Regret Algorithm for Bandits with Super Heavy-Tailed Payoffs
Han Zhong · Jiayi Huang · Lin Yang · Liwei Wang

Thu Dec 09 12:30 AM -- 02:00 AM (PST) @
in Poster Session 5 »
Event URL: https://github.com/jyhuang78/mom »
Despite a large amount of effort in dealing with heavy-tailed error in machine learning, little is known when moments of the error can become non-existential: the random noise $\eta$ satisfies Pr$\left[|\eta| > |y|\right] \le 1/|y|^{\alpha}$ for some $\alpha > 0$. We make the first attempt to actively handle such super heavy-tailed noise in bandit learning problems: We propose a novel robust statistical estimator, mean of medians, which estimates a random variable by computing the empirical mean of a sequence of empirical medians. We then present a generic reductionist algorithmic framework for solving bandit learning problems (including multi-armed and linear bandit problem): the mean of medians estimator can be applied to nearly any bandit learning algorithm as a black-box filtering for its reward signals and obtain similar regret bound as if the reward is sub-Gaussian. We show that the regret bound is near-optimal even with very heavy-tailed noise. We also empirically demonstrate the effectiveness of the proposed algorithm, which further corroborates our theoretical results.
Keywords: Machine Learning · Bandits

Author Information

Han Zhong (Peking University)
Jiayi Huang (Peking University)
Lin Yang (UCLA)
Liwei Wang (Peking University)

More from the Same Authors