Timezone: »

 
Poster
Greedy Model Averaging
Dong Dai · Tong Zhang

Mon Dec 12 10:00 AM -- 02:59 PM (PST) @
in Poster Session and Reception »
This paper considers the problem of combining multiple models to achieve a prediction accuracy not much worse than that of the best single model for least squares regression. It is known that if the models are mis-specified, model averaging is superior to model selection. Specifically, let $n$ be the sample size, then the worst case regret of the former decays at the rate of $O(1/n)$ while the worst case regret of the latter decays at the rate of $O(1/\sqrt{n})$. In the literature, the most important and widely studied model averaging method that achieves the optimal $O(1/n)$ average regret is the exponential weighted model averaging (EWMA) algorithm. However this method suffers from several limitations. The purpose of this paper is to present a new greedy model averaging procedure that improves EWMA. We prove strong theoretical guarantees for the new procedure and illustrate our theoretical results with empirical examples.
[ PDF] 

Author Information

Dong Dai (Rutgers University)
Tong Zhang (The Hong Kong University of Science and Technology)

More from the Same Authors