Timezone: »

 
Poster
Optimal Stochastic and Online Learning with Individual Iterates
Yunwen Lei · Peng Yang · Ke Tang · Ding-Xuan Zhou

Wed Dec 11 05:00 PM -- 07:00 PM (PST) @ East Exhibition Hall B + C #164
in Optimization -- Convex Optimization »

Stochastic composite mirror descent (SCMD) is a simple and efficient method able to capture both geometric and composite structures of optimization problems in machine learning. Existing strategies require to take either an average or a random selection of iterates to achieve optimal convergence rates, which, however, can either destroy the sparsity of solutions or slow down the practical training speed. In this paper, we propose a theoretically sound strategy to select an individual iterate of the vanilla SCMD, which is able to achieve optimal rates for both convex and strongly convex problems in a non-smooth learning setting. This strategy of outputting an individual iterate can preserve the sparsity of solutions which is crucial for a proper interpretation in sparse learning problems. We report experimental comparisons with several baseline methods to show the effectiveness of our method in achieving a fast training speed as well as in outputting sparse solutions.

Keywords: Learning Theory · Online Learning · Algorithms · Theory
[ Paper]  [ Poster] 

Author Information

Yunwen Lei (Technical University of Kaiserslautern)
Peng Yang (Southern University of Science and Technology)
Ke Tang (Southern University of Science and Technology)
Ding-Xuan Zhou (City University of Hong Kong)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors