Timezone: »

 
Poster
A Comprehensive Linear Speedup Analysis for Asynchronous Stochastic Parallel Optimization from Zeroth-Order to First-Order
Xiangru Lian · Huan Zhang · Cho-Jui Hsieh · Yijun Huang · Ji Liu

Mon Dec 05 09:00 AM -- 12:30 PM (PST) @ Area 5+6+7+8 #133
in Monday Posters »

Asynchronous parallel optimization received substantial successes and extensive attention recently. One of core theoretical questions is how much speedup (or benefit) the asynchronous parallelization can bring to us. This paper provides a comprehensive and generic analysis to study the speedup property for a broad range of asynchronous parallel stochastic algorithms from the zeroth order to the first order methods. Our result recovers or improves existing analysis on special cases, provides more insights for understanding the asynchronous parallel behaviors, and suggests a novel asynchronous parallel zeroth order method for the first time. Our experiments provide novel applications of the proposed asynchronous parallel zeroth order method on hyper parameter tuning and model blending problems.

Keywords: Convex Optimization

Author Information

Xiangru Lian (University of Rochester)
Huan Zhang (CMU)
Cho-Jui Hsieh (UCLA)
Yijun Huang (University of Rochester)
Ji Liu (Kwai Inc.)

More from the Same Authors