Timezone: »

SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
Aaron Defazio · Francis Bach · Simon Lacoste-Julien

Mon Dec 08 04:00 PM -- 08:59 PM (PST) @ Level 2, room 210D

In this work we introduce a new fast incremental gradient method SAGA, in the spirit of SAG, SDCA, MISO and SVRG. SAGA improves on the theory behind SAG and SVRG, with better theoretical convergence rates, and support for composite objectives where a proximal operator is used on the regulariser. Unlike SDCA, SAGA supports non-strongly convex problems directly, and is adaptive to any inherent strong convexity of the problem. We give experimental results showing the effectiveness of our method.

Author Information

Aaron Defazio (Ambiata)
Francis Bach (INRIA - Ecole Normale Superieure)
Simon Lacoste-Julien (Mila, Université de Montréal & SAIL Montreal)

More from the Same Authors