Timezone: »

 
Poster
A Stochastic Composite Gradient Method with Incremental Variance Reduction
Junyu Zhang · Lin Xiao

Wed Dec 11 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #158
in Optimization -- Stochastic Optimization »

We consider the problem of minimizing the composition of a smooth (nonconvex) function and a smooth vector mapping, where the inner mapping is in the form of an expectation over some random variable or a finite sum. We propose a stochastic composite gradient method that employs incremental variance-reduced estimators for both the inner vector mapping and its Jacobian. We show that this method achieves the same orders of complexity as the best known first-order methods for minimizing expected-value and finite-sum nonconvex functions, despite the additional outer composition which renders the composite gradient estimator biased. This finding enables a much broader range of applications in machine learning to benefit from the low complexity of incremental variance-reduction methods.

Keywords: Optimization · Stochastic Optimization · Algorithms -> Stochastic Methods; Optimization -> Non-Convex Optimization; Optimization
[ Paper]  [ Poster] 

Author Information

Junyu Zhang (University of Minnesota)
Lin Xiao (Microsoft Research)

More from the Same Authors