Timezone: »

 
Poster
A Stochastic Newton Algorithm for Distributed Convex Optimization
Brian Bullins · Kshitij Patel · Ohad Shamir · Nathan Srebro · Blake Woodworth

Wed Dec 08 04:30 PM -- 06:00 PM (PST) @
in Poster Session 4 »

We propose and analyze a stochastic Newton algorithm for homogeneous distributed stochastic convex optimization, where each machine can calculate stochastic gradients of the same population objective, as well as stochastic Hessian-vector products (products of an independent unbiased estimator of the Hessian of the population objective with arbitrary vectors), with many such stochastic computations performed between rounds of communication. We show that our method can reduce the number, and frequency, of required communication rounds, compared to existing methods without hurting performance, by proving convergence guarantees for quasi-self-concordant objectives (e.g., logistic regression), alongside empirical evidence.

Keywords: Optimization · Federated Learning

Author Information

Brian Bullins (Princeton University)
Kshitij Patel (Toyota Technological Institute at Chicago)
Ohad Shamir (Weizmann Institute of Science)
Nathan Srebro (University of Toronto)
Blake Woodworth (Inria)

More from the Same Authors