Timezone: »

 
A Unified Framework to Understand Decentralized and Federated Optimization Algorithms: A Multi-Rate Feedback Control Perspective
xinwei zhang · Mingyi Hong · Nicola Elia

@
in New Frontiers in Federated Learning: Privacy, Fairness, Robustness, Personalization and Data Ownership »

We propose a unified framework to analyze and design distributed optimization algorithms. Through the lens of multi-rate feedback control, we show that a wide class of distributed algorithms, including popular decentralized/federated schemes such as decentralized gradient descent, gradient tracking, and federated averaging, among others, can be viewed as discretizing a continuous-time feedback control system, but with different discretization patterns and/or multiple sampling rates. This key observation not only allows us to develop a generic framework to analyze the convergence of the entire algorithm class, more importantly, it leads to a new way of designing new distributed algorithms. We develop the theory behind our framework, and provide an example to highlight how the framework can be used to analyze and extend the well-known gradient tracking algorithm.

Author Information

xinwei zhang (University of Minnesota)
Mingyi Hong (University of Minnesota)
Nicola Elia (University of Minnesota)

More from the Same Authors