Timezone: »
We study the problem of estimating at a central server the mean of a set of vectors distributed across several nodes (one vector per node). When the vectors are high-dimensional, the communication cost of sending entire vectors may be prohibitive, and it may be imperative for them to use sparsification techniques. While most existing work on sparsified mean estimation is agnostic to the characteristics of the data vectors, in many practical applications such as federated learning, there may be spatial correlations (similarities in the vectors sent by different nodes) or temporal correlations (similarities in the data sent by a single node over different iterations of the algorithm) in the data vectors. We leverage these correlations by simply modifying the decoding method used by the server to estimate the mean. We provide an analysis of the resulting estimation error as well as experiments for PCA, K-Means and Logistic Regression, which show that our estimators consistently outperform more sophisticated and expensive sparsification methods.
Author Information
Divyansh Jhunjhunwala (Carnegie Mellon University)
Ankur Mallick (CMU, Carnegie Mellon University)
Advait Gadhikar (Carnegie Mellon University)
Swanand Kadhe (University of California Berkeley)
Gauri Joshi (Carnegie Mellon University)
More from the Same Authors
-
2022 : Federated Learning under Distributed Concept Drift »
Ellango Jothimurugesan · Kevin Hsieh · Jianyu Wang · Gauri Joshi · Phillip Gibbons -
2023 Poster: Correlation Aware Distributed Vector Mean Estimation »
Shuli Jiang · PRANAY SHARMA · Gauri Joshi -
2023 Workshop: Workshop on Federated Learning in the Age of Foundation Models in Conjunction with NeurIPS 2023 (FL@FM-NeurIPS'23) »
Jinghui Chen · Lixin Fan · Gauri Joshi · Sai Praneeth Karimireddy · Stacy Patterson · Shiqiang Wang · Han Yu -
2022 : To Federate or Not To Federate: Incentivizing Client Participation in Federated Learning »
Yae Jee Cho · Divyansh Jhunjhunwala · Tian Li · Virginia Smith · Gauri Joshi -
2022 Workshop: Federated Learning: Recent Advances and New Challenges »
Shiqiang Wang · Nathalie Baracaldo · Olivia Choudhury · Gauri Joshi · Peter Richtarik · Praneeth Vepakomma · Han Yu -
2020 Poster: Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization »
Jianyu Wang · Qinghua Liu · Hao Liang · Gauri Joshi · H. Vincent Poor