Timezone: »
We develop two new algorithms, called, FedDR and asyncFedDR, for solving a fundamental nonconvex composite optimization problem in federated learning. Our algorithms rely on a novel combination between a nonconvex Douglas-Rachford splitting method, randomized block-coordinate strategies, and asynchronous im- plementation. They can also handle convex regularizers. Unlike recent methods in the literature, e.g., FedSplit and FedPD, our algorithms update only a subset of users at each communication round, and possibly in an asynchronous manner, making them more practical. These new algorithms can handle statistical and sys- tem heterogeneity, which are the two main challenges in federated learning, while achieving the best known communication complexity. In fact, our new algorithms match the communication complexity lower bound up to a constant factor under standard assumptions. Our numerical experiments illustrate the advantages of our methods over existing algorithms on synthetic and real datasets.
Author Information
Quoc Tran Dinh (Department of Statistics and Operations Research, University of North Carolina at Chapel Hill, North Carolina)
Nhan H Pham (University of North Carolina at Chapel Hill)
I started my PhD in Operations Research in Department of Statistics and Operations Research at University of North Carolina at Chapel Hill in 2017. I am currently working on stochastic methods for non-convex optimization with applications in machine learning, deep learning, and reinforcement learning under supervision by Dr. Quoc Tran-Dinh. In addition, I am also collaborating with Dr. Lam M. Nguyen and Dr. Dzung T. Phan at IBM Thomas J. Watson Research Center. I come from Vietnam where I had my bachelor in Computer Engineering from Deparment of Computer Science and Engineering, Ho Chi Minh City University of Technology (Bach Khoa University). During my undergrad, I was a member of BKIT Hardware Club and participated in the Vietnam Robot Contest under BK4/BKIT Number One team in 2013. My hobbies are travelling with my wife and exploring new places.
Dzung Phan (IBM Research, T. J. Watson Research Center)
Lam Nguyen (IBM Research, Thomas J. Watson Research Center)
More from the Same Authors
-
2022 : c-MBA: Adversarial Attack for Cooperative MARL Using Learned Dynamics Model »
Nhan H Pham · Lam Nguyen · Jie Chen · Thanh Lam Hoang · Subhro Das · Lily Weng -
2021 Workshop: New Frontiers in Federated Learning: Privacy, Fairness, Robustness, Personalization and Data Ownership »
Nghia Hoang · Lam Nguyen · Pin-Yu Chen · Tsui-Wei Weng · Sara Magliacane · Bryan Kian Hsiang Low · Anoop Deoras -
2021 Poster: On the Equivalence between Neural Network and Support Vector Machine »
Yilan Chen · Wei Huang · Lam Nguyen · Tsui-Wei Weng -
2021 Poster: Ensembling Graph Predictions for AMR Parsing »
Thanh Lam Hoang · Gabriele Picco · Yufang Hou · Young-Suk Lee · Lam Nguyen · Dzung Phan · Vanessa Lopez · Ramon Fernandez Astudillo -
2021 Poster: Cardinality-Regularized Hawkes-Granger Model »
Tsuyoshi Ide · Georgios Kollias · Dzung Phan · Naoki Abe -
2020 Poster: Hybrid Variance-Reduced SGD Algorithms For Minimax Problems with Nonconvex-Linear Function »
Quoc Tran Dinh · Deyi Liu · Lam Nguyen -
2020 Poster: A Scalable MIP-based Method for Learning Optimal Multivariate Decision Trees »
Haoran Zhu · Pavankumar Murali · Dzung Phan · Lam Nguyen · Jayant Kalagnanam -
2019 Poster: Tight Dimension Independent Lower Bound on the Expected Convergence Rate for Diminishing Step Sizes in SGD »
Ha Nguyen · Lam Nguyen · Marten van Dijk -
2018 Poster: Non-Ergodic Alternating Proximal Augmented Lagrangian Algorithms with Optimal Rates »
Quoc Tran Dinh -
2017 Poster: Smooth Primal-Dual Coordinate Descent Algorithms for Nonsmooth Convex Optimization »
Ahmet Alacaoglu · Quoc Tran Dinh · Olivier Fercoq · Volkan Cevher -
2015 Poster: A Universal Primal-Dual Convex Optimization Framework »
Alp Yurtsever · Quoc Tran Dinh · Volkan Cevher