Timezone: »

FedDR – Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimization
Quoc Tran Dinh · Nhan H Pham · Dzung Phan · Lam Nguyen

Thu Dec 09 04:30 PM -- 06:00 PM (PST) @

We develop two new algorithms, called, FedDR and asyncFedDR, for solving a fundamental nonconvex composite optimization problem in federated learning. Our algorithms rely on a novel combination between a nonconvex Douglas-Rachford splitting method, randomized block-coordinate strategies, and asynchronous im- plementation. They can also handle convex regularizers. Unlike recent methods in the literature, e.g., FedSplit and FedPD, our algorithms update only a subset of users at each communication round, and possibly in an asynchronous manner, making them more practical. These new algorithms can handle statistical and sys- tem heterogeneity, which are the two main challenges in federated learning, while achieving the best known communication complexity. In fact, our new algorithms match the communication complexity lower bound up to a constant factor under standard assumptions. Our numerical experiments illustrate the advantages of our methods over existing algorithms on synthetic and real datasets.

Author Information

Quoc Tran Dinh (Department of Statistics and Operations Research, University of North Carolina at Chapel Hill, North Carolina)
Nhan H Pham (University of North Carolina at Chapel Hill)

I started my PhD in Operations Research in Department of Statistics and Operations Research at University of North Carolina at Chapel Hill in 2017. I am currently working on stochastic methods for non-convex optimization with applications in machine learning, deep learning, and reinforcement learning under supervision by Dr. Quoc Tran-Dinh. In addition, I am also collaborating with Dr. Lam M. Nguyen and Dr. Dzung T. Phan at IBM Thomas J. Watson Research Center. I come from Vietnam where I had my bachelor in Computer Engineering from Deparment of Computer Science and Engineering, Ho Chi Minh City University of Technology (Bach Khoa University). During my undergrad, I was a member of BKIT Hardware Club and participated in the Vietnam Robot Contest under BK4/BKIT Number One team in 2013. My hobbies are travelling with my wife and exploring new places.

Dzung Phan (IBM Research, T. J. Watson Research Center)
Lam Nguyen (IBM Research, Thomas J. Watson Research Center)

More from the Same Authors