Timezone: »

 
FL Games: A Federated Learning Framework for Distribution Shifts
Sharut Gupta · Kartik Ahuja · Mohammad Havaei · Niladri Chatterjee · Yoshua Bengio

Fri Dec 02 11:30 AM -- 11:37 AM (PST) @
in Federated Learning: Recent Advances and New Challenges »
Event URL: https://openreview.net/forum?id=UyfdXCeelfy »

Federated learning aims to train predictive models for data that is distributed across clients, under the orchestration of a server. However, participating clients typically each hold data from a different distribution, which can yield to catastrophic generalization on data from a different client, which represents a new domain. In this work, we argue that in order to generalize better across non-i.i.d. clients, it is imperative only to learn correlations that are stable and invariant across domains. We propose FL Games, a game-theoretic framework for federated learning that learns causal features that are invariant across clients. While training to achieve the Nash equilibrium, the traditional best response strategy suffers from high-frequency oscillations. We demonstrate that FL Games effectively resolves this challenge and exhibits smooth performance curves. Further, FL Games scales well in the number of clients, requires significantly fewer communication rounds, and is agnostic to device heterogeneity. Through empirical evaluation, we demonstrate that \flgames achieves high out-of-distribution performance on various benchmarks.

Author Information

Sharut Gupta (Massachusetts Institute of Technology)
Sharut Gupta

I am a first-year Ph.D. student in the Machine Learning Group at CSAIL under the MIT Electrical Engineering and Computer Science (EECS) program. I am advised by Prof. Stefanie Jegelka. My research mainly focuses on building robust and generalizable machine learning systems with minimal supervision. I enjoy working on Out-of-Distribution (OOD) generalization, causal inference, federated learning, self-supervised learning, and representation learning

Kartik Ahuja (Mila)
Mohammad Havaei (Imagia)
Niladri Chatterjee
Yoshua Bengio (Mila / U. Montreal)

Yoshua Bengio is Full Professor in the computer science and operations research department at U. Montreal, scientific director and founder of Mila and of IVADO, Turing Award 2018 recipient, Canada Research Chair in Statistical Learning Algorithms, as well as a Canada AI CIFAR Chair. He pioneered deep learning and has been getting the most citations per day in 2018 among all computer scientists, worldwide. He is an officer of the Order of Canada, member of the Royal Society of Canada, was awarded the Killam Prize, the Marie-Victorin Prize and the Radio-Canada Scientist of the year in 2017, and he is a member of the NeurIPS advisory board and co-founder of the ICLR conference, as well as program director of the CIFAR program on Learning in Machines and Brains. His goal is to contribute to uncover the principles giving rise to intelligence through learning, as well as favour the development of AI for the benefit of all.

More from the Same Authors