Timezone: »

 
PerFedSI: A Framework for Personalized Federated Learning with Side Information
Liam Collins · Enmao Diao · Tanya Roosta · Jie Ding · Tao Zhang

@
in Federated Learning: Recent Advances and New Challenges »
Event URL: https://openreview.net/forum?id=HRZjvFkX-faD »

With an ever-increasing number of smart edge devices with computation and communication constraints, Federated Learning (FL) is a promising paradigm for learning from distributed devices and their data. Typical approaches to FL aim to learn a single model that simultaneously performs well for all clients. But such an approach may be ineffective when the clients' data distributions are heterogeneous. In these cases, we aim to learn personalized models for each client's data yet still leverage shared information across clients. A critical avenue that may allow for such personalization is the presence of client-specific side information available to each client, such as client embeddings obtained from domain-specific knowledge, pre-trained models, or simply one-hot encodings. In this work, we propose a new FL framework for utilizing a general form of client-specific side information for personalized federated learning. We prove that incorporating side information can improve model performance for simplified multi-task linear regression and matrix completion problems. Further, we validate these results with image classification experiments on Omniglot, CIFAR-10, and CIFAR-100, revealing that proper use of side information can be beneficial for personalization.

Author Information

Liam Collins (The University of Texas at Austin)
Enmao Diao (Duke University)
Enmao Diao

I am a fourth-year Ph.D. candidate advised by Prof. Vahid Tarokh in Electrical Engineering at Duke University, Durham, North Carolina, USA. I was born in Chengdu, Sichuan, China in 1994. I received the B.S. degree in Computer Science and Electrical Engineering from Georgia Institute of Technology, Georgia, USA, in 2016 and the M.S. degree in Electrical Engineering from Harvard University, Cambridge, USA, in 2018.

Tanya Roosta (Amazon)
Jie Ding (University of Minnesota)
Tao Zhang (Alexa AI)

More from the Same Authors