Timezone: »
The most common approach for personalized federated learning is fine-tuning the global machine learning model to each client. While this addresses some issues of statistical diversity, we find that such personalization methods are vulnerable to spurious features, leading to bias and sacrificing generalization. Nevertheless, debiasing the personalized models is difficult. To this end, we propose a strategy to mitigate the effect of spurious features based on an observation that the global model in the federated learning step has a low bias degree due to statistical diversity. Then, we estimate and mitigate the bias degree difference between the personalized and global models using adversarial transferability in the personalization step. We theoretically establish the connection between the adversarial transferability and the bias degree difference between the global and personalized models. Empirical results on MNIST, CelebA, and Coil20 datasets show that our method improves the accuracy of the personalized model on the bias-conflicting data samples by up to 14.3%, compared to existing personalization approaches, while preserving the benefit of enhanced average accuracy from fine-tuning.
Author Information
Xiaoyang Wang (University of Illinois at Urbana-Champaign)
Han Zhao (University of Illinois at Urbana-Champaign)
Klara Nahrstedt (University of Illinois at Urbana-Champaign)
Sanmi Koyejo (University of Illinois at Urbana-Champaign & Google Research)

Sanmi Koyejo is an Assistant Professor in the Department of Computer Science at the University of Illinois at Urbana-Champaign and a research scientist at Google AI in Accra. Koyejo's research interests are in developing the principles and practice of adaptive and robust machine learning. Additionally, Koyejo focuses on applications to biomedical imaging and neuroscience. Koyejo co-founded the Black in AI organization and currently serves on its board.
More from the Same Authors
-
2021 : Probabilistic Performance Metric Elicitation »
Zachary Robertson · Hantao Zhang · Sanmi Koyejo -
2021 : RVFR: Robust Vertical Federated Learning via Feature Subspace Recovery »
Jing Liu · Chulin Xie · Krishnaram Kenthapadi · Sanmi Koyejo · Bo Li -
2021 : Secure Byzantine-Robust Distributed Learning via Clustering »
Raj Kiriti Velicheti · Sanmi Koyejo -
2021 : Exploiting Causal Chains for Domain Generalization »
Olawale Salaudeen · Sanmi Koyejo -
2021 : Distribution Preserving Bayesian Coresets using Set Constraints »
Shovik Guha · Rajiv Khanna · Sanmi Koyejo -
2020 Poster: Trade-offs and Guarantees of Adversarial Representation Learning for Information Obfuscation »
Han Zhao · Jianfeng Chi · Yuan Tian · Geoffrey Gordon -
2020 Poster: Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift »
Remi Tachet des Combes · Han Zhao · Yu-Xiang Wang · Geoffrey Gordon -
2020 Poster: Model-based Policy Optimization with Unsupervised Model Adaptation »
Jian Shen · Han Zhao · Weinan Zhang · Yong Yu -
2020 Spotlight: Model-based Policy Optimization with Unsupervised Model Adaptation »
Jian Shen · Han Zhao · Weinan Zhang · Yong Yu -
2020 Poster: Neural Methods for Point-wise Dependency Estimation »
Yao-Hung Hubert Tsai · Han Zhao · Makoto Yamada · Louis-Philippe Morency · Russ Salakhutdinov -
2020 Spotlight: Neural Methods for Point-wise Dependency Estimation »
Yao-Hung Hubert Tsai · Han Zhao · Makoto Yamada · Louis-Philippe Morency · Russ Salakhutdinov -
2019 Tutorial: Representation Learning and Fairness »
Moustapha Cisse · Sanmi Koyejo