Timezone: »

What Do We Mean by Generalization in Federated Learning?
Honglin Yuan · Warren Morningstar · Lin Ning

Federated learning data is drawn from a distribution of distributions: clients are drawn from a meta-distribution, and their data are drawn from personal data distributions. Thus generalization studies in federated learning should separate performance gaps from unseen client data (out-of-sample gap) from performance gaps from unseen client distributions (participation gap). In this work, we propose a framework for disentangling these performance gaps. Using this framework we observe and explain differences in behavior across natural and synthetic federated datasets, indicating that dataset synthesis strategy can be important for realistic simulations of generalization in federated learning. We propose a semantic synthesis strategy that enables realistic simulation without naturally-partitioned data.

Author Information

Honglin Yuan (Stanford University)
Warren Morningstar (Google)

I am an AI resident at google, studying how to model uncertainty in Neural Networks. Before Google, I was an astrophysicist at the Kavli Institute for Particle Astrophysics and Cosmology at Stanford University, working on statistical modeling and machine learning applied to astronomical observations.

Lin Ning (Google Research)

More from the Same Authors