Timezone: »

 
Tackling Distribution Shifts in Federated Learning with Superquantile Aggregation
Krishna Pillutla · Yassine Laguel · Jérôme Malick · Zaid Harchaoui

@
in Workshop on Distribution Shifts: Connecting Methods and Applications »
Event URL: https://openreview.net/forum?id=r2MgyybnfO »

Federated learning has emerged as the predominant framework for distributed machine learning over decentralized data, e.g. in mobile phones. The usual approaches suffer from a distribution shift: the model is trained to fit the average population distribution but is deployed on individual clients, whose data distributions can be quite different. We present a distributionally robust approach to federated learning based on a risk measure known as the superquantile and show how to optimize it by interleaving federated averaging steps with quantile computation. We demonstrate experimentally that our approach is competitive with usual ones in terms of average error and outperforms them in terms of tail statistics of the error.

Author Information

Krishna Pillutla (Google Research)
Yassine Laguel (Rutgers University, Newark)
Jérôme Malick (CNRS and LJK)
Zaid Harchaoui (University of Washington)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors