In many machine learning applications, data are collected by a large number of devices, calling for a distributed architecture for learning models. Federated learning (FL) aims to address this challenge by providing a decentralized mechanism for leveraging the individual data and computational power of users. Classical FL relies on a single shared model for users but tends to perform poorly in the presence of data and task heterogeneity across users.

This talk presents various approaches for developing multiple personalized” models for heterogeneous users. We first consider a meta-learning approach, where the goal is to generate an initial shared model that users adapt to their tasks using small number of additional local computations. Second, we consider a cluster-based approach which is more appropriate when there is substantial heterogeneity in user data distributions. We propose an algorithm that simultaneously learns cluster identities, while fully operating in a decentralized manner.