Timezone: »
The robustness of federated learning (FL) is vital for the distributed training of an accurate global model that is shared among large number of clients. The collaborative learning framework by typically aggregating model updates is vulnerable to model poisoning attacks from adversarial clients. Since the shared information between the global server and participants are only limited to model parameters, it is challenging to detect bad model updates. Moreover, real-world datasets are usually heterogeneous and not independent and identically distributed (Non-IID) among participants, which makes the design of such robust FL pipeline more difficult. In this work, we propose a novel robust aggregation method, Federated Robust Adaptive Distillation (FedRAD), to detect adversaries and robustly aggregate local models based on properties of the median statistic, and then performing an adapted version of ensemble Knowledge Distillation. We run extensive experiments to evaluate the proposed method against recently published works. The results show that FedRAD outperforms all other aggregators in the presence of adversaries, as well as in heterogeneous data distributions.
Author Information
Stefán Sturluson (Imperial College London)
Luis Muñoz-González (Imperial College London)
Matei George Nicolae Grama (Imperial College London)
Jonathan Passerat-Palmbach (Imperial College London / ConsenSys Health)
Daniel Rueckert (Imperial College London)
Amir Alansary (Imperial College London)
More from the Same Authors
-
2021 : An automatic differentiation system for the age of differential privacy »
Dmitrii Usynin · Alexander Ziller · Moritz Knolle · Daniel Rueckert · Georgios Kaissis -
2021 : Feature-level privacy loss modelling in differentially private machine learning »
Dmitrii Usynin · Alexander Ziller · Moritz Knolle · Daniel Rueckert · Georgios Kaissis -
2021 : Contribution Evaluation in Federated Learning: Examining Current Approaches »
Jonathan Passerat-Palmbach · Vasilis Siomos -
2021 : Modelling Patient Journeys with Sharded Encoder Blocks and Federated Split Learning »
Jonathan Passerat-Palmbach · Francesca Anna-Sophia Beer -
2018 : Poster Session »
Phillipp Schoppmann · Patrick Yu · Valerie Chen · Travis Dick · Marc Joye · Ningshan Zhang · Frederik Harder · Olli Saarikivi · Théo Ryffel · Yunhui Long · Théo JOURDAN · Di Wang · Antonio Marcedone · Negev Shekel Nosatzki · Yatharth A Dubey · Antti Koskela · Peter Bloem · Aleksandra Korolova · Martin Bertran · Hao Chen · Galen Andrew · Natalia Martinez · Janardhan Kulkarni · Jonathan Passerat-Palmbach · Guillermo Sapiro · Amrita Roy Chowdhury -
2017 : Deep learning for image reconstruction, segmentation and super-resolution in medical imaging (Imperial College London) »
Daniel Rueckert