Timezone: »
Privacy-preserving federated learning allows multiple users to jointly train a model with coordination of a central server. The server only learns the final aggregation result, thereby preventing leakage of the users' (private) training data from the individual model updates. However, keeping the individual updates private allows malicious users to perform Byzantine attacks and degrade the model accuracy without being detected. Best existing defenses against Byzantine workers rely on robust rank-based statistics, e.g., the median, to find malicious updates. However, implementing privacy-preserving rank-based statistics is nontrivial and unscalable in the secure domain, as it requires sorting of all individual updates. We establish the first private robustness check that uses high break point rank-based statistics on aggregated model updates. By exploiting randomized clustering, we significantly improve the scalability of our defense without compromising privacy. We leverage the derived statistical bounds in zero-knowledge proofs to detect and remove malicious updates without revealing the private user updates. Our novel framework, zPROBE, enables Byzantine resilient and secure federated learning. Empirical evaluations demonstrate that zPROBE provides a low overhead solution to defend against state-of-the-art Byzantine attacks while preserving privacy.
Author Information
Zahra Ghodsi (University of California, San Diego)
Mojan Javaheripi (University of California San Diego)
I am a PhD student at UC San Diego working under supervision of Prof. Farinaz Koushanfar. My research lies at the intersection of machine learning algorithm and systems. I tackle challenges to enable hardware-aware and secure Deep Learning (DL). I have worked in the areas of efficient DL training and execution on constrained devices as well as adversarially robust DL models. I am the recipient of the 2019 Qualcomm Innovation Fellowship award. Prior to my PhD, I obtained my Bachelor's in Electrical Engineering majoring in digital system design. Skills: Deep Learning, AutoML, Computer Vision, Discrete and Continuous Optimization, Computer Architecture
Nojan Sheybani (University of California San Diego)
Xinqiao Zhang (University of California, San Diego, University of California, San Diego)
Ke Huang (San Diego State University)
Farinaz Koushanfar (William Marsh Rice University)
More from the Same Authors
-
2022 : FL-Talk: Covert Communication in Federated Learning via Spectral Steganography »
Huili Chen · Farinaz Koushanfar -
2022 : Contributed Talk: zPROBE: Zero Peek Robustness Checks for Federated Learning »
Zahra Ghodsi · Mojan Javaheripi · Nojan Sheybani · Xinqiao Zhang · Ke Huang · Farinaz Koushanfar -
2022 Spotlight: Lightning Talks 5B-2 »
Conglong Li · Mohammad Azizmalayeri · Mojan Javaheripi · Pratik Vaishnavi · Jon Hasselgren · Hao Lu · Kevin Eykholt · Arshia Soltani Moakhar · Wenze Liu · Gustavo de Rosa · Nikolai Hofmann · Minjia Zhang · Zixuan Ye · Jacob Munkberg · Amir Rahmati · Arman Zarei · Subhabrata Mukherjee · Yuxiong He · Shital Shah · Reihaneh Zohrabi · Hongtao Fu · Tomasz Religa · Yuliang Liu · Mohammad Manzuri · Mohammad Hossein Rohban · Zhiguo Cao · Caio Cesar Teodoro Mendes · Sebastien Bubeck · Farinaz Koushanfar · Debadeepta Dey -
2022 Spotlight: LiteTransformerSearch: Training-free Neural Architecture Search for Efficient Language Models »
Mojan Javaheripi · Gustavo de Rosa · Subhabrata Mukherjee · Shital Shah · Tomasz Religa · Caio Cesar Teodoro Mendes · Sebastien Bubeck · Farinaz Koushanfar · Debadeepta Dey -
2022 Poster: LiteTransformerSearch: Training-free Neural Architecture Search for Efficient Language Models »
Mojan Javaheripi · Gustavo de Rosa · Subhabrata Mukherjee · Shital Shah · Tomasz Religa · Caio Cesar Teodoro Mendes · Sebastien Bubeck · Farinaz Koushanfar · Debadeepta Dey -
2021 Poster: Circa: Stochastic ReLUs for Private Deep Learning »
Zahra Ghodsi · Nandan Kumar Jha · Brandon Reagen · Siddharth Garg