Timezone: »
We introduce the multi-dimensional Skellam mechanism, a discrete differential privacy mechanism based on the difference of two independent Poisson random variables. To quantify its privacy guarantees, we analyze the privacy loss distribution via a numerical evaluation and provide a sharp bound on the Rényi divergence between two shifted Skellam distributions. While useful in both centralized and distributed privacy applications, we investigate how it can be applied in the context of federated learning with secure aggregation under communication constraints. Our theoretical findings and extensive experimental evaluations demonstrate that the Skellam mechanism provides the same privacy-accuracy trade-offs as the continuous Gaussian mechanism, even when the precision is low. More importantly, Skellam is closed under summation and sampling from it only requires sampling from a Poisson distribution -- an efficient routine that ships with all machine learning and data analysis software packages. These features, along with its discrete nature and competitive privacy-accuracy trade-offs, make it an attractive practical alternative to the newly introduced discrete Gaussian mechanism.
Author Information
Naman Agarwal (Google)
I am a PhD student in the Computer Science Department at Princeton University. I am advised by Elad Hazan. I am interested in Optimization for Machine Learning with a focus on faster optimization methods for Deep Learning. I am also interested in Data Privacy and its interplay with Machine Learning. Previously, I graduated with a Masters of Science in Computer Science at the University of Illinois Urbana-Champaign. I was advised at UIUC by Prof. Alexandra Kolla
Peter Kairouz (Google)
Ziyu Liu (Carnegie Mellon University)
More from the Same Authors
-
2021 : Communication Efficient Federated Learning with Secure Aggregation and Differential Privacy »
Wei-Ning Chen · Christopher Choquette-Choo · Peter Kairouz -
2021 Poster: Pointwise Bounds for Distribution Estimation under Communication Constraints »
Wei-Ning Chen · Peter Kairouz · Ayfer Ozgur -
2020 Tutorial: (Track1) Federated Learning and Analytics: Industry Meets Academia Q&A »
Peter Kairouz · Brendan McMahan · Virginia Smith -
2020 Poster: Learning Implicit Credit Assignment for Cooperative Multi-Agent Reinforcement Learning »
Meng Zhou · Ziyu Liu · Pengwei Sui · Yixuan Li · Yuk Ying Chung