Timezone: »

 
Differentially Private Federated Learning with Normalized Updates
Rudrajit Das · Abolfazl Hashemi · Sujay Sanghavi · Inderjit Dhillon

@
in OPT 2022: Optimization for Machine Learning »
Event URL: https://openreview.net/forum?id=0FllaTqjor7 »

The customary approach for client-level differentially private federated learning (FL) is to add Gaussian noise to the average of the clipped client updates. Clipping is associated with the following issue: as the client updates fall below the clipping threshold, they get drowned out by the added noise, inhibiting convergence. To mitigate this issue, we propose replacing clipping with normalization, where we use only a scaled version of the unit vector along the client updates. Normalization ensures that the noise does not drown out the client updates even when the original updates are small. We theoretically show that the resulting normalization-based private FL algorithm attains better convergence than its clipping-based counterpart on convex objectives in over-parameterized settings.

Author Information

Rudrajit Das (University of Texas at Austin)
Abolfazl Hashemi (Purdue University)
Sujay Sanghavi (UT-Austin)
Inderjit Dhillon (Google & UT Austin)

More from the Same Authors