Revisiting Gradient Pruning: A Dual Realization for Defending against
  Gradient Attacks
Lulu Xue Shengshan Hu Ruizhi Zhao Leo Yu Zhang Shengqing Hu Lichao Sun Dezhong Yao
Collaborative learning (CL) is a distributed learning framework that aims to
protect user privacy by allowing users to jointly train a model by sharing
their gradient updates only. However, gradient inversion attacks (GIAs), which
recover users' training data from shared gradients, impose severe privacy
threats to CL. Existing defense methods adopt different techniques, e.g.,
differential privacy, cryptography, and perturbation defenses, to defend
against the GIAs. Nevertheless, all current defense methods suffer from a poor
trade-off between privacy, utility, and efficiency. To mitigate the weaknesses
of existing solutions, we propose a novel defense method, Dual Gradient Pruning
(DGP), based on gradient pruning, which can improve communication efficiency
while preserving the utility and privacy of CL. Specifically, DGP slightly
changes gradient pruning with a stronger privacy guarantee. And DGP can also
significantly improve communication efficiency with a theoretical analysis of
its convergence and generalization. Our extensive experiments show that DGP can
effectively defend against the most powerful GIAs and reduce the communication
cost without sacrificing the model's utility.