Low-Cost Privacy-Aware Decentralized Learning
Sayan Biswas Davide Frey Romaric Gaudel Anne-Marie Kermarrec Dimitri Lerévérend Rafael Pires Rishi Sharma François Taïani
This paper introduces ZIP-DL, a novel privacy-aware decentralized learning
(DL) algorithm that relies on adding correlated noise to each model update
during the model training process. This technique ensures that the added noise
almost neutralizes itself during the aggregation process due to its
correlation, thus minimizing the impact on model accuracy. In addition, ZIP-DL
does not require multiple communication rounds for noise cancellation,
addressing the common trade-off between privacy protection and communication
overhead. We provide theoretical guarantees for both convergence speed and
privacy guarantees, thereby making ZIP-DL applicable to practical scenarios.
Our extensive experimental study shows that ZIP-DL achieves the best trade-off
between vulnerability and accuracy. In particular, ZIP-DL (i) reduces the
effectiveness of a linkability attack by up to 52 points compared to baseline
DL, and (ii) achieves up to 37 more accuracy points for the same vulnerability
under membership inference attacks against a privacy-preserving competitor