Beyond Noise: Privacy-Preserving Decentralized Learning with Virtual
  Nodes
Sayan Biswas Mathieu Even Anne-Marie Kermarrec Laurent Massoulie Rafael Pires Rishi Sharma Martijn de Vos
Decentralized learning (DL) enables collaborative learning without a server
and without training data leaving the users' devices. However, the models
shared in DL can still be used to infer training data. Conventional privacy
defenses such as differential privacy and secure aggregation fall short in
effectively safeguarding user privacy in DL. We introduce Shatter, a novel DL
approach in which nodes create virtual nodes (VNs) to disseminate chunks of
their full model on their behalf. This enhances privacy by (i) preventing
attackers from collecting full models from other nodes, and (ii) hiding the
identity of the original node that produced a given model chunk. We
theoretically prove the convergence of Shatter and provide a formal analysis
demonstrating how Shatter reduces the efficacy of attacks compared to when
exchanging full models between participating nodes. We evaluate the convergence
and attack resilience of Shatter with existing DL algorithms, with
heterogeneous datasets, and against three standard privacy attacks, including
gradient inversion. Our evaluation shows that Shatter not only renders these
privacy attacks infeasible when each node operates 16 VNs but also exhibits a
positive impact on model convergence compared to standard DL. This enhanced
privacy comes with a manageable increase in communication volume.