Privacy-Preserving Distributed Optimization and Learning
Ziqin Chen Yongqiang Wang
Distributed optimization and learning has recently garnered great attention
due to its wide applications in sensor networks, smart grids, machine learning,
and so forth. Despite rapid development, existing distributed optimization and
learning algorithms require each agent to exchange messages with its neighbors,
which may expose sensitive information and raise significant privacy concerns.
In this survey paper, we overview privacy-preserving distributed optimization
and learning methods. We first discuss cryptography, differential privacy, and
other techniques that can be used for privacy preservation and indicate their
pros and cons for privacy protection in distributed optimization and learning.
We believe that among these approaches, differential privacy is most promising
due to its low computational and communication complexities, which are
extremely appealing for modern learning based applications with high dimensions
of optimization variables. We then introduce several differential-privacy
algorithms that can simultaneously ensure privacy and optimization accuracy.
Moreover, we provide example applications in several machine learning problems
to confirm the real-world effectiveness of these algorithms. Finally, we
highlight some challenges in this research domain and discuss future
directions.