When Decentralized Optimization Meets Federated Learning
Hongchang Gao My T. Thai Jie Wu
Federated learning is a new learning paradigm for extracting knowledge from
distributed data. Due to its favorable properties in preserving privacy and
saving communication costs, it has been extensively studied and widely applied
to numerous data analysis applications. However, most existing federated
learning approaches concentrate on the centralized setting, which is vulnerable
to a single-point failure. An alternative strategy for addressing this issue is
the decentralized communication topology. In this article, we systematically
investigate the challenges and opportunities when renovating decentralized
optimization for federated learning. In particular, we discussed them from the
model, data, and communication sides, respectively, which can deepen our
understanding about decentralized federated learning.