Decentralized Personalized Online Federated Learning
Renzhi Wu Saayan Mitra Xiang Chen Anup Rao
Vanilla federated learning does not support learning in an online
environment, learning a personalized model on each client, and learning in a
decentralized setting. There are existing methods extending federated learning
in each of the three aspects. However, some important applications on
enterprise edge servers (e.g. online item recommendation at global scale)
involve the three aspects at the same time. Therefore, we propose a new
learning setting \textit{Decentralized Personalized Online Federated Learning}
that considers all the three aspects at the same time.
  In this new setting for learning, the first technical challenge is how to
aggregate the shared model parameters from neighboring clients to obtain a
personalized local model with good performance on each client. We propose to
directly learn an aggregation by optimizing the performance of the local model
with respect to the aggregation weights. This not only improves personalization
of each local model but also helps the local model adapting to potential data
shift by intelligently incorporating the right amount of information from its
neighbors. The second challenge is how to select the neighbors for each client.
We propose a peer selection method based on the learned aggregation weights
enabling each client to select the most helpful neighbors and reduce
communication cost at the same time. We verify the effectiveness and robustness
of our proposed method on three real-world item recommendation datasets and one
air quality prediction dataset.