Concept-aware clustering for decentralized deep learning under temporal
  shift
Marcus Toftås Emilie Klefbom Edvin Listo Zec Martin Willbo Olof Mogren
Decentralized deep learning requires dealing with non-iid data across
clients, which may also change over time due to temporal shifts. While non-iid
data has been extensively studied in distributed settings, temporal shifts have
received no attention. To the best of our knowledge, we are first with tackling
the novel and challenging problem of decentralized learning with non-iid and
dynamic data. We propose a novel algorithm that can automatically discover and
adapt to the evolving concepts in the network, without any prior knowledge or
estimation of the number of concepts. We evaluate our algorithm on standard
benchmark datasets and demonstrate that it outperforms previous methods for
decentralized learning.