AEDFL: Efficient Asynchronous Decentralized Federated Learning with
  Heterogeneous Devices
Ji Liu Tianshi Che Yang Zhou Ruoming Jin Huaiyu Dai Dejing Dou Patrick Valduriez
Federated Learning (FL) has achieved significant achievements recently,
enabling collaborative model training on distributed data over edge devices.
Iterative gradient or model exchanges between devices and the centralized
server in the standard FL paradigm suffer from severe efficiency bottlenecks on
the server. While enabling collaborative training without a central server,
existing decentralized FL approaches either focus on the synchronous mechanism
that deteriorates FL convergence or ignore device staleness with an
asynchronous mechanism, resulting in inferior FL accuracy. In this paper, we
propose an Asynchronous Efficient Decentralized FL framework, i.e., AEDFL, in
heterogeneous environments with three unique contributions. First, we propose
an asynchronous FL system model with an efficient model aggregation method for
improving the FL convergence. Second, we propose a dynamic staleness-aware
model update approach to achieve superior accuracy. Third, we propose an
adaptive sparse training method to reduce communication and computation costs
without significant accuracy degradation. Extensive experimentation on four
public datasets and four models demonstrates the strength of AEDFL in terms of
accuracy (up to 16.3% higher), efficiency (up to 92.9% faster), and computation
costs (up to 42.3% lower).