Federated Learning for Computationally-Constrained Heterogeneous
  Devices: A Survey
Kilian Pfeiffer Martin Rapp Ramin Khalili Jörg Henkel
With an increasing number of smart devices like internet of things (IoT)
devices deployed in the field, offloadingtraining of neural networks (NNs) to a
central server becomes more and more infeasible. Recent efforts toimprove
users' privacy have led to on-device learning emerging as an alternative.
However, a model trainedonly on a single device, using only local data, is
unlikely to reach a high accuracy. Federated learning (FL)has been introduced
as a solution, offering a privacy-preserving trade-off between communication
overheadand model accuracy by sharing knowledge between devices but disclosing
the devices' private data. Theapplicability and the benefit of applying
baseline FL are, however, limited in many relevant use cases dueto the
heterogeneity present in such environments. In this survey, we outline the
heterogeneity challengesFL has to overcome to be widely applicable in
real-world applications. We especially focus on the aspect ofcomputation
heterogeneity among the participating devices and provide a comprehensive
overview of recentworks on heterogeneity-aware FL. We discuss two groups: works
that adapt the NN architecture and worksthat approach heterogeneity on a system
level, covering Federated Averaging (FedAvg), distillation, and
splitlearning-based approaches, as well as synchronous and asynchronous
aggregation schemes.