Toward efficient resource utilization at edge nodes in federated
  learning
Sadi Alawadi Addi Ait-Mlouk Salman Toor Andreas Hellander
Federated learning (FL) enables edge nodes to collaboratively contribute to
constructing a global model without sharing their data. This is accomplished by
devices computing local, private model updates that are then aggregated by a
server. However, computational resource constraints and network communication
can become a severe bottleneck for larger model sizes typical for deep learning
applications. Edge nodes tend to have limited hardware resources (RAM, CPU),
and the network bandwidth and reliability at the edge is a concern for scaling
federated fleet applications. In this paper, we propose and evaluate a FL
strategy inspired by transfer learning in order to reduce resource utilization
on devices, as well as the load on the server and network in each global
training round. For each local model update, we randomly select layers to
train, freezing the remaining part of the model. In doing so, we can reduce
both server load and communication costs per round by excluding all untrained
layer weights from being transferred to the server. The goal of this study is
to empirically explore the potential trade-off between resource utilization on
devices and global model convergence under the proposed strategy. We implement
the approach using the federated learning framework FEDn. A number of
experiments were carried out over different datasets (CIFAR-10, CASA, and
IMDB), performing different tasks using different deep-learning model
architectures. Our results show that training the model partially can
accelerate the training process, efficiently utilizes resources on-device, and
reduce the data transmission by around 75% and 53% when we train 25%, and 50%
of the model layers, respectively, without harming the resulting global model
accuracy.