Federated Learning based on Pruning and Recovery
Chengjie Ma
A novel federated learning training framework for heterogeneous environments
is presented, taking into account the diverse network speeds of clients in
realistic settings. This framework integrates asynchronous learning algorithms
and pruning techniques, effectively addressing the inefficiencies of
traditional federated learning algorithms in scenarios involving heterogeneous
devices, as well as tackling the staleness issue and inadequate training of
certain clients in asynchronous algorithms. Through the incremental restoration
of model size during training, the framework expedites model training while
preserving model accuracy. Furthermore, enhancements to the federated learning
aggregation process are introduced, incorporating a buffering mechanism to
enable asynchronous federated learning to operate akin to synchronous learning.
Additionally, optimizations in the process of the server transmitting the
global model to clients reduce communication overhead. Our experiments across
various datasets demonstrate that: (i) significant reductions in training time
and improvements in convergence accuracy are achieved compared to conventional
asynchronous FL and HeteroFL; (ii) the advantages of our approach are more
pronounced in scenarios with heterogeneous clients and non-IID client data.