EvoFed: Leveraging Evolutionary Strategies for Communication-Efficient
  Federated Learning
Mohammad Mahdi Rahimi Hasnain Irshad Bhatti Younghyun Park Humaira Kousar Jaekyun Moon
Federated Learning (FL) is a decentralized machine learning paradigm that
enables collaborative model training across dispersed nodes without having to
force individual nodes to share data. However, its broad adoption is hindered
by the high communication costs of transmitting a large number of model
parameters. This paper presents EvoFed, a novel approach that integrates
Evolutionary Strategies (ES) with FL to address these challenges. EvoFed
employs a concept of 'fitness-based information sharing', deviating
significantly from the conventional model-based FL. Rather than exchanging the
actual updated model parameters, each node transmits a distance-based
similarity measure between the locally updated model and each member of the
noise-perturbed model population. Each node, as well as the server, generates
an identical population set of perturbed models in a completely synchronized
fashion using the same random seeds. With properly chosen noise variance and
population size, perturbed models can be combined to closely reflect the actual
model updated using the local dataset, allowing the transmitted similarity
measures (or fitness values) to carry nearly the complete information about the
model parameters. As the population size is typically much smaller than the
number of model parameters, the savings in communication load is large. The
server aggregates these fitness values and is able to update the global model.
This global fitness vector is then disseminated back to the nodes, each of
which applies the same update to be synchronized to the global model. Our
analysis shows that EvoFed converges, and our experimental results validate
that at the cost of increased local processing loads, EvoFed achieves
performance comparable to FedAvg while reducing overall communication
requirements drastically in various practical settings.