Accelerating Federated Learning by Selecting Beneficial Herd of Local
  Gradients
Ping Luo Xiaoge Deng Ziqing Wen Tao Sun Dongsheng Li
Federated Learning (FL) is a distributed machine learning framework in
communication network systems. However, the systems' Non-Independent and
Identically Distributed (Non-IID) data negatively affect the convergence
efficiency of the global model, since only a subset of these data samples are
beneficial for model convergence. In pursuit of this subset, a reliable
approach involves determining a measure of validity to rank the samples within
the dataset. In this paper, We propose the BHerd strategy which selects a
beneficial herd of local gradients to accelerate the convergence of the FL
model. Specifically, we map the distribution of the local dataset to the local
gradients and use the Herding strategy to obtain a permutation of the set of
gradients, where the more advanced gradients in the permutation are closer to
the average of the set of gradients. These top portion of the gradients will be
selected and sent to the server for global aggregation. We conduct experiments
on different datasets, models and scenarios by building a prototype system, and
experimental results demonstrate that our BHerd strategy is effective in
selecting beneficial local gradients to mitigate the effects brought by the
Non-IID dataset, thus accelerating model convergence.