Aggregation Weighting of Federated Learning via Generalization Bound
  Estimation
Mingwei Xu Xiaofeng Cao Ivor W. Tsang James T. Kwok
Federated Learning (FL) typically aggregates client model parameters using a
weighting approach determined by sample proportions. However, this naive
weighting method may lead to unfairness and degradation in model performance
due to statistical heterogeneity and the inclusion of noisy data among clients.
Theoretically, distributional robustness analysis has shown that the
generalization performance of a learning model with respect to any shifted
distribution is bounded. This motivates us to reconsider the weighting approach
in federated learning. In this paper, we replace the aforementioned weighting
method with a new strategy that considers the generalization bounds of each
local model. Specifically, we estimate the upper and lower bounds of the
second-order origin moment of the shifted distribution for the current local
model, and then use these bounds disagreements as the aggregation proportions
for weightings in each communication round. Experiments demonstrate that the
proposed weighting strategy significantly improves the performance of several
representative FL algorithms on benchmark datasets.