FedMR: Federated Learning via Model Recombination
Ming Hu Zhihao Yue Zhiwei Ling Yihao Huang Cheng Chen Xian Wei Yang Liu Mingsong Chen
Although Federated Learning (FL) enables global model training across clients
without compromising their raw data, existing Federated Averaging
(FedAvg)-based methods suffer from the problem of low inference performance,
especially for unevenly distributed data among clients. This is mainly because
i) FedAvg initializes client models with the same global models, which makes
the local training hard to escape from the local search for optimal solutions;
and ii) by averaging model parameters in a coarse manner, FedAvg eclipses the
individual characteristics of local models. To address such issues that
strongly limit the inference capability of FL, we propose a novel and effective
FL paradigm named FedMR (Federated Model Recombination). Unlike conventional
FedAvg-based methods, the cloud server of FedMR shuffles each layer of
collected local models and recombines them to achieve new models for local
training on clients. Due to the diversified initialization models for clients
coupled with fine-grained model recombination, FedMR can converge to a
well-generalized global model for all the clients, leading to a superior
inference performance. Experimental results show that, compared with
state-of-the-art FL methods, FedMR can significantly improve inference accuracy
in a quicker manner without exposing client privacy.