Deep Equilibrium Models Meet Federated Learning
Alexandros Gkillas Dimitris Ampeliotis Kostas Berberidis
In this study the problem of Federated Learning (FL) is explored under a new
perspective by utilizing the Deep Equilibrium (DEQ) models instead of
conventional deep learning networks. We claim that incorporating DEQ models
into the federated learning framework naturally addresses several open problems
in FL, such as the communication overhead due to the sharing large models and
the ability to incorporate heterogeneous edge devices with significantly
different computation capabilities. Additionally, a weighted average fusion
rule is proposed at the server-side of the FL framework to account for the
different qualities of models from heterogeneous edge devices. To the best of
our knowledge, this study is the first to establish a connection between DEQ
models and federated learning, contributing to the development of an efficient
and effective FL framework. Finally, promising initial experimental results are
presented, demonstrating the potential of this approach in addressing
challenges of FL.