Communication Efficient ConFederated Learning: An Event-Triggered SAGA
  Approach
Bin Wang Jun Fang Hongbin Li Yonina C. Eldar
Federated learning (FL) is a machine learning paradigm that targets model
training without gathering the local data dispersed over various data sources.
Standard FL, which employs a single server, can only support a limited number
of users, leading to degraded learning capability. In this work, we consider a
multi-server FL framework, referred to as \emph{Confederated Learning} (CFL),
in order to accommodate a larger number of users. A CFL system is composed of
multiple networked edge servers, with each server connected to an individual
set of users. Decentralized collaboration among servers is leveraged to harness
all users' data for model training. Due to the potentially massive number of
users involved, it is crucial to reduce the communication overhead of the CFL
system. We propose a stochastic gradient method for distributed learning in the
CFL framework. The proposed method incorporates a conditionally-triggered user
selection (CTUS) mechanism as the central component to effectively reduce
communication overhead. Relying on a delicately designed triggering condition,
the CTUS mechanism allows each server to select only a small number of users to
upload their gradients, without significantly jeopardizing the convergence
performance of the algorithm. Our theoretical analysis reveals that the
proposed algorithm enjoys a linear convergence rate. Simulation results show
that it achieves substantial improvement over state-of-the-art algorithms in
terms of communication efficiency.