Over-the-Air Federated Learning and Optimization
Jingyang Zhu Yuanming Shi Yong Zhou Chunxiao Jiang Wei Chen Khaled B. Letaief
Federated learning (FL), as an emerging distributed machine learning
paradigm, allows a mass of edge devices to collaboratively train a global model
while preserving privacy. In this tutorial, we focus on FL via over-the-air
computation (AirComp), which is proposed to reduce the communication overhead
for FL over wireless networks at the cost of compromising in the learning
performance due to model aggregation error arising from channel fading and
noise. We first provide a comprehensive study on the convergence of
AirComp-based FedAvg (AirFedAvg) algorithms under both strongly convex and
non-convex settings with constant and diminishing learning rates in the
presence of data heterogeneity. Through convergence and asymptotic analysis, we
characterize the impact of aggregation error on the convergence bound and
provide insights for system design with convergence guarantees. Then we derive
convergence rates for AirFedAvg algorithms for strongly convex and non-convex
objectives. For different types of local updates that can be transmitted by
edge devices (i.e., local model, gradient, and model difference), we reveal
that transmitting local model in AirFedAvg may cause divergence in the training
procedure. In addition, we consider more practical signal processing schemes to
improve the communication efficiency and further extend the convergence
analysis to different forms of model aggregation error caused by these signal
processing schemes. Extensive simulation results under different settings of
objective functions, transmitted local information, and communication schemes
verify the theoretical conclusions.