Semi-Federated Learning: Convergence Analysis and Optimization of A
  Hybrid Learning Framework
Jingheng Zheng Wanli Ni Hui Tian Deniz Gunduz Tony Q. S. Quek Zhu Han
Under the organization of the base station (BS), wireless federated learning
(FL) enables collaborative model training among multiple devices. However, the
BS is merely responsible for aggregating local updates during the training
process, which incurs a waste of the computational resource at the BS. To
tackle this issue, we propose a semi-federated learning (SemiFL) paradigm to
leverage the computing capabilities of both the BS and devices for a hybrid
implementation of centralized learning (CL) and FL. Specifically, each device
sends both local gradients and data samples to the BS for training a shared
global model. To improve communication efficiency over the same time-frequency
resources, we integrate over-the-air computation for aggregation and
non-orthogonal multiple access for transmission by designing a novel
transceiver structure. To gain deep insights, we conduct convergence analysis
by deriving a closed-form optimality gap for SemiFL and extend the result to
two extra cases. In the first case, the BS uses all accumulated data samples to
calculate the CL gradient, while a decreasing learning rate is adopted in the
second case. Our analytical results capture the destructive effect of wireless
communication and show that both FL and CL are special cases of SemiFL. Then,
we formulate a non-convex problem to reduce the optimality gap by jointly
optimizing the transmit power and receive beamformers. Accordingly, we propose
a two-stage algorithm to solve this intractable problem, in which we provide
the closed-form solutions to the beamformers. Extensive simulation results on
two real-world datasets corroborate our theoretical analysis, and show that the
proposed SemiFL outperforms conventional FL and achieves 3.2% accuracy gain on
the MNIST dataset compared to state-of-the-art benchmarks.