Adaptive Federated Pruning in Hierarchical Wireless Networks
Xiaonan Liu Shiqiang Wang Yansha Deng Arumugam Nallanathan
Federated Learning (FL) is a promising privacy-preserving distributed
learning framework where a server aggregates models updated by multiple devices
without accessing their private datasets. Hierarchical FL (HFL), as a
device-edge-cloud aggregation hierarchy, can enjoy both the cloud server's
access to more datasets and the edge servers' efficient communications with
devices. However, the learning latency increases with the HFL network scale due
to the increasing number of edge servers and devices with limited local
computation capability and communication bandwidth. To address this issue, in
this paper, we introduce model pruning for HFL in wireless networks to reduce
the neural network scale. We present the convergence analysis of an upper on
the l2 norm of gradients for HFL with model pruning, analyze the computation
and communication latency of the proposed model pruning scheme, and formulate
an optimization problem to maximize the convergence rate under a given latency
threshold by jointly optimizing the pruning ratio and wireless resource
allocation. By decoupling the optimization problem and using Karush Kuhn Tucker
(KKT) conditions, closed-form solutions of pruning ratio and wireless resource
allocation are derived. Simulation results show that our proposed HFL with
model pruning achieves similar learning accuracy compared with the HFL without
model pruning and reduces about 50 percent communication cost.