Submodel Partitioning in Hierarchical Federated Learning: Algorithm
  Design and Convergence Analysis
Wenzhi Fang Dong-Jun Han Christopher G. Brinton
Hierarchical federated learning (HFL) has demonstrated promising scalability
advantages over the traditional "star-topology" architecture-based federated
learning (FL). However, HFL still imposes significant computation,
communication, and storage burdens on the edge, especially when training a
large-scale model over resource-constrained Internet of Things (IoT) devices.
In this paper, we propose hierarchical independent submodel training (HIST), a
new FL methodology that aims to address these issues in hierarchical settings.
The key idea behind HIST is a hierarchical version of model partitioning, where
we partition the global model into disjoint submodels in each round, and
distribute them across different cells, so that each cell is responsible for
training only one partition of the full model. This enables each client to save
computation/storage costs while alleviating the communication loads throughout
the hierarchy. We characterize the convergence behavior of HIST for non-convex
loss functions under mild assumptions, showing the impact of several attributes
(e.g., number of cells, local and global aggregation frequency) on the
performance-efficiency tradeoff. Finally, through numerical experiments, we
verify that HIST is able to save communication costs by a wide margin while
achieving the same target testing accuracy.