Bayesian Federated Model Compression for Communication and Computation
  Efficiency
Chengyu Xia Danny H. K. Tsang Vincent K. N. Lau
In this paper, we investigate Bayesian model compression in federated
learning (FL) to construct sparse models that can achieve both communication
and computation efficiencies. We propose a decentralized Turbo variational
Bayesian inference (D-Turbo-VBI) FL framework where we firstly propose a
hierarchical sparse prior to promote a clustered sparse structure in the weight
matrix. Then, by carefully integrating message passing and VBI with a
decentralized turbo framework, we propose the D-Turbo-VBI algorithm which can
(i) reduce both upstream and downstream communication overhead during federated
training, and (ii) reduce the computational complexity during local inference.
Additionally, we establish the convergence property for thr proposed
D-Turbo-VBI algorithm. Simulation results show the significant gain of our
proposed algorithm over the baselines in reducing communication overhead during
federated training and computational complexity of final model.