Federated PAC-Bayesian Learning on Non-IID data
Zihao Zhao Yang Liu Wenbo Ding Xiao-Ping Zhang
Existing research has either adapted the Probably Approximately Correct (PAC)
Bayesian framework for federated learning (FL) or used information-theoretic
PAC-Bayesian bounds while introducing their theorems, but few considering the
non-IID challenges in FL. Our work presents the first non-vacuous federated
PAC-Bayesian bound tailored for non-IID local data. This bound assumes unique
prior knowledge for each client and variable aggregation weights. We also
introduce an objective function and an innovative Gibbs-based algorithm for the
optimization of the derived bound. The results are validated on real-world
datasets.