Federated Generative Learning with Foundation Models
Jie Zhang Xiaohua Qi Bo Zhao
Existing federated learning solutions focus on transmitting features,
parameters or gadients between clients and server, which suffer from serious
low-efficiency and privacy-leakage problems. Thanks to the emerging foundation
generative models, we propose a novel federated learning framework, namely
Federated Generative Learning, that transmits prompts associated with
distributed training data between clients and server. The informative training
data can be synthesized remotely based on received prompts containing little
privacy and the foundation generative models. The new framework possesses
multiple advantages, including improved communication efficiency, better
resilience to distribution shift, substantial performance gains, and enhanced
privacy protection, which are verified in extensive experiments on ImageNet and
DomainNet datasets.