Binary Federated Learning with Client-Level Differential Privacy
Lumin Liu Jun Zhang Shenghui Song Khaled B. Letaief
Federated learning (FL) is a privacy-preserving collaborative learning
framework, and differential privacy can be applied to further enhance its
privacy protection. Existing FL systems typically adopt Federated Average
(FedAvg) as the training algorithm and implement differential privacy with a
Gaussian mechanism. However, the inherent privacy-utility trade-off in these
systems severely degrades the training performance if a tight privacy budget is
enforced. Besides, the Gaussian mechanism requires model weights to be of
high-precision. To improve communication efficiency and achieve a better
privacy-utility trade-off, we propose a communication-efficient FL training
algorithm with differential privacy guarantee. Specifically, we propose to
adopt binary neural networks (BNNs) and introduce discrete noise in the FL
setting. Binary model parameters are uploaded for higher communication
efficiency and discrete noise is added to achieve the client-level differential
privacy protection. The achieved performance guarantee is rigorously proved,
and it is shown to depend on the level of discrete noise. Experimental results
based on MNIST and Fashion-MNIST datasets will demonstrate that the proposed
training algorithm achieves client-level privacy protection with performance
gain while enjoying the benefits of low communication overhead from binary
model updates.