Federated Learning with Reduced Information Leakage and Computation
Tongxin Yin Xueru Zhang Mohammad Mahdi Khalili Mingyan Liu
Federated learning (FL) is a distributed learning paradigm that allows
multiple decentralized clients to collaboratively learn a common model without
sharing local data. Although local data is not exposed directly, privacy
concerns nonetheless exist as clients' sensitive information can be inferred
from intermediate computations. Moreover, such information leakage accumulates
substantially over time as the same data is repeatedly used during the
iterative learning process. As a result, it can be particularly difficult to
balance the privacy-accuracy trade-off when designing privacy-preserving FL
algorithms. In this paper, we introduce Upcycled-FL, a novel federated learning
framework with first-order approximation applied at every even iteration. Under
this framework, half of the FL updates incur no information leakage and require
much less computation. We first conduct the theoretical analysis on the
convergence (rate) of Upcycled-FL, and then apply perturbation mechanisms to
preserve privacy. Experiments on real-world data show that Upcycled-FL
consistently outperforms existing methods over heterogeneous data, and
significantly improves privacy-accuracy trade-off while reducing 48% of the
training time on average.