Randomized Quantization is All You Need for Differential Privacy in
  Federated Learning
Yeojoon Youn Zihao Hu Juba Ziani Jacob Abernethy
Federated learning (FL) is a common and practical framework for learning a
machine model in a decentralized fashion. A primary motivation behind this
decentralized approach is data privacy, ensuring that the learner never sees
the data of each local source itself. Federated learning then comes with two
majors challenges: one is handling potentially complex model updates between a
server and a large number of data sources; the other is that de-centralization
may, in fact, be insufficient for privacy, as the local updates themselves can
reveal information about the sources' data. To address these issues, we
consider an approach to federated learning that combines quantization and
differential privacy. Absent privacy, Federated Learning often relies on
quantization to reduce communication complexity. We build upon this approach
and develop a new algorithm called the \textbf{R}andomized
\textbf{Q}uantization \textbf{M}echanism (RQM), which obtains privacy through a
two-levels of randomization. More precisely, we randomly sub-sample feasible
quantization levels, then employ a randomized rounding procedure using these
sub-sampled discrete levels. We are able to establish that our results preserve
``Renyi differential privacy'' (Renyi DP). We empirically study the performance
of our algorithm and demonstrate that compared to previous work it yields
improved privacy-accuracy trade-offs for DP federated learning. To the best of
our knowledge, this is the first study that solely relies on randomized
quantization without incorporating explicit discrete noise to achieve Renyi DP
guarantees in Federated Learning systems.