Wasserstein Differential Privacy
Chengyi Yang Jiayin Qi Aimin Zhou
Differential privacy (DP) has achieved remarkable results in the field of
privacy-preserving machine learning. However, existing DP frameworks do not
satisfy all the conditions for becoming metrics, which prevents them from
deriving better basic private properties and leads to exaggerated values on
privacy budgets. We propose Wasserstein differential privacy (WDP), an
alternative DP framework to measure the risk of privacy leakage, which
satisfies the properties of symmetry and triangle inequality. We show and prove
that WDP has 13 excellent properties, which can be theoretical supports for the
better performance of WDP than other DP frameworks. In addition, we derive a
general privacy accounting method called Wasserstein accountant, which enables
WDP to be applied in stochastic gradient descent (SGD) scenarios containing
sub-sampling. Experiments on basic mechanisms, compositions and deep learning
show that the privacy budgets obtained by Wasserstein accountant are relatively
stable and less influenced by order. Moreover, the overestimation on privacy
budgets can be effectively alleviated. The code is available at
https://github.com/Hifipsysta/WDP.