Privacy Amplification for the Gaussian Mechanism via Bounded Support
Shengyuan Hu Saeed Mahloujifar Virginia Smith Kamalika Chaudhuri Chuan Guo
Data-dependent privacy accounting frameworks such as per-instance
differential privacy (pDP) and Fisher information loss (FIL) confer
fine-grained privacy guarantees for individuals in a fixed training dataset.
These guarantees can be desirable compared to vanilla DP in real world settings
as they tightly upper-bound the privacy leakage for a $\textit{specific}$
individual in an $\textit{actual}$ dataset, rather than considering worst-case
datasets. While these frameworks are beginning to gain popularity, to date,
there is a lack of private mechanisms that can fully leverage advantages of
data-dependent accounting. To bridge this gap, we propose simple modifications
of the Gaussian mechanism with bounded support, showing that they amplify
privacy guarantees under data-dependent accounting. Experiments on model
training with DP-SGD show that using bounded support Gaussian mechanisms can
provide a reduction of the pDP bound $\epsilon$ by as much as 30% without
negative effects on model utility.