Diffence: Fencing Membership Privacy With Diffusion Models
Yuefeng Peng Ali Naseh Amir Houmansadr
Deep learning models, while achieving remarkable performance across various
tasks, are vulnerable to member inference attacks, wherein adversaries identify
if a specific data point was part of a model's training set. This
susceptibility raises substantial privacy concerns, especially when models are
trained on sensitive datasets. Current defense methods often struggle to
provide robust protection without hurting model utility, and they often require
retraining the model or using extra data. In this work, we introduce a novel
defense framework against membership attacks by leveraging generative models.
The key intuition of our defense is to remove the differences between member
and non-member inputs which can be used to perform membership attacks, by
re-generating input samples before feeding them to the target model. Therefore,
our defense works \emph{pre-inference}, which is unlike prior defenses that are
either training-time (modify the model) or post-inference time (modify the
model's output).
  A unique feature of our defense is that it works on input samples only,
without modifying the training or inference phase of the target model.
Therefore, it can be cascaded with other defense mechanisms as we demonstrate
through experiments. Through extensive experimentation, we show that our
approach can serve as a robust plug-n-play defense mechanism, enhancing
membership privacy without compromising model utility in both baseline and
defended settings. For example, our method enhanced the effectiveness of recent
state-of-the-art defenses, reducing attack accuracy by an average of 5.7\% to
12.4\% across three datasets, without any impact on the model's accuracy. By
integrating our method with prior defenses, we achieve new state-of-the-art
performance in the privacy-utility trade-off.