Mitigating Privacy Risk in Membership Inference by Convex-Concave Loss
Zhenlong Liu Lei Feng Huiping Zhuang Xiaofeng Cao Hongxin Wei
Machine learning models are susceptible to membership inference attacks
(MIAs), which aim to infer whether a sample is in the training set. Existing
work utilizes gradient ascent to enlarge the loss variance of training data,
alleviating the privacy risk. However, optimizing toward a reverse direction
may cause the model parameters to oscillate near local minima, leading to
instability and suboptimal performance. In this work, we propose a novel method
-- Convex-Concave Loss, which enables a high variance of training loss
distribution by gradient descent. Our method is motivated by the theoretical
analysis that convex losses tend to decrease the loss variance during training.
Thus, our key idea behind CCL is to reduce the convexity of loss functions with
a concave term. Trained with CCL, neural networks produce losses with high
variance for training data, reinforcing the defense against MIAs. Extensive
experiments demonstrate the superiority of CCL, achieving state-of-the-art
balance in the privacy-utility trade-off.