P-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models
Haoran Li Dadi Guo Donghao Li Wei Fan Qi Hu Xin Liu Chunkit Chan Duanyi Yao Yangqiu Song
The rapid development of language models (LMs) brings unprecedented
accessibility and usage for both models and users. On the one hand, powerful
LMs, trained with massive textual data, achieve state-of-the-art performance
over numerous downstream NLP tasks. On the other hand, more and more attention
is paid to unrestricted model accesses that may bring malicious privacy risks
of data leakage. To address these issues, many recent works propose
privacy-preserving language models (PPLMs) with differential privacy (DP).
Unfortunately, different DP implementations make it challenging for a fair
comparison among existing PPLMs. In this paper, we present P-Bench, a
multi-perspective privacy evaluation benchmark to empirically and intuitively
quantify the privacy leakage of LMs. Instead of only protecting and measuring
the privacy of protected data with DP parameters, P-Bench sheds light on the
neglected inference data privacy during actual usage. P-Bench first clearly
defines multi-faceted privacy objectives during private fine-tuning. Then,
P-Bench constructs a unified pipeline to perform private fine-tuning. Lastly,
P-Bench performs existing privacy attacks on LMs with pre-defined privacy
objectives as the empirical evaluation results. The empirical attack results
are used to fairly and intuitively evaluate the privacy leakage of various
PPLMs. We conduct extensive experiments on three datasets of GLUE for
mainstream LMs.