De-identification is not always enough
Atiquer Rahman Sarkar Yao-Shun Chuang Noman Mohammed Xiaoqian Jiang
For sharing privacy-sensitive data, de-identification is commonly regarded as
adequate for safeguarding privacy. Synthetic data is also being considered as a
privacy-preserving alternative. Recent successes with numerical and tabular
data generative models and the breakthroughs in large generative language
models raise the question of whether synthetically generated clinical notes
could be a viable alternative to real notes for research purposes. In this
work, we demonstrated that (i) de-identification of real clinical notes does
not protect records against a membership inference attack, (ii) proposed a
novel approach to generate synthetic clinical notes using the current
state-of-the-art large language models, (iii) evaluated the performance of the
synthetically generated notes in a clinical domain task, and (iv) proposed a
way to mount a membership inference attack where the target model is trained
with synthetic data. We observed that when synthetically generated notes
closely match the performance of real data, they also exhibit similar privacy
concerns to the real data. Whether other approaches to synthetically generated
clinical notes could offer better trade-offs and become a better alternative to
sensitive real notes warrants further investigation.