Amplifying Training Data Exposure through Fine-Tuning with
  Pseudo-Labeled Memberships
Myung Gyo Oh Hong Eun Ahn Leo Hyun Park Taekyoung Kwon
Neural language models (LMs) are vulnerable to training data extraction
attacks due to data memorization. This paper introduces a novel attack scenario
wherein an attacker adversarially fine-tunes pre-trained LMs to amplify the
exposure of the original training data. This strategy differs from prior
studies by aiming to intensify the LM's retention of its pre-training dataset.
To achieve this, the attacker needs to collect generated texts that are closely
aligned with the pre-training data. However, without knowledge of the actual
dataset, quantifying the amount of pre-training data within generated texts is
challenging. To address this, we propose the use of pseudo-labels for these
generated texts, leveraging membership approximations indicated by
machine-generated probabilities from the target LM. We subsequently fine-tune
the LM to favor generations with higher likelihoods of originating from the
pre-training data, based on their membership probabilities. Our empirical
findings indicate a remarkable outcome: LMs with over 1B parameters exhibit a
four to eight-fold increase in training data exposure. We discuss potential
mitigations and suggest future research directions.