Information Leakage from Data Updates in Machine Learning Models
Tian Hui Farhad Farokhi Olga Ohrimenko
In this paper we consider the setting where machine learning models are
retrained on updated datasets in order to incorporate the most up-to-date
information or reflect distribution shifts. We investigate whether one can
infer information about these updates in the training data (e.g., changes to
attribute values of records). Here, the adversary has access to snapshots of
the machine learning model before and after the change in the dataset occurs.
Contrary to the existing literature, we assume that an attribute of a single or
multiple training data points are changed rather than entire data records are
removed or added. We propose attacks based on the difference in the prediction
confidence of the original model and the updated model. We evaluate our attack
methods on two public datasets along with multi-layer perceptron and logistic
regression models. We validate that two snapshots of the model can result in
higher information leakage in comparison to having access to only the updated
model. Moreover, we observe that data records with rare values are more
vulnerable to attacks, which points to the disparate vulnerability of privacy
attacks in the update setting. When multiple records with the same original
attribute value are updated to the same new value (i.e., repeated changes), the
attacker is more likely to correctly guess the updated values since repeated
changes leave a larger footprint on the trained model. These observations point
to vulnerability of machine learning models to attribute inference attacks in
the update setting.