One-Shot Machine Unlearning with Mnemonic Code
Tomoya Yamashita Masanori Yamada Takashi Shibata
Deep learning has achieved significant improvements in accuracy and has been
applied to various fields. With the spread of deep learning, a new problem has
also emerged; deep learning models can sometimes have undesirable information
from an ethical standpoint. This problem must be resolved if deep learning is
to make sensitive decisions such as hiring and prison sentencing. Machine
unlearning (MU) is the research area that responds to such demands. MU aims at
forgetting about undesirable training data from a trained deep learning model.
A naive MU approach is to re-train the whole model with the training data from
which the undesirable data has been removed. However, re-training the whole
model can take a huge amount of time and consumes significant computer
resources. To make MU even more practical, a simple-yet-effective MU method is
required. In this paper, we propose a one-shot MU method, which does not need
additional training. To design one-shot MU, we add noise to the model
parameters that are sensitive to undesirable information. In our proposed
method, we use the Fisher information matrix (FIM) to estimate the sensitive
model parameters. Training data were usually used to evaluate the FIM in
existing methods. In contrast, we avoid the need to retain the training data
for calculating the FIM by using class-specific synthetic signals called
mnemonic code. Extensive experiments using artificial and natural datasets
demonstrate that our method outperforms the existing methods.