Loss-Free Machine Unlearning
Jack Foster Stefan Schoepf Alexandra Brintrup
We present a machine unlearning approach that is both retraining- and
label-free. Most existing machine unlearning approaches require a model to be
fine-tuned to remove information while preserving performance. This is
computationally expensive and necessitates the storage of the whole dataset for
the lifetime of the model. Retraining-free approaches often utilise Fisher
information, which is derived from the loss and requires labelled data which
may not be available. Thus, we present an extension to the Selective Synaptic
Dampening algorithm, substituting the diagonal of the Fisher information matrix
for the gradient of the l2 norm of the model output to approximate sensitivity.
We evaluate our method in a range of experiments using ResNet18 and Vision
Transformer. Results show our label-free method is competitive with existing
state-of-the-art approaches.