Adapt then Unlearn: Exploiting Parameter Space Semantics for Unlearning
  in Generative Adversarial Networks
Piyush Tiwary Atri Guha Subhodip Panda Prathosh A. P
The increased attention to regulating the outputs of deep generative models,
driven by growing concerns about privacy and regulatory compliance, has
highlighted the need for effective control over these models. This necessity
arises from instances where generative models produce outputs containing
undesirable, offensive, or potentially harmful content. To tackle this
challenge, the concept of machine unlearning has emerged, aiming to forget
specific learned information or to erase the influence of undesired data
subsets from a trained model. The objective of this work is to prevent the
generation of outputs containing undesired features from a pre-trained GAN
where the underlying training data set is inaccessible. Our approach is
inspired by a crucial observation: the parameter space of GANs exhibits
meaningful directions that can be leveraged to suppress specific undesired
features. However, such directions usually result in the degradation of the
quality of generated samples. Our proposed method, known as
'Adapt-then-Unlearn,' excels at unlearning such undesirable features while also
maintaining the quality of generated samples. This method unfolds in two
stages: in the initial stage, we adapt the pre-trained GAN using negative
samples provided by the user, while in the subsequent stage, we focus on
unlearning the undesired feature. During the latter phase, we train the
pre-trained GAN using positive samples, incorporating a repulsion regularizer.
This regularizer encourages the model's parameters to be away from the
parameters associated with the adapted model from the first stage while also
maintaining the quality of generated samples. To the best of our knowledge, our
approach stands as first method addressing unlearning in GANs. We validate the
effectiveness of our method through comprehensive experiments.