Separable Multi-Concept Erasure from Diffusion Models
Mengnan Zhao Lihe Zhang Tianhang Zheng Yuqiu Kong Baocai Yin
Large-scale diffusion models, known for their impressive image generation
capabilities, have raised concerns among researchers regarding social impacts,
such as the imitation of copyrighted artistic styles. In response, existing
approaches turn to machine unlearning techniques to eliminate unsafe concepts
from pre-trained models. However, these methods compromise the generative
performance and neglect the coupling among multi-concept erasures, as well as
the concept restoration problem. To address these issues, we propose a
Separable Multi-concept Eraser (SepME), which mainly includes two parts: the
generation of concept-irrelevant representations and the weight decoupling. The
former aims to avoid unlearning substantial information that is irrelevant to
forgotten concepts. The latter separates optimizable model weights, making each
weight increment correspond to a specific concept erasure without affecting
generative performance on other concepts. Specifically, the weight increment
for erasing a specified concept is formulated as a linear combination of
solutions calculated based on other known undesirable concepts. Extensive
experiments indicate the efficacy of our approach in eliminating concepts,
preserving model performance, and offering flexibility in the erasure or
recovery of various concepts.