Editing Massive Concepts in Text-to-Image Diffusion Models
Tianwei Xiong Yue Wu Enze Xie Yue Wu Zhenguo Li Xihui Liu
Text-to-image diffusion models suffer from the risk of generating outdated,
copyrighted, incorrect, and biased content. While previous methods have
mitigated the issues on a small scale, it is essential to handle them
simultaneously in larger-scale real-world scenarios. We propose a two-stage
method, Editing Massive Concepts In Diffusion Models (EMCID). The first stage
performs memory optimization for each individual concept with dual
self-distillation from text alignment loss and diffusion noise prediction loss.
The second stage conducts massive concept editing with multi-layer, closed form
model editing. We further propose a comprehensive benchmark, named ImageNet
Concept Editing Benchmark (ICEB), for evaluating massive concept editing for
T2I models with two subtasks, free-form prompts, massive concept categories,
and extensive evaluation metrics. Extensive experiments conducted on our
proposed benchmark and previous benchmarks demonstrate the superior scalability
of EMCID for editing up to 1,000 concepts, providing a practical approach for
fast adjustment and re-deployment of T2I diffusion models in real-world
applications.