Responsible Visual Editing
Minheng Ni Yeli Shen Lei Zhang Wangmeng Zuo
With recent advancements in visual synthesis, there is a growing risk of
encountering images with detrimental effects, such as hate, discrimination, or
privacy violations. The research on transforming harmful images into
responsible ones remains unexplored. In this paper, we formulate a new task,
responsible visual editing, which entails modifying specific concepts within an
image to render it more responsible while minimizing changes. However, the
concept that needs to be edited is often abstract, making it challenging to
locate what needs to be modified and plan how to modify it. To tackle these
challenges, we propose a Cognitive Editor (CoEditor) that harnesses the large
multimodal model through a two-stage cognitive process: (1) a perceptual
cognitive process to focus on what needs to be modified and (2) a behavioral
cognitive process to strategize how to modify. To mitigate the negative
implications of harmful images on research, we create a transparent and public
dataset, AltBear, which expresses harmful information using teddy bears instead
of humans. Experiments demonstrate that CoEditor can effectively comprehend
abstract concepts within complex scenes and significantly surpass the
performance of baseline models for responsible visual editing. We find that the
AltBear dataset corresponds well to the harmful content found in real images,
offering a consistent experimental evaluation, thereby providing a safer
benchmark for future research. Moreover, CoEditor also shows great results in
general editing. We release our code and dataset at
https://github.com/kodenii/Responsible-Visual-Editing.