Make Me Happier: Evoking Emotions Through Image Diffusion Models
Qing Lin Jingfeng Zhang Yew Soon Ong Mengmi Zhang
Despite the rapid progress in image generation, emotional image editing
remains under-explored. The semantics, context, and structure of an image can
evoke emotional responses, making emotional image editing techniques valuable
for various real-world applications, including treatment of psychological
disorders, commercialization of products, and artistic design. For the first
time, we present a novel challenge of emotion-evoked image generation, aiming
to synthesize images that evoke target emotions while retaining the semantics
and structures of the original scenes. To address this challenge, we propose a
diffusion model capable of effectively understanding and editing source images
to convey desired emotions and sentiments. Moreover, due to the lack of emotion
editing datasets, we provide a unique dataset consisting of 340,000 pairs of
images and their emotion annotations. Furthermore, we conduct human
psychophysics experiments and introduce four new evaluation metrics to
systematically benchmark all the methods. Experimental results demonstrate that
our method surpasses all competitive baselines. Our diffusion model is capable
of identifying emotional cues from original images, editing images that elicit
desired emotions, and meanwhile, preserving the semantic structure of the
original images. All code, model, and data will be made public.