Personalized Image Enhancement Featuring Masked Style Modeling
Satoshi Kosugi Toshihiko Yamasaki
We address personalized image enhancement in this study, where we enhance
input images for each user based on the user's preferred images. Previous
methods apply the same preferred style to all input images (i.e., only one
style for each user); in contrast to these methods, we aim to achieve
content-aware personalization by applying different styles to each image
considering the contents. For content-aware personalization, we make two
contributions. First, we propose a method named masked style modeling, which
can predict a style for an input image considering the contents by using the
framework of masked language modeling. Second, to allow this model to consider
the contents of images, we propose a novel training scheme where we download
images from Flickr and create pseudo input and retouched image pairs using a
degrading model. We conduct quantitative evaluations and a user study, and our
method trained using our training scheme successfully achieves content-aware
personalization; moreover, our method outperforms other previous methods in
this field. Our source code is available at
https://github.com/satoshi-kosugi/masked-style-modeling.