Perceptual Artifacts Localization for Image Synthesis Tasks
Lingzhi Zhang Zhengjie Xu Connelly Barnes Yuqian Zhou Qing Liu He Zhang Sohrab Amirghodsi Zhe Lin Eli Shechtman Jianbo Shi
Recent advancements in deep generative models have facilitated the creation
of photo-realistic images across various tasks. However, these generated images
often exhibit perceptual artifacts in specific regions, necessitating manual
correction. In this study, we present a comprehensive empirical examination of
Perceptual Artifacts Localization (PAL) spanning diverse image synthesis
endeavors. We introduce a novel dataset comprising 10,168 generated images,
each annotated with per-pixel perceptual artifact labels across ten synthesis
tasks. A segmentation model, trained on our proposed dataset, effectively
localizes artifacts across a range of tasks. Additionally, we illustrate its
proficiency in adapting to previously unseen models using minimal training
samples. We further propose an innovative zoom-in inpainting pipeline that
seamlessly rectifies perceptual artifacts in the generated images. Through our
experimental analyses, we elucidate several practical downstream applications,
such as automated artifact rectification, non-referential image quality
evaluation, and abnormal region detection in images. The dataset and code are
released.