PROMPT-IML: Image Manipulation Localization with Pre-trained Foundation
  Models Through Prompt Tuning
Xuntao Liu Yuzhou Yang Qichao Ying Zhenxing Qian Xinpeng Zhang Sheng Li
Deceptive images can be shared in seconds with social networking services,
posing substantial risks. Tampering traces, such as boundary artifacts and
high-frequency information, have been significantly emphasized by massive
networks in the Image Manipulation Localization (IML) field. However, they are
prone to image post-processing operations, which limit the generalization and
robustness of existing methods. We present a novel Prompt-IML framework. We
observe that humans tend to discern the authenticity of an image based on both
semantic and high-frequency information, inspired by which, the proposed
framework leverages rich semantic knowledge from pre-trained visual foundation
models to assist IML. We are the first to design a framework that utilizes
visual foundation models specially for the IML task. Moreover, we design a
Feature Alignment and Fusion module to align and fuse features of semantic
features with high-frequency features, which aims at locating tampered regions
from multiple perspectives. Experimental results demonstrate that our model can
achieve better performance on eight typical fake image datasets and outstanding
robustness.