Pixel-Inconsistency Modeling for Image Manipulation Localization
Chenqi Kong Anwei Luo Shiqi Wang Haoliang Li Anderson Rocha Alex C. Kot
Digital image forensics plays a crucial role in image authentication and
manipulation localization. Despite the progress powered by deep neural
networks, existing forgery localization methodologies exhibit limitations when
deployed to unseen datasets and perturbed images (i.e., lack of generalization
and robustness to real-world applications). To circumvent these problems and
aid image integrity, this paper presents a generalized and robust manipulation
localization model through the analysis of pixel inconsistency artifacts. The
rationale is grounded on the observation that most image signal processors
(ISP) involve the demosaicing process, which introduces pixel correlations in
pristine images. Moreover, manipulating operations, including splicing,
copy-move, and inpainting, directly affect such pixel regularity. We,
therefore, first split the input image into several blocks and design masked
self-attention mechanisms to model the global pixel dependency in input images.
Simultaneously, we optimize another local pixel dependency stream to mine local
manipulation clues within input forgery images. In addition, we design novel
Learning-to-Weight Modules (LWM) to combine features from the two streams,
thereby enhancing the final forgery localization performance. To improve the
training process, we propose a novel Pixel-Inconsistency Data Augmentation
(PIDA) strategy, driving the model to focus on capturing inherent pixel-level
artifacts instead of mining semantic forgery traces. This work establishes a
comprehensive benchmark integrating 15 representative detection models across
12 datasets. Extensive experiments show that our method successfully extracts
inherent pixel-inconsistency forgery fingerprints and achieve state-of-the-art
generalization and robustness performances in image manipulation localization.