Towards Generic Image Manipulation Detection with Weakly-Supervised
  Self-Consistency Learning
Yuanhao Zhai Tianyu Luan David Doermann Junsong Yuan
As advanced image manipulation techniques emerge, detecting the manipulation
becomes increasingly important. Despite the success of recent learning-based
approaches for image manipulation detection, they typically require expensive
pixel-level annotations to train, while exhibiting degraded performance when
testing on images that are differently manipulated compared with training
images. To address these limitations, we propose weakly-supervised image
manipulation detection, such that only binary image-level labels (authentic or
tampered with) are required for training purpose. Such a weakly-supervised
setting can leverage more training images and has the potential to adapt
quickly to new manipulation techniques. To improve the generalization ability,
we propose weakly-supervised self-consistency learning (WSCL) to leverage the
weakly annotated images. Specifically, two consistency properties are learned:
multi-source consistency (MSC) and inter-patch consistency (IPC). MSC exploits
different content-agnostic information and enables cross-source learning via an
online pseudo label generation and refinement process. IPC performs global
pair-wise patch-patch relationship reasoning to discover a complete region of
manipulation. Extensive experiments validate that our WSCL, even though is
weakly supervised, exhibits competitive performance compared with
fully-supervised counterpart under both in-distribution and out-of-distribution
evaluations, as well as reasonable manipulation localization ability.