SwiMDiff: Scene-wide Matching Contrastive Learning with Diffusion
  Constraint for Remote Sensing Image
Jiayuan Tian Jie Lei Jiaqing Zhang Weiying Xie Yunsong Li
With recent advancements in aerospace technology, the volume of unlabeled
remote sensing image (RSI) data has increased dramatically. Effectively
leveraging this data through self-supervised learning (SSL) is vital in the
field of remote sensing. However, current methodologies, particularly
contrastive learning (CL), a leading SSL method, encounter specific challenges
in this domain. Firstly, CL often mistakenly identifies geographically adjacent
samples with similar semantic content as negative pairs, leading to confusion
during model training. Secondly, as an instance-level discriminative task, it
tends to neglect the essential fine-grained features and complex details
inherent in unstructured RSIs. To overcome these obstacles, we introduce
SwiMDiff, a novel self-supervised pre-training framework designed for RSIs.
SwiMDiff employs a scene-wide matching approach that effectively recalibrates
labels to recognize data from the same scene as false negatives. This
adjustment makes CL more applicable to the nuances of remote sensing.
Additionally, SwiMDiff seamlessly integrates CL with a diffusion model. Through
the implementation of pixel-level diffusion constraints, we enhance the
encoder's ability to capture both the global semantic information and the
fine-grained features of the images more comprehensively. Our proposed
framework significantly enriches the information available for downstream tasks
in remote sensing. Demonstrating exceptional performance in change detection
and land-cover classification tasks, SwiMDiff proves its substantial utility
and value in the field of remote sensing.