SSDiff: Spatial-spectral Integrated Diffusion Model for Remote Sensing
  Pansharpening
Yu Zhong Xiao Wu Liang-Jian Deng Zihan Cao
Pansharpening is a significant image fusion technique that merges the spatial
content and spectral characteristics of remote sensing images to generate
high-resolution multispectral images. Recently, denoising diffusion
probabilistic models have been gradually applied to visual tasks, enhancing
controllable image generation through low-rank adaptation (LoRA). In this
paper, we introduce a spatial-spectral integrated diffusion model for the
remote sensing pansharpening task, called SSDiff, which considers the
pansharpening process as the fusion process of spatial and spectral components
from the perspective of subspace decomposition. Specifically, SSDiff utilizes
spatial and spectral branches to learn spatial details and spectral features
separately, then employs a designed alternating projection fusion module (APFM)
to accomplish the fusion. Furthermore, we propose a frequency modulation
inter-branch module (FMIM) to modulate the frequency distribution between
branches. The two components of SSDiff can perform favorably against the APFM
when utilizing a LoRA-like branch-wise alternative fine-tuning method. It
refines SSDiff to capture component-discriminating features more sufficiently.
Finally, extensive experiments on four commonly used datasets, i.e.,
WorldView-3, WorldView-2, GaoFen-2, and QuickBird, demonstrate the superiority
of SSDiff both visually and quantitatively. The code will be made open source
after possible acceptance.