Adaptive Semantic-Enhanced Denoising Diffusion Probabilistic Model for
  Remote Sensing Image Super-Resolution
Jialu Sui Xianping Ma Xiaokang Zhang Man-On Pun
Remote sensing image super-resolution (SR) is a crucial task to restore
high-resolution (HR) images from low-resolution (LR) observations. Recently,
the Denoising Diffusion Probabilistic Model (DDPM) has shown promising
performance in image reconstructions by overcoming problems inherent in
generative models, such as over-smoothing and mode collapse. However, the
high-frequency details generated by DDPM often suffer from misalignment with HR
images due to the model's tendency to overlook long-range semantic contexts.
This is attributed to the widely used U-Net decoder in the conditional noise
predictor, which tends to overemphasize local information, leading to the
generation of noises with significant variances during the prediction process.
To address these issues, an adaptive semantic-enhanced DDPM (ASDDPM) is
proposed to enhance the detail-preserving capability of the DDPM by
incorporating low-frequency semantic information provided by the Transformer.
Specifically, a novel adaptive diffusion Transformer decoder (ADTD) is
developed to bridge the semantic gap between the encoder and decoder through
regulating the noise prediction with the global contextual relationships and
long-range dependencies in the diffusion process. Additionally, a residual
feature fusion strategy establishes information exchange between the two
decoders at multiple levels. As a result, the predicted noise generated by our
approach closely approximates that of the real noise distribution.Extensive
experiments on two SR and two semantic segmentation datasets confirm the
superior performance of the proposed ASDDPM in both SR and the subsequent
downstream applications. The source code will be available at
https://github.com/littlebeen/ASDDPM-Adaptive-Semantic-Enhanced-DDPM.