Physics-Inspired Degradation Models for Hyperspectral Image Fusion
Jie Lian Lizhi Wang Lin Zhu Renwei Dian Zhiwei Xiong Hua Huang
The fusion of a low-spatial-resolution hyperspectral image (LR-HSI) with a
high-spatial-resolution multispectral image (HR-MSI) has garnered increasing
research interest. However, most fusion methods solely focus on the fusion
algorithm itself and overlook the degradation models, which results in
unsatisfactory performance in practical scenarios. To fill this gap, we propose
physics-inspired degradation models (PIDM) to model the degradation of LR-HSI
and HR-MSI, which comprises a spatial degradation network (SpaDN) and a
spectral degradation network (SpeDN). SpaDN and SpeDN are designed based on two
insights. First, we employ spatial warping and spectral modulation operations
to simulate lens aberrations, thereby introducing non-uniformity into the
spatial and spectral degradation processes. Second, we utilize asymmetric
downsampling and parallel downsampling operations to separately reduce the
spatial and spectral resolutions of the images, thus ensuring the matching of
spatial and spectral degradation processes with specific physical
characteristics. Once SpaDN and SpeDN are established, we adopt a
self-supervised training strategy to optimize the network parameters and
provide a plug-and-play solution for fusion methods. Comprehensive experiments
demonstrate that our proposed PIDM can boost the fusion performance of existing
fusion methods in practical scenarios.