Band-wise Hyperspectral Image Pansharpening using CNN Model Propagation
Giuseppe Guarino Matteo Ciotola Gemine Vivone Giuseppe Scarpa
Hyperspectral pansharpening is receiving a growing interest since the last
few years as testified by a large number of research papers and challenges. It
consists in a pixel-level fusion between a lower-resolution hyperspectral
datacube and a higher-resolution single-band image, the panchromatic image,
with the goal of providing a hyperspectral datacube at panchromatic resolution.
Thanks to their powerful representational capabilities, deep learning models
have succeeded to provide unprecedented results on many general purpose image
processing tasks. However, when moving to domain specific problems, as in this
case, the advantages with respect to traditional model-based approaches are
much lesser clear-cut due to several contextual reasons. Scarcity of training
data, lack of ground-truth, data shape variability, are some such factors that
limit the generalization capacity of the state-of-the-art deep learning
networks for hyperspectral pansharpening. To cope with these limitations, in
this work we propose a new deep learning method which inherits a simple
single-band unsupervised pansharpening model nested in a sequential band-wise
adaptive scheme, where each band is pansharpened refining the model tuned on
the preceding one. By doing so, a simple model is propagated along the
wavelength dimension, adaptively and flexibly, with no need to have a fixed
number of spectral bands, and, with no need to dispose of large, expensive and
labeled training datasets. The proposed method achieves very good results on
our datasets, outperforming both traditional and deep learning reference
methods. The implementation of the proposed method can be found on
https://github.com/giu-guarino/R-PNN