ESSAformer: Efficient Transformer for Hyperspectral Image
  Super-resolution
Mingjin Zhang Chi Zhang Qiming Zhang Jie Guo Xinbo Gao Jing Zhang
Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a
high-resolution hyperspectral image from a low-resolution observation. However,
the prevailing CNN-based approaches have shown limitations in building
long-range dependencies and capturing interaction information between spectral
features. This results in inadequate utilization of spectral information and
artifacts after upsampling. To address this issue, we propose ESSAformer, an
ESSA attention-embedded Transformer network for single-HSI-SR with an iterative
refining structure. Specifically, we first introduce a robust and
spectral-friendly similarity metric, \ie, the spectral correlation coefficient
of the spectrum (SCC), to replace the original attention matrix and
incorporates inductive biases into the model to facilitate training. Built upon
it, we further utilize the kernelizable attention technique with theoretical
support to form a novel efficient SCC-kernel-based self-attention (ESSA) and
reduce attention computation to linear complexity. ESSA enlarges the receptive
field for features after upsampling without bringing much computation and
allows the model to effectively utilize spatial-spectral information from
different scales, resulting in the generation of more natural high-resolution
images. Without the need for pretraining on large-scale datasets, our
experiments demonstrate ESSA's effectiveness in both visual quality and
quantitative results.