SpectralMamba: Efficient Mamba for Hyperspectral Image Classification
Jing Yao Danfeng Hong Chenyu Li Jocelyn Chanussot
Recurrent neural networks and Transformers have recently dominated most
applications in hyperspectral (HS) imaging, owing to their capability to
capture long-range dependencies from spectrum sequences. However, despite the
success of these sequential architectures, the non-ignorable inefficiency
caused by either difficulty in parallelization or computationally prohibitive
attention still hinders their practicality, especially for large-scale
observation in remote sensing scenarios. To address this issue, we herein
propose SpectralMamba -- a novel state space model incorporated efficient deep
learning framework for HS image classification. SpectralMamba features the
simplified but adequate modeling of HS data dynamics at two levels. First, in
spatial-spectral space, a dynamical mask is learned by efficient convolutions
to simultaneously encode spatial regularity and spectral peculiarity, thus
attenuating the spectral variability and confusion in discriminative
representation learning. Second, the merged spectrum can then be efficiently
operated in the hidden state space with all parameters learned input-dependent,
yielding selectively focused responses without reliance on redundant attention
or imparallelizable recurrence. To explore the room for further computational
downsizing, a piece-wise scanning mechanism is employed in-between,
transferring approximately continuous spectrum into sequences with squeezed
length while maintaining short- and long-term contextual profiles among
hundreds of bands. Through extensive experiments on four benchmark HS datasets
acquired by satellite-, aircraft-, and UAV-borne imagers, SpectralMamba
surprisingly creates promising win-wins from both performance and efficiency
perspectives.