Multiview Transformer: Rethinking Spatial Information in Hyperspectral
  Image Classification
Jie Zhang Yongshan Zhang Yicong Zhou
Identifying the land cover category for each pixel in a hyperspectral image
(HSI) relies on spectral and spatial information. An HSI cuboid with a specific
patch size is utilized to extract spatial-spectral feature representation for
the central pixel. In this article, we investigate that scene-specific but not
essential correlations may be recorded in an HSI cuboid. This additional
information improves the model performance on existing HSI datasets and makes
it hard to properly evaluate the ability of a model. We refer to this problem
as the spatial overfitting issue and utilize strict experimental settings to
avoid it. We further propose a multiview transformer for HSI classification,
which consists of multiview principal component analysis (MPCA), spectral
encoder-decoder (SED), and spatial-pooling tokenization transformer (SPTT).
MPCA performs dimension reduction on an HSI via constructing spectral multiview
observations and applying PCA on each view data to extract low-dimensional view
representation. The combination of view representations, named multiview
representation, is the dimension reduction output of the MPCA. To aggregate the
multiview information, a fully-convolutional SED with a U-shape in spectral
dimension is introduced to extract a multiview feature map. SPTT transforms the
multiview features into tokens using the spatial-pooling tokenization strategy
and learns robust and discriminative spatial-spectral features for land cover
identification. Classification is conducted with a linear classifier.
Experiments on three HSI datasets with rigid settings demonstrate the
superiority of the proposed multiview transformer over the state-of-the-art
methods.