Spectrum-driven Mixed-frequency Network for Hyperspectral Salient Object
  Detection
Peifu Liu Tingfa Xu Huan Chen Shiyun Zhou Haolin Qin Jianan Li
Hyperspectral salient object detection (HSOD) aims to detect spectrally
salient objects in hyperspectral images (HSIs). However, existing methods
inadequately utilize spectral information by either converting HSIs into
false-color images or converging neural networks with clustering. We propose a
novel approach that fully leverages the spectral characteristics by extracting
two distinct frequency components from the spectrum: low-frequency Spectral
Saliency and high-frequency Spectral Edge. The Spectral Saliency approximates
the region of salient objects, while the Spectral Edge captures edge
information of salient objects. These two complementary components, crucial for
HSOD, are derived by computing from the inter-layer spectral angular distance
of the Gaussian pyramid and the intra-neighborhood spectral angular gradients,
respectively. To effectively utilize this dual-frequency information, we
introduce a novel lightweight Spectrum-driven Mixed-frequency Network (SMN).
SMN incorporates two parameter-free plug-and-play operators, namely Spectral
Saliency Generator and Spectral Edge Operator, to extract the Spectral Saliency
and Spectral Edge components from the input HSI independently. Subsequently,
the Mixed-frequency Attention module, comprised of two frequency-dependent
heads, intelligently combines the embedded features of edge and saliency
information, resulting in a mixed-frequency feature representation.
Furthermore, a saliency-edge-aware decoder progressively scales up the
mixed-frequency feature while preserving rich detail and saliency information
for accurate salient object prediction. Extensive experiments conducted on the
HS-SOD benchmark and our custom dataset HSOD-BIT demonstrate that our SMN
outperforms state-of-the-art methods regarding HSOD performance. Code and
dataset will be available at https://github.com/laprf/SMN.