WCCNet: Wavelet-integrated CNN with Crossmodal Rearranging Fusion for
  Fast Multispectral Pedestrian Detection
Xingjian Wang Li Chai Jiming Chen Zhiguo Shi
Multispectral pedestrian detection achieves better visibility in challenging
conditions and thus has a broad application in various tasks, for which both
the accuracy and computational cost are of paramount importance. Most existing
approaches treat RGB and infrared modalities equally, typically adopting two
symmetrical CNN backbones for multimodal feature extraction, which ignores the
substantial differences between modalities and brings great difficulty for the
reduction of the computational cost as well as effective crossmodal fusion. In
this work, we propose a novel and efficient framework named WCCNet that is able
to differentially extract rich features of different spectra with lower
computational complexity and semantically rearranges these features for
effective crossmodal fusion. Specifically, the discrete wavelet transform (DWT)
allowing fast inference and training speed is embedded to construct a
dual-stream backbone for efficient feature extraction. The DWT layers of WCCNet
extract frequency components for infrared modality, while the CNN layers
extract spatial-domain features for RGB modality. This methodology not only
significantly reduces the computational complexity, but also improves the
extraction of infrared features to facilitate the subsequent crossmodal fusion.
Based on the well extracted features, we elaborately design the crossmodal
rearranging fusion module (CMRF), which can mitigate spatial misalignment and
merge semantically complementary features of spatially-related local regions to
amplify the crossmodal complementary information. We conduct comprehensive
evaluations on KAIST and FLIR benchmarks, in which WCCNet outperforms
state-of-the-art methods with considerable computational efficiency and
competitive accuracy. We also perform the ablation study and analyze thoroughly
the impact of different components on the performance of WCCNet.