Multi-Spectral Image Stitching via Spatial Graph Reasoning
Zhiying Jiang Zengxi Zhang Jinyuan Liu Xin Fan Risheng Liu
Multi-spectral image stitching leverages the complementarity between infrared
and visible images to generate a robust and reliable wide field-of-view (FOV)
scene. The primary challenge of this task is to explore the relations between
multi-spectral images for aligning and integrating multi-view scenes.
Capitalizing on the strengths of Graph Convolutional Networks (GCNs) in
modeling feature relationships, we propose a spatial graph reasoning based
multi-spectral image stitching method that effectively distills the deformation
and integration of multi-spectral images across different viewpoints. To
accomplish this, we embed multi-scale complementary features from the same view
position into a set of nodes. The correspondence across different views is
learned through powerful dense feature embeddings, where both inter- and
intra-correlations are developed to exploit cross-view matching and enhance
inner feature disparity. By introducing long-range coherence along spatial and
channel dimensions, the complementarity of pixel relations and channel
interdependencies aids in the reconstruction of aligned multi-view features,
generating informative and reliable wide FOV scenes. Moreover, we release a
challenging dataset named ChaMS, comprising both real-world and synthetic sets
with significant parallax, providing a new option for comprehensive evaluation.
Extensive experiments demonstrate that our method surpasses the
state-of-the-arts.