FMRT: Learning Accurate Feature Matching with Reconciliatory Transformer
Xinyu Zhang Li Wang Zhiqiang Jiang Kun Dai Tao Xie Lei Yang Wenhao Yu Yang Shen Jun Li
Local Feature Matching, an essential component of several computer vision
tasks (e.g., structure from motion and visual localization), has been
effectively settled by Transformer-based methods. However, these methods only
integrate long-range context information among keypoints with a fixed receptive
field, which constrains the network from reconciling the importance of features
with different receptive fields to realize complete image perception, hence
limiting the matching accuracy. In addition, these methods utilize a
conventional handcrafted encoding approach to integrate the positional
information of keypoints into the visual descriptors, which limits the
capability of the network to extract reliable positional encoding message. In
this study, we propose Feature Matching with Reconciliatory Transformer (FMRT),
a novel Transformer-based detector-free method that reconciles different
features with multiple receptive fields adaptively and utilizes parallel
networks to realize reliable positional encoding. Specifically, FMRT proposes a
dedicated Reconciliatory Transformer (RecFormer) that consists of a Global
Perception Attention Layer (GPAL) to extract visual descriptors with different
receptive fields and integrate global context information under various scales,
Perception Weight Layer (PWL) to measure the importance of various receptive
fields adaptively, and Local Perception Feed-forward Network (LPFFN) to extract
deep aggregated multi-scale local feature representation. Extensive experiments
demonstrate that FMRT yields extraordinary performance on multiple benchmarks,
including pose estimation, visual localization, homography estimation, and
image matching.