XoFTR: Cross-modal Feature Matching Transformer
Önder Tuzcuoğlu Aybora Köksal Buğra Sofu Sinan Kalkan A. Aydın Alatan
We introduce, XoFTR, a cross-modal cross-view method for local feature
matching between thermal infrared (TIR) and visible images. Unlike visible
images, TIR images are less susceptible to adverse lighting and weather
conditions but present difficulties in matching due to significant texture and
intensity differences. Current hand-crafted and learning-based methods for
visible-TIR matching fall short in handling viewpoint, scale, and texture
diversities. To address this, XoFTR incorporates masked image modeling
pre-training and fine-tuning with pseudo-thermal image augmentation to handle
the modality differences. Additionally, we introduce a refined matching
pipeline that adjusts for scale discrepancies and enhances match reliability
through sub-pixel level refinement. To validate our approach, we collect a
comprehensive visible-thermal dataset, and show that our method outperforms
existing methods on many benchmarks.