Local Consensus Enhanced Siamese Network with Reciprocal Loss for
  Two-view Correspondence Learning
Linbo Wang Jing Wu Xianyong Fang Zhengyi Liu Chenjie Cao Yanwei Fu
Recent studies of two-view correspondence learning usually establish an
end-to-end network to jointly predict correspondence reliability and relative
pose. We improve such a framework from two aspects. First, we propose a Local
Feature Consensus (LFC) plugin block to augment the features of existing
models. Given a correspondence feature, the block augments its neighboring
features with mutual neighborhood consensus and aggregates them to produce an
enhanced feature. As inliers obey a uniform cross-view transformation and share
more consistent learned features than outliers, feature consensus strengthens
inlier correlation and suppresses outlier distraction, which makes output
features more discriminative for classifying inliers/outliers. Second, existing
approaches supervise network training with the ground truth correspondences and
essential matrix projecting one image to the other for an input image pair,
without considering the information from the reverse mapping. We extend
existing models to a Siamese network with a reciprocal loss that exploits the
supervision of mutual projection, which considerably promotes the matching
performance without introducing additional model parameters. Building upon
MSA-Net, we implement the two proposals and experimentally achieve
state-of-the-art performance on benchmark datasets.