NeRF-Supervised Feature Point Detection and Description
Ali Youssef Francisco Vasconcelos
Feature point detection and description is the backbone for various computer
vision applications, such as Structure-from-Motion, visual SLAM, and visual
place recognition. While learning-based methods have surpassed traditional
handcrafted techniques, their training often relies on simplistic
homography-based simulations of multi-view perspectives, limiting model
generalisability. This paper introduces a novel approach leveraging neural
radiance fields (NeRFs) for realistic multi-view training data generation. We
create a diverse multi-view dataset using NeRFs, consisting of indoor and
outdoor scenes. Our proposed methodology adapts state-of-the-art feature
detectors and descriptors to train on NeRF-synthesised views supervised by
perspective projective geometry. Our experiments demonstrate that the proposed
methods achieve competitive or superior performance on standard benchmarks for
relative pose estimation, point cloud registration, and homography estimation
while requiring significantly less training data compared to existing
approaches.