RLSAC: Reinforcement Learning enhanced Sample Consensus for End-to-End
  Robust Estimation
Chang Nie Guangming Wang Zhe Liu Luca Cavalli Marc Pollefeys Hesheng Wang
Robust estimation is a crucial and still challenging task, which involves
estimating model parameters in noisy environments. Although conventional
sampling consensus-based algorithms sample several times to achieve robustness,
these algorithms cannot use data features and historical information
effectively. In this paper, we propose RLSAC, a novel Reinforcement Learning
enhanced SAmple Consensus framework for end-to-end robust estimation. RLSAC
employs a graph neural network to utilize both data and memory features to
guide exploring directions for sampling the next minimum set. The feedback of
downstream tasks serves as the reward for unsupervised training. Therefore,
RLSAC can avoid differentiating to learn the features and the feedback of
downstream tasks for end-to-end robust estimation. In addition, RLSAC
integrates a state transition module that encodes both data and memory
features. Our experimental results demonstrate that RLSAC can learn from
features to gradually explore a better hypothesis. Through analysis, it is
apparent that RLSAC can be easily transferred to other sampling consensus-based
robust estimation tasks. To the best of our knowledge, RLSAC is also the first
method that uses reinforcement learning to sample consensus for end-to-end
robust estimation. We release our codes at https://github.com/IRMVLab/RLSAC.