Breaking of brightness consistency in optical flow with a lightweight
  CNN network
Yicheng Lin Shuo Wang Yunlong Jiang Bin Han
Sparse optical flow is widely used in various computer vision tasks, however
assuming brightness consistency limits its performance in High Dynamic Range
(HDR) environments. In this work, a lightweight network is used to extract
illumination robust convolutional features and corners with strong invariance.
Modifying the typical brightness consistency of the optical flow method to the
convolutional feature consistency yields the light-robust hybrid optical flow
method. The proposed network runs at 190 FPS on a commercial CPU because it
uses only four convolutional layers to extract feature maps and score maps
simultaneously. Since the shallow network is difficult to train directly, a
deep network is designed to compute the reliability map that helps it. An
end-to-end unsupervised training mode is used for both networks. To validate
the proposed method, we compare corner repeatability and matching performance
with origin optical flow under dynamic illumination. In addition, a more
accurate visual inertial system is constructed by replacing the optical flow
method in VINS-Mono. In a public HDR dataset, it reduces translation errors by
93\%. The code is publicly available at https://github.com/linyicheng1/LET-NET.