YOIO: You Only Iterate Once by mining and fusing multiple necessary
  global information in the optical flow estimation
Yu Jing Tan Yujuan Ren Ao Liu Duo
Occlusions pose a significant challenge to optical flow algorithms that even
rely on global evidences. We consider an occluded point to be one that is
imaged in the reference frame but not in the next. Estimating the motion of
these points is extremely difficult, particularly in the two-frame setting.
Previous work only used the current frame as the only input, which could not
guarantee providing correct global reference information for occluded points,
and had problems such as long calculation time and poor accuracy in predicting
optical flow at occluded points. To enable both high accuracy and efficiency,
We fully mine and utilize the spatiotemporal information provided by the frame
pair, design a loopback judgment algorithm to ensure that correct global
reference information is obtained, mine multiple necessary global information,
and design an efficient refinement module that fuses these global information.
Specifically, we propose a YOIO framework, which consists of three main
components: an initial flow estimator, a multiple global information extraction
module, and a unified refinement module. We demonstrate that optical flow
estimates in the occluded regions can be significantly improved in only one
iteration without damaging the performance in non-occluded regions. Compared
with GMA, the optical flow prediction accuracy of this method in the occluded
area is improved by more than 10%, and the occ_out area exceeds 15%, while the
calculation time is 27% shorter. This approach, running up to 18.9fps with
436*1024 image resolution, obtains new state-of-the-art results on the
challenging Sintel dataset among all published and unpublished approaches that
can run in real-time, suggesting a new paradigm for accurate and efficient
optical flow estimation.