SciFlow: Empowering Lightweight Optical Flow Models with Self-Cleaning
  Iterations
Jamie Menjay Lin Jisoo Jeong Hong Cai Risheek Garrepalli Kai Wang Fatih Porikli
Optical flow estimation is crucial to a variety of vision tasks. Despite
substantial recent advancements, achieving real-time on-device optical flow
estimation remains a complex challenge. First, an optical flow model must be
sufficiently lightweight to meet computation and memory constraints to ensure
real-time performance on devices. Second, the necessity for real-time on-device
operation imposes constraints that weaken the model's capacity to adequately
handle ambiguities in flow estimation, thereby intensifying the difficulty of
preserving flow accuracy. This paper introduces two synergistic techniques,
Self-Cleaning Iteration (SCI) and Regression Focal Loss (RFL), designed to
enhance the capabilities of optical flow models, with a focus on addressing
optical flow regression ambiguities. These techniques prove particularly
effective in mitigating error propagation, a prevalent issue in optical flow
models that employ iterative refinement. Notably, these techniques add
negligible to zero overhead in model parameters and inference latency, thereby
preserving real-time on-device efficiency. The effectiveness of our proposed
SCI and RFL techniques, collectively referred to as SciFlow for brevity, is
demonstrated across two distinct lightweight optical flow model architectures
in our experiments. Remarkably, SciFlow enables substantial reduction in error
metrics (EPE and Fl-all) over the baseline models by up to 6.3% and 10.5% for
in-domain scenarios and by up to 6.2% and 13.5% for cross-domain scenarios on
the Sintel and KITTI 2015 datasets, respectively.