GAFlow: Incorporating Gaussian Attention into Optical Flow
Ao Luo Fan Yang Xin Li Lang Nie Chunyu Lin Haoqiang Fan Shuaicheng Liu
Optical flow, or the estimation of motion fields from image sequences, is one
of the fundamental problems in computer vision. Unlike most pixel-wise tasks
that aim at achieving consistent representations of the same category, optical
flow raises extra demands for obtaining local discrimination and smoothness,
which yet is not fully explored by existing approaches. In this paper, we push
Gaussian Attention (GA) into the optical flow models to accentuate local
properties during representation learning and enforce the motion affinity
during matching. Specifically, we introduce a novel Gaussian-Constrained Layer
(GCL) which can be easily plugged into existing Transformer blocks to highlight
the local neighborhood that contains fine-grained structural information.
Moreover, for reliable motion analysis, we provide a new Gaussian-Guided
Attention Module (GGAM) which not only inherits properties from Gaussian
distribution to instinctively revolve around the neighbor fields of each point
but also is empowered to put the emphasis on contextually related regions
during matching. Our fully-equipped model, namely Gaussian Attention Flow
network (GAFlow), naturally incorporates a series of novel Gaussian-based
modules into the conventional optical flow framework for reliable motion
analysis. Extensive experiments on standard optical flow datasets consistently
demonstrate the exceptional performance of the proposed approach in terms of
both generalization ability evaluation and online benchmark testing. Code is
available at https://github.com/LA30/GAFlow.