Self-Supervised Motion Magnification by Backpropagating Through Optical
  Flow
Zhaoying Pan Daniel Geng Andrew Owens
This paper presents a simple, self-supervised method for magnifying subtle
motions in video: given an input video and a magnification factor, we
manipulate the video such that its new optical flow is scaled by the desired
amount. To train our model, we propose a loss function that estimates the
optical flow of the generated video and penalizes how far if deviates from the
given magnification factor. Thus, training involves differentiating through a
pretrained optical flow network. Since our model is self-supervised, we can
further improve its performance through test-time adaptation, by finetuning it
on the input video. It can also be easily extended to magnify the motions of
only user-selected objects. Our approach avoids the need for synthetic
magnification datasets that have been used to train prior learning-based
approaches. Instead, it leverages the existing capabilities of off-the-shelf
motion estimators. We demonstrate the effectiveness of our method through
evaluations of both visual quality and quantitative metrics on a range of
real-world and synthetic videos, and we show our method works for both
supervised and unsupervised optical flow methods.