Revisiting Learning-based Video Motion Magnification for Real-time
  Processing
Hyunwoo Ha Oh Hyun-Bin Kim Jun-Seong Kwon Byung-Ki Kim Sung-Bin Linh-Tam Tran Ji-Yun Kim Sung-Ho Bae Tae-Hyun Oh
Video motion magnification is a technique to capture and amplify subtle
motion in a video that is invisible to the naked eye. The deep learning-based
prior work successfully demonstrates the modelling of the motion magnification
problem with outstanding quality compared to conventional signal
processing-based ones. However, it still lags behind real-time performance,
which prevents it from being extended to various online applications. In this
paper, we investigate an efficient deep learning-based motion magnification
model that runs in real time for full-HD resolution videos. Due to the
specified network design of the prior art, i.e. inhomogeneous architecture, the
direct application of existing neural architecture search methods is
complicated. Instead of automatic search, we carefully investigate the
architecture module by module for its role and importance in the motion
magnification task. Two key findings are 1) Reducing the spatial resolution of
the latent motion representation in the decoder provides a good trade-off
between computational efficiency and task quality, and 2) surprisingly, only a
single linear layer and a single branch in the encoder are sufficient for the
motion magnification task. Based on these findings, we introduce a real-time
deep learning-based motion magnification model with4.2X fewer FLOPs and is 2.7X
faster than the prior art while maintaining comparable quality.