Event-Based Motion Magnification
Yutian Chen Shi Guo Fangzheng Yu Feng Zhang Jinwei Gu Tianfan Xue
Detecting and magnifying imperceptible high-frequency motions in real-world
scenarios has substantial implications for industrial and medical applications.
These motions are characterized by small amplitudes and high frequencies.
Traditional motion magnification methods rely on costly high-speed cameras or
active light sources, which limit the scope of their applications. In this
work, we propose a dual-camera system consisting of an event camera and a
conventional RGB camera for video motion magnification, containing
temporally-dense information from the event stream and spatially-dense data
from the RGB images. This innovative combination enables a broad and
cost-effective amplification of high-frequency motions. By revisiting the
physical camera model, we observe that estimating motion direction and
magnitude necessitates the integration of event streams with additional image
features. On this basis, we propose a novel deep network for event-based video
motion magnification that addresses two primary challenges: firstly, the high
frequency of motion induces a large number of interpolated frames (up to 80),
which our network mitigates with a Second-order Recurrent Propagation module
for better handling of long-term frame interpolations; and secondly, magnifying
subtle motions is sensitive to noise, which we address by utilizing a temporal
filter to amplify motion at specific frequencies and reduce noise impact. We
demonstrate the effectiveness and accuracy of our dual-camera system and
network through extensive experiments in magnifying small-amplitude,
high-frequency motions, offering a cost-effective and flexible solution for
motion detection and magnification.