RMES: Real-Time Micro-Expression Spotting Using Phase From Riesz Pyramid
Yini Fang Didan Deng Liang Wu Frederic Jumelle Bertram Shi
Micro-expressions (MEs) are involuntary and subtle facial expressions that
are thought to reveal feelings people are trying to hide. ME spotting detects
the temporal intervals containing MEs in videos. Detecting such quick and
subtle motions from long videos is difficult. Recent works leverage detailed
facial motion representations, such as the optical flow, and deep learning
models, leading to high computational complexity. To reduce computational
complexity and achieve real-time operation, we propose RMES, a real-time ME
spotting framework. We represent motion using phase computed by Riesz Pyramid,
and feed this motion representation into a three-stream shallow CNN, which
predicts the likelihood of each frame belonging to an ME. In comparison to
optical flow, phase provides more localized motion estimates, which are
essential for ME spotting, resulting in higher performance. Using phase also
reduces the required computation of the ME spotting pipeline by 77.8%. Despite
its relative simplicity and low computational complexity, our framework
achieves state-of-the-art performance on two public datasets: CAS(ME)2 and SAMM
Long Videos.