Learning to Rank Onset-Occurring-Offset Representations for
  Micro-Expression Recognition
Jie Zhu Yuan Zong Jingang Shi Cheng Lu Hongli Chang Wenming Zheng
This paper focuses on the research of micro-expression recognition (MER) and
proposes a flexible and reliable deep learning method called learning to rank
onset-occurring-offset representations (LTR3O). The LTR3O method introduces a
dynamic and reduced-size sequence structure known as 3O, which consists of
onset, occurring, and offset frames, for representing micro-expressions (MEs).
This structure facilitates the subsequent learning of ME-discriminative
features. A noteworthy advantage of the 3O structure is its flexibility, as the
occurring frame is randomly extracted from the original ME sequence without the
need for accurate frame spotting methods. Based on the 3O structures, LTR3O
generates multiple 3O representation candidates for each ME sample and
incorporates well-designed modules to measure and calibrate their emotional
expressiveness. This calibration process ensures that the distribution of these
candidates aligns with that of macro-expressions (MaMs) over time.
Consequently, the visibility of MEs can be implicitly enhanced, facilitating
the reliable learning of more discriminative features for MER. Extensive
experiments were conducted to evaluate the performance of LTR3O using three
widely-used ME databases: CASME II, SMIC, and SAMM. The experimental results
demonstrate the effectiveness and superior performance of LTR3O, particularly
in terms of its flexibility and reliability, when compared to recent
state-of-the-art MER methods.