CMOSE: Comprehensive Multi-Modality Online Student Engagement Dataset
  with High-Quality Labels
Chi-hsuan Wu Shih-yang Liu Xijie Huang Xingbo Wang Rong Zhang Luca Minciullo Wong Kai Yiu Kenny Kwan Kwang-Ting Cheng
Online learning is a rapidly growing industry due to its convenience.
However, a major challenge in online learning is whether students are as
engaged as they are in face-to-face classes. An engagement recognition system
can significantly improve the learning experience in online classes. Current
challenges in engagement detection involve poor label quality in the dataset,
intra-class variation, and extreme data imbalance. To address these problems,
we present the CMOSE dataset, which contains a large number of data in
different engagement levels and high-quality labels generated according to the
psychological advice. We demonstrate the advantage of transferability by
analyzing the model performance on other engagement datasets. We also developed
a training mechanism, MocoRank, to handle the intra-class variation, the
ordinal relationship between different classes, and the data imbalance problem.
MocoRank outperforms prior engagement detection losses, achieving a 1.32%
enhancement in overall accuracy and 5.05% improvement in average accuracy. We
further demonstrate the effectiveness of multi-modality by conducting ablation
studies on features such as pre-trained video features, high-level facial
features, and audio features.