Cross-Modal Video to Body-joints Augmentation for Rehabilitation
  Exercise Quality Assessment
Ali Abedi Mobin Malmirian Shehroz S. Khan
Exercise-based rehabilitation programs have been shown to enhance quality of
life and reduce mortality and rehospitalizations. AI-driven virtual
rehabilitation programs enable patients to complete exercises independently at
home while AI algorithms can analyze exercise data to provide feedback to
patients and report their progress to clinicians. This paper introduces a novel
approach to assessing the quality of rehabilitation exercises using RGB video.
Sequences of skeletal body joints are extracted from consecutive RGB video
frames and analyzed by many-to-one sequential neural networks to evaluate
exercise quality. Existing datasets for exercise rehabilitation lack adequate
samples for training deep sequential neural networks to generalize effectively.
A cross-modal data augmentation approach is proposed to resolve this problem.
Visual augmentation techniques are applied to video data, and body joints
extracted from the resulting augmented videos are used for training sequential
neural networks. Extensive experiments conducted on the KInematic assessment of
MOvement and clinical scores for remote monitoring of physical REhabilitation
(KIMORE) dataset, demonstrate the superiority of the proposed method over
previous baseline approaches. The ablation study highlights a significant
enhancement in exercise quality assessment following cross-modal augmentation.