PoseAction: Action Recognition for Patients in the Ward using Deep
  Learning Approaches
Zherui Li Raye Chen-Hua Yeow
Real-time intelligent detection and prediction of subjects' behavior
particularly their movements or actions is critical in the ward. This approach
offers the advantage of reducing in-hospital care costs and improving the
efficiency of healthcare workers, which is especially true for scenarios at
night or during peak admission periods. Therefore, in this work, we propose
using computer vision (CV) and deep learning (DL) methods for detecting
subjects and recognizing their actions. We utilize OpenPose as an accurate
subject detector for recognizing the positions of human subjects in the video
stream. Additionally, we employ AlphAction's Asynchronous Interaction
Aggregation (AIA) network to predict the actions of detected subjects. This
integrated model, referred to as PoseAction, is proposed. At the same time, the
proposed model is further trained to predict 12 common actions in ward areas,
such as staggering, chest pain, and falling down, using medical-related video
clips from the NTU RGB+D and NTU RGB+D 120 datasets. The results demonstrate
that PoseAction achieves the highest classification mAP of 98.72% (IoU@0.5).
Additionally, this study develops an online real-time mode for action
recognition, which strongly supports the clinical translation of PoseAction.
Furthermore, using OpenPose's function for recognizing face key points, we also
implement face blurring, which is a practical solution to address the privacy
protection concerns of patients and healthcare workers. Nevertheless, the
training data for PoseAction is currently limited, particularly in terms of
label diversity. Consequently, the subsequent step involves utilizing a more
diverse dataset (including general actions) to train the model's parameters for
improved generalization.