From Detection to Action Recognition: An Edge-Based Pipeline for Robot
  Human Perception
Petros Toupas Georgios Tsamis Dimitrios Giakoumis Konstantinos Votis Dimitrios Tzovaras
Mobile service robots are proving to be increasingly effective in a range of
applications, such as healthcare, monitoring Activities of Daily Living (ADL),
and facilitating Ambient Assisted Living (AAL). These robots heavily rely on
Human Action Recognition (HAR) to interpret human actions and intentions.
However, for HAR to function effectively on service robots, it requires prior
knowledge of human presence (human detection) and identification of individuals
to monitor (human tracking). In this work, we propose an end-to-end pipeline
that encompasses the entire process, starting from human detection and
tracking, leading to action recognition. The pipeline is designed to operate in
near real-time while ensuring all stages of processing are performed on the
edge, reducing the need for centralised computation. To identify the most
suitable models for our mobile robot, we conducted a series of experiments
comparing state-of-the-art solutions based on both their detection performance
and efficiency. To evaluate the effectiveness of our proposed pipeline, we
proposed a dataset comprising daily household activities. By presenting our
findings and analysing the results, we demonstrate the efficacy of our approach
in enabling mobile robots to understand and respond to human behaviour in
real-world scenarios relying mainly on the data from their RGB cameras.