Neuro-Symbolic Approaches for Context-Aware Human Activity Recognition
Luca Arrotta Gabriele Civitarese Claudio Bettini
Deep Learning models are a standard solution for sensor-based Human Activity
Recognition (HAR), but their deployment is often limited by labeled data
scarcity and models' opacity. Neuro-Symbolic AI (NeSy) provides an interesting
research direction to mitigate these issues by infusing knowledge about context
information into HAR deep learning classifiers. However, existing NeSy methods
for context-aware HAR require computationally expensive symbolic reasoners
during classification, making them less suitable for deployment on
resource-constrained devices (e.g., mobile devices). Additionally, NeSy
approaches for context-aware HAR have never been evaluated on in-the-wild
datasets, and their generalization capabilities in real-world scenarios are
questionable. In this work, we propose a novel approach based on a semantic
loss function that infuses knowledge constraints in the HAR model during the
training phase, avoiding symbolic reasoning during classification. Our results
on scripted and in-the-wild datasets show the impact of different semantic loss
functions in outperforming a purely data-driven model. We also compare our
solution with existing NeSy methods and analyze each approach's strengths and
weaknesses. Our semantic loss remains the only NeSy solution that can be
deployed as a single DNN without the need for symbolic reasoning modules,
reaching recognition rates close (and better in some cases) to existing
approaches.