Balancing Continual Learning and Fine-tuning for Human Activity
  Recognition
Chi Ian Tang Lorena Qendro Dimitris Spathis Fahim Kawsar Akhil Mathur Cecilia Mascolo
Wearable-based Human Activity Recognition (HAR) is a key task in
human-centric machine learning due to its fundamental understanding of human
behaviours. Due to the dynamic nature of human behaviours, continual learning
promises HAR systems that are tailored to users' needs. However, because of the
difficulty in collecting labelled data with wearable sensors, existing
approaches that focus on supervised continual learning have limited
applicability, while unsupervised continual learning methods only handle
representation learning while delaying classifier training to a later stage.
This work explores the adoption and adaptation of CaSSLe, a continual
self-supervised learning model, and Kaizen, a semi-supervised continual
learning model that balances representation learning and down-stream
classification, for the task of wearable-based HAR. These schemes re-purpose
contrastive learning for knowledge retention and, Kaizen combines that with
self-training in a unified scheme that can leverage unlabelled and labelled
data for continual learning. In addition to comparing state-of-the-art
self-supervised continual learning schemes, we further investigated the
importance of different loss terms and explored the trade-off between knowledge
retention and learning from new tasks. In particular, our extensive evaluation
demonstrated that the use of a weighting factor that reflects the ratio between
learned and new classes achieves the best overall trade-off in continual
learning.