Evaluating and Improving Continual Learning in Spoken Language
  Understanding
Muqiao Yang Xiang Li Umberto Cappellazzo Shinji Watanabe Bhiksha Raj
Continual learning has emerged as an increasingly important challenge across
various tasks, including Spoken Language Understanding (SLU). In SLU, its
objective is to effectively handle the emergence of new concepts and evolving
environments. The evaluation of continual learning algorithms typically
involves assessing the model's stability, plasticity, and generalizability as
fundamental aspects of standards. However, existing continual learning metrics
primarily focus on only one or two of the properties. They neglect the overall
performance across all tasks, and do not adequately disentangle the plasticity
versus stability/generalizability trade-offs within the model. In this work, we
propose an evaluation methodology that provides a unified evaluation on
stability, plasticity, and generalizability in continual learning. By employing
the proposed metric, we demonstrate how introducing various knowledge
distillations can improve different aspects of these three properties of the
SLU model. We further show that our proposed metric is more sensitive in
capturing the impact of task ordering in continual learning, making it better
suited for practical use-case scenarios.