Mitigating Catastrophic Forgetting in Multi-domain Chinese Spelling
  Correction by Multi-stage Knowledge Transfer Framework
Peng Xing Yinghui Li Shirong Ma Xinnian Liang Haojing Huang Yangning Li Hai-Tao Zheng Wenhao Jiang Ying Shen
Chinese Spelling Correction (CSC) aims to detect and correct spelling errors
in given sentences. Recently, multi-domain CSC has gradually attracted the
attention of researchers because it is more practicable. In this paper, we
focus on the key flaw of the CSC model when adapting to multi-domain scenarios:
the tendency to forget previously acquired knowledge upon learning new
domain-specific knowledge (i.e., catastrophic forgetting). To address this, we
propose a novel model-agnostic Multi-stage Knowledge Transfer (MKT) framework,
which utilizes a continuously evolving teacher model for knowledge transfer in
each domain, rather than focusing solely on new domain knowledge. It deserves
to be mentioned that we are the first to apply continual learning methods to
the multi-domain CSC task. Experiments prove the effectiveness of our proposed
method, and further analyses demonstrate the importance of overcoming
catastrophic forgetting for improving the model performance.