Do You Remember? Overcoming Catastrophic Forgetting for Fake Audio
  Detection
Xiaohui Zhang Jiangyan Yi Jianhua Tao Chenglong Wang Chuyuan Zhang
Current fake audio detection algorithms have achieved promising performances
on most datasets. However, their performance may be significantly degraded when
dealing with audio of a different dataset. The orthogonal weight modification
to overcome catastrophic forgetting does not consider the similarity of genuine
audio across different datasets. To overcome this limitation, we propose a
continual learning algorithm for fake audio detection to overcome catastrophic
forgetting, called Regularized Adaptive Weight Modification (RAWM). When
fine-tuning a detection network, our approach adaptively computes the direction
of weight modification according to the ratio of genuine utterances and fake
utterances. The adaptive modification direction ensures the network can
effectively detect fake audio on the new dataset while preserving its knowledge
of old model, thus mitigating catastrophic forgetting. In addition, genuine
audio collected from quite different acoustic conditions may skew their feature
distribution, so we introduce a regularization constraint to force the network
to remember the old distribution in this regard. Our method can easily be
generalized to related fields, like speech emotion recognition. We also
evaluate our approach across multiple datasets and obtain a significant
performance improvement on cross-dataset experiments.