What to Remember: Self-Adaptive Continual Learning for Audio Deepfake
  Detection
Xiaohui Zhang Jiangyan Yi Chenglong Wang Chuyuan Zhang Siding Zeng Jianhua Tao
The rapid evolution of speech synthesis and voice conversion has raised
substantial concerns due to the potential misuse of such technology, prompting
a pressing need for effective audio deepfake detection mechanisms. Existing
detection models have shown remarkable success in discriminating known deepfake
audio, but struggle when encountering new attack types. To address this
challenge, one of the emergent effective approaches is continual learning. In
this paper, we propose a continual learning approach called Radian Weight
Modification (RWM) for audio deepfake detection. The fundamental concept
underlying RWM involves categorizing all classes into two groups: those with
compact feature distributions across tasks, such as genuine audio, and those
with more spread-out distributions, like various types of fake audio. These
distinctions are quantified by means of the in-class cosine distance, which
subsequently serves as the basis for RWM to introduce a trainable gradient
modification direction for distinct data types. Experimental evaluations
against mainstream continual learning methods reveal the superiority of RWM in
terms of knowledge acquisition and mitigating forgetting in audio deepfake
detection. Furthermore, RWM's applicability extends beyond audio deepfake
detection, demonstrating its potential significance in diverse machine learning
domains such as image recognition.