Improved DeepFake Detection Using Whisper Features
Piotr Kawa Marcin Plata Michał Czuba Piotr Szymański Piotr Syga
With a recent influx of voice generation methods, the threat introduced by
audio DeepFake (DF) is ever-increasing. Several different detection methods
have been presented as a countermeasure. Many methods are based on so-called
front-ends, which, by transforming the raw audio, emphasize features crucial
for assessing the genuineness of the audio sample. Our contribution contains
investigating the influence of the state-of-the-art Whisper automatic speech
recognition model as a DF detection front-end. We compare various combinations
of Whisper and well-established front-ends by training 3 detection models
(LCNN, SpecRNet, and MesoNet) on a widely used ASVspoof 2021 DF dataset and
later evaluating them on the DF In-The-Wild dataset. We show that using
Whisper-based features improves the detection for each model and outperforms
recent results on the In-The-Wild dataset by reducing Equal Error Rate by 21%.