AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset
Zhixi Cai Shreya Ghosh Aman Pankaj Adatia Munawar Hayat Abhinav Dhall Kalin Stefanov
The detection and localization of highly realistic deepfake audio-visual
content are challenging even for the most advanced state-of-the-art methods.
While most of the research efforts in this domain are focused on detecting
high-quality deepfake images and videos, only a few works address the problem
of the localization of small segments of audio-visual manipulations embedded in
real videos. In this research, we emulate the process of such content
generation and propose the AV-Deepfake1M dataset. The dataset contains
content-driven (i) video manipulations, (ii) audio manipulations, and (iii)
audio-visual manipulations for more than 2K subjects resulting in a total of
more than 1M videos. The paper provides a thorough description of the proposed
data generation pipeline accompanied by a rigorous analysis of the quality of
the generated data. The comprehensive benchmark of the proposed dataset
utilizing state-of-the-art deepfake detection and localization methods
indicates a significant drop in performance compared to previous datasets. The
proposed dataset will play a vital role in building the next-generation
deepfake localization methods. The dataset and associated code are available at
https://github.com/ControlNet/AV-Deepfake1M .