Text-to-Video: a Two-stage Framework for Zero-shot Identity-agnostic
  Talking-head Generation
Zhichao Wang Mengyu Dai Keld Lundgaard
The advent of ChatGPT has introduced innovative methods for information
gathering and analysis. However, the information provided by ChatGPT is limited
to text, and the visualization of this information remains constrained.
Previous research has explored zero-shot text-to-video (TTV) approaches to
transform text into videos. However, these methods lacked control over the
identity of the generated audio, i.e., not identity-agnostic, hindering their
effectiveness. To address this limitation, we propose a novel two-stage
framework for person-agnostic video cloning, specifically focusing on TTV
generation. In the first stage, we leverage pretrained zero-shot models to
achieve text-to-speech (TTS) conversion. In the second stage, an audio-driven
talking head generation method is employed to produce compelling videos
privided the audio generated in the first stage. This paper presents a
comparative analysis of different TTS and audio-driven talking head generation
methods, identifying the most promising approach for future research and
development. Some audio and videos samples can be found in the following link:
https://github.com/ZhichaoWang970201/Text-to-Video/tree/main.