VideoGigaGAN: Towards Detail-rich Video Super-Resolution
Yiran Xu Taesung Park Richard Zhang Yang Zhou Eli Shechtman Feng Liu Jia-Bin Huang Difan Liu
Video super-resolution (VSR) approaches have shown impressive temporal
consistency in upsampled videos. However, these approaches tend to generate
blurrier results than their image counterparts as they are limited in their
generative capability. This raises a fundamental question: can we extend the
success of a generative image upsampler to the VSR task while preserving the
temporal consistency? We introduce VideoGigaGAN, a new generative VSR model
that can produce videos with high-frequency details and temporal consistency.
VideoGigaGAN builds upon a large-scale image upsampler -- GigaGAN. Simply
inflating GigaGAN to a video model by adding temporal modules produces severe
temporal flickering. We identify several key issues and propose techniques that
significantly improve the temporal consistency of upsampled videos. Our
experiments show that, unlike previous VSR methods, VideoGigaGAN generates
temporally consistent videos with more fine-grained appearance details. We
validate the effectiveness of VideoGigaGAN by comparing it with
state-of-the-art VSR models on public datasets and showcasing video results
with $8\times$ super-resolution.