Spectrum Translation for Refinement of Image Generation (STIG) Based on
  Contrastive Learning and Spectral Filter Profile
Seokjun Lee Seung-Won Jung Hyunseok Seo
Currently, image generation and synthesis have remarkably progressed with
generative models. Despite photo-realistic results, intrinsic discrepancies are
still observed in the frequency domain. The spectral discrepancy appeared not
only in generative adversarial networks but in diffusion models. In this study,
we propose a framework to effectively mitigate the disparity in frequency
domain of the generated images to improve generative performance of both GAN
and diffusion models. This is realized by spectrum translation for the
refinement of image generation (STIG) based on contrastive learning. We adopt
theoretical logic of frequency components in various generative networks. The
key idea, here, is to refine the spectrum of the generated image via the
concept of image-to-image translation and contrastive learning in terms of
digital signal processing. We evaluate our framework across eight fake image
datasets and various cutting-edge models to demonstrate the effectiveness of
STIG. Our framework outperforms other cutting-edges showing significant
decreases in FID and log frequency distance of spectrum. We further emphasize
that STIG improves image quality by decreasing the spectral anomaly.
Additionally, validation results present that the frequency-based deepfake
detector confuses more in the case where fake spectrums are manipulated by
STIG.