Vector quantization loss analysis in VQGANs: a single-GPU ablation study
  for image-to-image synthesis
Luv Verma Varun Mohan
This study performs an ablation analysis of Vector Quantized Generative
Adversarial Networks (VQGANs), concentrating on image-to-image synthesis
utilizing a single NVIDIA A100 GPU. The current work explores the nuanced
effects of varying critical parameters including the number of epochs, image
count, and attributes of codebook vectors and latent dimensions, specifically
within the constraint of limited resources. Notably, our focus is pinpointed on
the vector quantization loss, keeping other hyperparameters and loss components
(GAN loss) fixed. This was done to delve into a deeper understanding of the
discrete latent space, and to explore how varying its size affects the
reconstruction. Though, our results do not surpass the existing benchmarks,
however, our findings shed significant light on VQGAN's behaviour for a smaller
dataset, particularly concerning artifacts, codebook size optimization, and
comparative analysis with Principal Component Analysis (PCA). The study also
uncovers the promising direction by introducing 2D positional encodings,
revealing a marked reduction in artifacts and insights into balancing clarity
and overfitting.