Style Transfer to Calvin and Hobbes comics using Stable Diffusion
Sloke Shrestha Sundar Sripada V. S. Asvin Venkataramanan
This project report summarizes our journey to perform stable diffusion
fine-tuning on a dataset containing Calvin and Hobbes comics. The purpose is to
convert any given input image into the comic style of Calvin and Hobbes,
essentially performing style transfer. We train stable-diffusion-v1.5 using Low
Rank Adaptation (LoRA) to efficiently speed up the fine-tuning process. The
diffusion itself is handled by a Variational Autoencoder (VAE), which is a
U-net. Our results were visually appealing for the amount of training time and
the quality of input data that went into training.