Task-Oriented Diffusion Model Compression
Geonung Kim Beomsu Kim Eunhyeok Park Sunghyun Cho
As recent advancements in large-scale Text-to-Image (T2I) diffusion models
have yielded remarkable high-quality image generation, diverse downstream
Image-to-Image (I2I) applications have emerged. Despite the impressive results
achieved by these I2I models, their practical utility is hampered by their
large model size and the computational burden of the iterative denoising
process. In this paper, we explore the compression potential of these I2I
models in a task-oriented manner and introduce a novel method for reducing both
model size and the number of timesteps. Through extensive experiments, we
observe key insights and use our empirical knowledge to develop practical
solutions that aim for near-optimal results with minimal exploration costs. We
validate the effectiveness of our method by applying it to InstructPix2Pix for
image editing and StableSR for image restoration. Our approach achieves
satisfactory output quality with 39.2% and 56.4% reduction in model footprint
and 81.4% and 68.7% decrease in latency to InstructPix2Pix and StableSR,
respectively.