Model-Agnostic Human Preference Inversion in Diffusion Models
Jeeyung Kim Ze Wang Qiang Qiu
Efficient text-to-image generation remains a challenging task due to the high
computational costs associated with the multi-step sampling in diffusion
models. Although distillation of pre-trained diffusion models has been
successful in reducing sampling steps, low-step image generation often falls
short in terms of quality. In this study, we propose a novel sampling design to
achieve high-quality one-step image generation aligning with human preferences,
particularly focusing on exploring the impact of the prior noise distribution.
Our approach, Prompt Adaptive Human Preference Inversion (PAHI), optimizes the
noise distributions for each prompt based on human preferences without the need
for fine-tuning diffusion models. Our experiments showcase that the tailored
noise distributions significantly improve image quality with only a marginal
increase in computational cost. Our findings underscore the importance of noise
optimization and pave the way for efficient and high-quality text-to-image
synthesis.