XPSR: Cross-modal Priors for Diffusion-based Image Super-Resolution
Yunpeng Qu Kun Yuan Kai Zhao Qizhi Xie Jinhua Hao Ming Sun Chao Zhou
Diffusion-based methods, endowed with a formidable generative prior, have
received increasing attention in Image Super-Resolution (ISR) recently.
However, as low-resolution (LR) images often undergo severe degradation, it is
challenging for ISR models to perceive the semantic and degradation
information, resulting in restoration images with incorrect content or
unrealistic artifacts. To address these issues, we propose a
\textit{Cross-modal Priors for Super-Resolution (XPSR)} framework. Within XPSR,
to acquire precise and comprehensive semantic conditions for the diffusion
model, cutting-edge Multimodal Large Language Models (MLLMs) are utilized. To
facilitate better fusion of cross-modal priors, a \textit{Semantic-Fusion
Attention} is raised. To distill semantic-preserved information instead of
undesired degradations, a \textit{Degradation-Free Constraint} is attached
between LR and its high-resolution (HR) counterpart. Quantitative and
qualitative results show that XPSR is capable of generating high-fidelity and
high-realism images across synthetic and real-world datasets. Codes will be
released at \url{https://github.com/qyp2000/XPSR}.