Towards High-quality HDR Deghosting with Conditional Diffusion Models
Qingsen Yan Tao Hu Yuan Sun Hao Tang Yu Zhu Wei Dong Luc Van Gool Yanning Zhang
High Dynamic Range (HDR) images can be recovered from several Low Dynamic
Range (LDR) images by existing Deep Neural Networks (DNNs) techniques. Despite
the remarkable progress, DNN-based methods still generate ghosting artifacts
when LDR images have saturation and large motion, which hinders potential
applications in real-world scenarios. To address this challenge, we formulate
the HDR deghosting problem as an image generation that leverages LDR features
as the diffusion model's condition, consisting of the feature condition
generator and the noise predictor. Feature condition generator employs
attention and Domain Feature Alignment (DFA) layer to transform the
intermediate features to avoid ghosting artifacts. With the learned features as
conditions, the noise predictor leverages a stochastic iterative denoising
process for diffusion models to generate an HDR image by steering the sampling
process. Furthermore, to mitigate semantic confusion caused by the saturation
problem of LDR images, we design a sliding window noise estimator to sample
smooth noise in a patch-based manner. In addition, an image space loss is
proposed to avoid the color distortion of the estimated HDR results. We
empirically evaluate our model on benchmark datasets for HDR imaging. The
results demonstrate that our approach achieves state-of-the-art performances
and well generalization to real-world images.