DSR-Diff: Depth Map Super-Resolution with Diffusion Model
Yuan Shi Bin Xia Rui Zhu Qingmin Liao Wenming Yang
Color-guided depth map super-resolution (CDSR) improve the spatial resolution
of a low-quality depth map with the corresponding high-quality color map,
benefiting various applications such as 3D reconstruction, virtual reality, and
augmented reality. While conventional CDSR methods typically rely on
convolutional neural networks or transformers, diffusion models (DMs) have
demonstrated notable effectiveness in high-level vision tasks. In this work, we
present a novel CDSR paradigm that utilizes a diffusion model within the latent
space to generate guidance for depth map super-resolution. The proposed method
comprises a guidance generation network (GGN), a depth map super-resolution
network (DSRN), and a guidance recovery network (GRN). The GGN is specifically
designed to generate the guidance while managing its compactness. Additionally,
we integrate a simple but effective feature fusion module and a
transformer-style feature extraction module into the DSRN, enabling it to
leverage guided priors in the extraction, fusion, and reconstruction of
multi-model images. Taking into account both accuracy and efficiency, our
proposed method has shown superior performance in extensive experiments when
compared to state-of-the-art methods. Our codes will be made available at
https://github.com/shiyuan7/DSR-Diff.