D4D: An RGBD diffusion model to boost monocular depth estimation
L. Papa P. Russo I. Amerini
Ground-truth RGBD data are fundamental for a wide range of computer vision
applications; however, those labeled samples are difficult to collect and
time-consuming to produce. A common solution to overcome this lack of data is
to employ graphic engines to produce synthetic proxies; however, those data do
not often reflect real-world images, resulting in poor performance of the
trained models at the inference step. In this paper we propose a novel training
pipeline that incorporates Diffusion4D (D4D), a customized 4-channels diffusion
model able to generate realistic RGBD samples. We show the effectiveness of the
developed solution in improving the performances of deep learning models on the
monocular depth estimation task, where the correspondence between RGB and depth
map is crucial to achieving accurate measurements. Our supervised training
pipeline, enriched by the generated samples, outperforms synthetic and original
data performances achieving an RMSE reduction of (8.2%, 11.9%) and (8.1%, 6.1%)
respectively on the indoor NYU Depth v2 and the outdoor KITTI dataset.