LRRU: Long-short Range Recurrent Updating Networks for Depth Completion
Yufei Wang Bo Li Ge Zhang Qi Liu Tao Gao Yuchao Dai
Existing deep learning-based depth completion methods generally employ
massive stacked layers to predict the dense depth map from sparse input data.
Although such approaches greatly advance this task, their accompanied huge
computational complexity hinders their practical applications. To accomplish
depth completion more efficiently, we propose a novel lightweight deep network
framework, the Long-short Range Recurrent Updating (LRRU) network. Without
learning complex feature representations, LRRU first roughly fills the sparse
input to obtain an initial dense depth map, and then iteratively updates it
through learned spatially-variant kernels. Our iterative update process is
content-adaptive and highly flexible, where the kernel weights are learned by
jointly considering the guidance RGB images and the depth map to be updated,
and large-to-small kernel scopes are dynamically adjusted to capture
long-to-short range dependencies. Our initial depth map has coarse but complete
scene depth information, which helps relieve the burden of directly regressing
the dense depth from sparse ones, while our proposed method can effectively
refine it to an accurate depth map with less learnable parameters and inference
time. Experimental results demonstrate that our proposed LRRU variants achieve
state-of-the-art performance across different parameter regimes. In particular,
the LRRU-Base model outperforms competing approaches on the NYUv2 dataset, and
ranks 1st on the KITTI depth completion benchmark at the time of submission.
Project page: https://npucvr.github.io/LRRU/.