SparseDC: Depth Completion from sparse and non-uniform inputs
Chen Long Wenxiao Zhang Zhe Chen Haiping Wang Yuan Liu Zhen Cao Zhen Dong Bisheng Yang
We propose SparseDC, a model for Depth Completion of Sparse and non-uniform
depth inputs. Unlike previous methods focusing on completing fixed
distributions on benchmark datasets (e.g., NYU with 500 points, KITTI with 64
lines), SparseDC is specifically designed to handle depth maps with poor
quality in real usage. The key contributions of SparseDC are two-fold. First,
we design a simple strategy, called SFFM, to improve the robustness under
sparse input by explicitly filling the unstable depth features with stable
image features. Second, we propose a two-branch feature embedder to predict
both the precise local geometry of regions with available depth values and
accurate structures in regions with no depth. The key of the embedder is an
uncertainty-based fusion module called UFFM to balance the local and long-term
information extracted by CNNs and ViTs. Extensive indoor and outdoor
experiments demonstrate the robustness of our framework when facing sparse and
non-uniform input depths. The pre-trained model and code are available at
https://github.com/WHU-USI3DV/SparseDC.