A Concise but Effective Network for Image Guided Depth Completion in
  Autonomous Driving
Moyun Liu Youping Chen Jingming Xie Lei Yao Yang Zhang Joey Tianyi Zhou
Depth completion is a crucial task in autonomous driving, aiming to convert a
sparse depth map into a dense depth prediction. Due to its potentially rich
semantic information, RGB image is commonly fused to enhance the completion
effect. Image-guided depth completion involves three key challenges: 1) how to
effectively fuse the two modalities; 2) how to better recover depth
information; and 3) how to achieve real-time prediction for practical
autonomous driving. To solve the above problems, we propose a concise but
effective network, named CENet, to achieve high-performance depth completion
with a simple and elegant structure. Firstly, we use a fast guidance module to
fuse the two sensor features, utilizing abundant auxiliary features extracted
from the color space. Unlike other commonly used complicated guidance modules,
our approach is intuitive and low-cost. In addition, we find and analyze the
optimization inconsistency problem for observed and unobserved positions, and a
decoupled depth prediction head is proposed to alleviate the issue. The
proposed decoupled head can better output the depth of valid and invalid
positions with very few extra inference time. Based on the simple structure of
dual-encoder and single-decoder, our CENet can achieve superior balance between
accuracy and efficiency. In the KITTI depth completion benchmark, our CENet
attains competitive performance and inference speed compared with the
state-of-the-art methods. To validate the generalization of our method, we also
evaluate on indoor NYUv2 dataset, and our CENet still achieve impressive
results. The code of this work will be available at
https://github.com/lmomoy/CENet.