RBF Weighted Hyper-Involution for RGB-D Object Detection
Mehfuz A Rahman Jiju Peethambaran Neil London
A vast majority of conventional augmented reality devices are equipped with
depth sensors. Depth images produced by such sensors contain complementary
information for object detection when used with color images. Despite the
benefits, it remains a complex task to simultaneously extract photometric and
depth features in real time due to the immanent difference between depth and
color images. Moreover, standard convolution operations are not sufficient to
properly extract information directly from raw depth images leading to
intermediate representations of depth which is inefficient. To address these
issues, we propose a real-time and two stream RGBD object detection model. The
proposed model consists of two new components: a depth guided hyper-involution
that adapts dynamically based on the spatial interaction pattern in the raw
depth map and an up-sampling based trainable fusion layer that combines the
extracted depth and color image features without blocking the information
transfer between them. We show that the proposed model outperforms other RGB-D
based object detection models on NYU Depth v2 dataset and achieves comparable
(second best) results on SUN RGB-D. Additionally, we introduce a new outdoor
RGB-D object detection dataset where our proposed model outperforms other
models. The performance evaluation on diverse synthetic data generated from CAD
models and images shows the potential of the proposed model to be adapted to
augmented reality based applications.