ImGeoNet: Image-induced Geometry-aware Voxel Representation for
  Multi-view 3D Object Detection
Tao Tu Shun-Po Chuang Yu-Lun Liu Cheng Sun Ke Zhang Donna Roy Cheng-Hao Kuo Min Sun
We propose ImGeoNet, a multi-view image-based 3D object detection framework
that models a 3D space by an image-induced geometry-aware voxel representation.
Unlike previous methods which aggregate 2D features into 3D voxels without
considering geometry, ImGeoNet learns to induce geometry from multi-view images
to alleviate the confusion arising from voxels of free space, and during the
inference phase, only images from multiple views are required. Besides, a
powerful pre-trained 2D feature extractor can be leveraged by our
representation, leading to a more robust performance. To evaluate the
effectiveness of ImGeoNet, we conduct quantitative and qualitative experiments
on three indoor datasets, namely ARKitScenes, ScanNetV2, and ScanNet200. The
results demonstrate that ImGeoNet outperforms the current state-of-the-art
multi-view image-based method, ImVoxelNet, on all three datasets in terms of
detection accuracy. In addition, ImGeoNet shows great data efficiency by
achieving results comparable to ImVoxelNet with 100 views while utilizing only
40 views. Furthermore, our studies indicate that our proposed image-induced
geometry-aware representation can enable image-based methods to attain superior
detection accuracy than the seminal point cloud-based method, VoteNet, in two
practical scenarios: (1) scenarios where point clouds are sparse and noisy,
such as in ARKitScenes, and (2) scenarios involve diverse object classes,
particularly classes of small objects, as in the case in ScanNet200.