Anyview: Generalizable Indoor 3D Object Detection with Variable Frames
Zhenyu Wu Xiuwei Xu Ziwei Wang Chong Xia Linqing Zhao Jiwen Lu Haibin Yan
In this paper, we propose a novel network framework for indoor 3D object
detection to handle variable input frame numbers in practical scenarios.
Existing methods only consider fixed frames of input data for a single
detector, such as monocular RGB-D images or point clouds reconstructed from
dense multi-view RGB-D images. While in practical application scenes such as
robot navigation and manipulation, the raw input to the 3D detectors is the
RGB-D images with variable frame numbers instead of the reconstructed scene
point cloud. However, the previous approaches can only handle fixed frame input
data and have poor performance with variable frame input. In order to
facilitate 3D object detection methods suitable for practical tasks, we present
a novel 3D detection framework named AnyView for our practical applications,
which generalizes well across different numbers of input frames with a single
model. To be specific, we propose a geometric learner to mine the local
geometric features of each input RGB-D image frame and implement local-global
feature interaction through a designed spatial mixture module. Meanwhile, we
further utilize a dynamic token strategy to adaptively adjust the number of
extracted features for each frame, which ensures consistent global feature
density and further enhances the generalization after fusion. Extensive
experiments on the ScanNet dataset show our method achieves both great
generalizability and high detection accuracy with a simple and clean
architecture containing a similar amount of parameters with the baselines.