Enhancing 3D Object Detection with 2D Detection-Guided Query Anchors
Haoxuanye Ji Pengpeng Liang Erkang Cheng
Multi-camera-based 3D object detection has made notable progress in the past
several years. However, we observe that there are cases (e.g. faraway regions)
in which popular 2D object detectors are more reliable than state-of-the-art 3D
detectors. In this paper, to improve the performance of query-based 3D object
detectors, we present a novel query generating approach termed QAF2D, which
infers 3D query anchors from 2D detection results. A 2D bounding box of an
object in an image is lifted to a set of 3D anchors by associating each sampled
point within the box with depth, yaw angle, and size candidates. Then, the
validity of each 3D anchor is verified by comparing its projection in the image
with its corresponding 2D box, and only valid anchors are kept and used to
construct queries. The class information of the 2D bounding box associated with
each query is also utilized to match the predicted boxes with ground truth for
the set-based loss. The image feature extraction backbone is shared between the
3D detector and 2D detector by adding a small number of prompt parameters. We
integrate QAF2D into three popular query-based 3D object detectors and carry
out comprehensive evaluations on the nuScenes dataset. The largest improvement
that QAF2D can bring about on the nuScenes validation subset is $2.3\%$ NDS and
$2.7\%$ mAP. Code is available at https://github.com/nullmax-vision/QAF2D.