Pixel-Aligned Recurrent Queries for Multi-View 3D Object Detection
Yiming Xie Huaizu Jiang Georgia Gkioxari Julian Straub
We present PARQ - a multi-view 3D object detector with transformer and
pixel-aligned recurrent queries. Unlike previous works that use learnable
features or only encode 3D point positions as queries in the decoder, PARQ
leverages appearance-enhanced queries initialized from reference points in 3D
space and updates their 3D location with recurrent cross-attention operations.
Incorporating pixel-aligned features and cross attention enables the model to
encode the necessary 3D-to-2D correspondences and capture global contextual
information of the input images. PARQ outperforms prior best methods on the
ScanNet and ARKitScenes datasets, learns and detects faster, is more robust to
distribution shifts in reference points, can leverage additional input views
without retraining, and can adapt inference compute by changing the number of
recurrent iterations.