FastCAD: Real-Time CAD Retrieval and Alignment from Scans and Videos
Florian Langer Jihong Ju Georgi Dikov Gerhard Reitmayr Mohsen Ghafoorian
Digitising the 3D world into a clean, CAD model-based representation has
important applications for augmented reality and robotics. Current
state-of-the-art methods are computationally intensive as they individually
encode each detected object and optimise CAD alignments in a second stage. In
this work, we propose FastCAD, a real-time method that simultaneously retrieves
and aligns CAD models for all objects in a given scene. In contrast to previous
works, we directly predict alignment parameters and shape embeddings. We
achieve high-quality shape retrievals by learning CAD embeddings in a
contrastive learning framework and distilling those into FastCAD. Our
single-stage method accelerates the inference time by a factor of 50 compared
to other methods operating on RGB-D scans while outperforming them on the
challenging Scan2CAD alignment benchmark. Further, our approach collaborates
seamlessly with online 3D reconstruction techniques. This enables the real-time
generation of precise CAD model-based reconstructions from videos at 10 FPS.
Doing so, we significantly improve the Scan2CAD alignment accuracy in the video
setting from 43.0% to 48.2% and the reconstruction accuracy from 22.9% to
29.6%.