Symbol as Points: Panoptic Symbol Spotting via Point-based
  Representation
Wenlong Liu Tianyu Yang Yuhan Wang Qizhi Yu Lei Zhang
This work studies the problem of panoptic symbol spotting, which is to spot
and parse both countable object instances (windows, doors, tables, etc.) and
uncountable stuff (wall, railing, etc.) from computer-aided design (CAD)
drawings. Existing methods typically involve either rasterizing the vector
graphics into images and using image-based methods for symbol spotting, or
directly building graphs and using graph neural networks for symbol
recognition. In this paper, we take a different approach, which treats graphic
primitives as a set of 2D points that are locally connected and use point cloud
segmentation methods to tackle it. Specifically, we utilize a point transformer
to extract the primitive features and append a mask2former-like spotting head
to predict the final output. To better use the local connection information of
primitives and enhance their discriminability, we further propose the attention
with connection module (ACM) and contrastive connection learning scheme (CCL).
Finally, we propose a KNN interpolation mechanism for the mask attention module
of the spotting head to better handle primitive mask downsampling, which is
primitive-level in contrast to pixel-level for the image. Our approach, named
SymPoint, is simple yet effective, outperforming recent state-of-the-art method
GAT-CADNet by an absolute increase of 9.6% PQ and 10.4% RQ on the FloorPlanCAD
dataset. The source code and models will be available at
https://github.com/nicehuster/SymPoint.