Box2Poly: Memory-Efficient Polygon Prediction of Arbitrarily Shaped and
  Rotated Text
Xuyang Chen Dong Wang Konrad Schindler Mingwei Sun Yongliang Wang Nicolo Savioli Liqiu Meng
Recently, Transformer-based text detection techniques have sought to predict
polygons by encoding the coordinates of individual boundary vertices using
distinct query features. However, this approach incurs a significant memory
overhead and struggles to effectively capture the intricate relationships
between vertices belonging to the same instance. Consequently, irregular text
layouts often lead to the prediction of outlined vertices, diminishing the
quality of results. To address these challenges, we present an innovative
approach rooted in Sparse R-CNN: a cascade decoding pipeline for polygon
prediction. Our method ensures precision by iteratively refining polygon
predictions, considering both the scale and location of preceding results.
Leveraging this stabilized regression pipeline, even employing just a single
feature vector to guide polygon instance regression yields promising detection
results. Simultaneously, the leverage of instance-level feature proposal
substantially enhances memory efficiency (>50% less vs. the state-of-the-art
method DPText-DETR) and reduces inference speed (>40% less vs. DPText-DETR)
with minor performance drop on benchmarks.