Spot the Error: Non-autoregressive Graphic Layout Generation with
  Wireframe Locator
Jieru Lin Danqing Huang Tiejun Zhao Dechen Zhan Chin-Yew Lin
Layout generation is a critical step in graphic design to achieve meaningful
compositions of elements. Most previous works view it as a sequence generation
problem by concatenating element attribute tokens (i.e., category, size,
position). So far the autoregressive approach (AR) has achieved promising
results, but is still limited in global context modeling and suffers from error
propagation since it can only attend to the previously generated tokens. Recent
non-autoregressive attempts (NAR) have shown competitive results, which
provides a wider context range and the flexibility to refine with iterative
decoding. However, current works only use simple heuristics to recognize
erroneous tokens for refinement which is inaccurate. This paper first conducts
an in-depth analysis to better understand the difference between the AR and NAR
framework. Furthermore, based on our observation that pixel space is more
sensitive in capturing spatial patterns of graphic layouts (e.g., overlap,
alignment), we propose a learning-based locator to detect erroneous tokens
which takes the wireframe image rendered from the generated layout sequence as
input. We show that it serves as a complementary modality to the element
sequence in object space and contributes greatly to the overall performance.
Experiments on two public datasets show that our approach outperforms both AR
and NAR baselines. Extensive studies further prove the effectiveness of
different modules with interesting findings. Our code will be available at
https://github.com/ffffatgoose/SpotError.