Looking at words and points with attention: a benchmark for
  text-to-shape coherence
Andrea Amaduzzi Giuseppe Lisanti Samuele Salti Luigi Di Stefano
While text-conditional 3D object generation and manipulation have seen rapid
progress, the evaluation of coherence between generated 3D shapes and input
textual descriptions lacks a clear benchmark. The reason is twofold: a) the low
quality of the textual descriptions in the only publicly available dataset of
text-shape pairs; b) the limited effectiveness of the metrics used to
quantitatively assess such coherence. In this paper, we propose a comprehensive
solution that addresses both weaknesses. Firstly, we employ large language
models to automatically refine textual descriptions associated with shapes.
Secondly, we propose a quantitative metric to assess text-to-shape coherence,
through cross-attention mechanisms. To validate our approach, we conduct a user
study and compare quantitatively our metric with existing ones. The refined
dataset, the new metric and a set of text-shape pairs validated by the user
study comprise a novel, fine-grained benchmark that we publicly release to
foster research on text-to-shape coherence of text-conditioned 3D generative
models. Benchmark available at
https://cvlab-unibo.github.io/CrossCoherence-Web/.