Investigating the Nature of Disagreements on Mid-Scale Ratings: A Case
  Study on the Abstractness-Concreteness Continuum
Urban Knupleš Diego Frassinelli Sabine Schulte im Walde
Humans tend to strongly agree on ratings on a scale for extreme cases (e.g.,
a CAT is judged as very concrete), but judgements on mid-scale words exhibit
more disagreement. Yet, collected rating norms are heavily exploited across
disciplines. Our study focuses on concreteness ratings and (i) implements
correlations and supervised classification to identify salient multi-modal
characteristics of mid-scale words, and (ii) applies a hard clustering to
identify patterns of systematic disagreement across raters. Our results suggest
to either fine-tune or filter mid-scale target words before utilising them.