A Dataset for Physical and Abstract Plausibility and Sources of Human
  Disagreement
Annerose Eichel Sabine Schulte im Walde
We present a novel dataset for physical and abstract plausibility of events
in English. Based on naturally occurring sentences extracted from Wikipedia, we
infiltrate degrees of abstractness, and automatically generate perturbed
pseudo-implausible events. We annotate a filtered and balanced subset for
plausibility using crowd-sourcing, and perform extensive cleansing to ensure
annotation quality. In-depth quantitative analyses indicate that annotators
favor plausibility over implausibility and disagree more on implausible events.
Furthermore, our plausibility dataset is the first to capture abstractness in
events to the same extent as concreteness, and we find that event abstractness
has an impact on plausibility ratings: more concrete event participants trigger
a perception of implausibility.