Latent Feature-based Data Splits to Improve Generalisation Evaluation: A
  Hate Speech Detection Case Study
Maike Züfle Verna Dankers Ivan Titov
With the ever-growing presence of social media platforms comes the increased
spread of harmful content and the need for robust hate speech detection
systems. Such systems easily overfit to specific targets and keywords, and
evaluating them without considering distribution shifts that might occur
between train and test data overestimates their benefit. We challenge hate
speech models via new train-test splits of existing datasets that rely on the
clustering of models' hidden representations. We present two split variants
(Subset-Sum-Split and Closest-Split) that, when applied to two datasets using
four pretrained models, reveal how models catastrophically fail on blind spots
in the latent space. This result generalises when developing a split with one
model and evaluating it on another. Our analysis suggests that there is no
clear surface-level property of the data split that correlates with the
decreased performance, which underscores that task difficulty is not always
humanly interpretable. We recommend incorporating latent feature-based splits
in model development and release two splits via the GenBench benchmark.