NLP Systems That Can't Tell Use from Mention Censor Counterspeech, but
  Teaching the Distinction Helps
Kristina Gligoric Myra Cheng Lucia Zheng Esin Durmus Dan Jurafsky
The use of words to convey speaker's intent is traditionally distinguished
from the `mention' of words for quoting what someone said, or pointing out
properties of a word. Here we show that computationally modeling this
use-mention distinction is crucial for dealing with counterspeech online.
Counterspeech that refutes problematic content often mentions harmful language
but is not harmful itself (e.g., calling a vaccine dangerous is not the same as
expressing disapproval of someone for calling vaccines dangerous). We show that
even recent language models fail at distinguishing use from mention, and that
this failure propagates to two key downstream tasks: misinformation and hate
speech detection, resulting in censorship of counterspeech. We introduce
prompting mitigations that teach the use-mention distinction, and show they
reduce these errors. Our work highlights the importance of the use-mention
distinction for NLP and CSS and offers ways to address it.