Detecting out-of-distribution text using topological features of
  transformer-based language models
Andres Pollano Anupam Chaudhuri Anj Simmons
We attempt to detect out-of-distribution (OOD) text samples though applying
Topological Data Analysis (TDA) to attention maps in transformer-based language
models. We evaluate our proposed TDA-based approach for out-of-distribution
detection on BERT, a transformer-based language model, and compare the to a
more traditional OOD approach based on BERT CLS embeddings. We found that our
TDA approach outperforms the CLS embedding approach at distinguishing
in-distribution data (politics and entertainment news articles from HuffPost)
from far out-of-domain samples (IMDB reviews), but its effectiveness
deteriorates with near out-of-domain (CNN/Dailymail) or same-domain (business
news articles from HuffPost) datasets.