CommunityFish: A Poisson-based Document Scaling With Hierarchical
  Clustering
Sami Diaf
Document scaling has been a key component in text-as-data applications for
social scientists and a major field of interest for political researchers, who
aim at uncovering differences between speakers or parties with the help of
different probabilistic and non-probabilistic approaches. Yet, most of these
techniques are either built upon the agnostically bag-of-word hypothesis or use
prior information borrowed from external sources that might embed the results
with a significant bias. If the corpus has long been considered as a collection
of documents, it can also be seen as a dense network of connected words whose
structure could be clustered to differentiate independent groups of words,
based on their co-occurrences in documents, known as communities. This paper
introduces CommunityFish as an augmented version of Wordfish based on a
hierarchical clustering, namely the Louvain algorithm, on the word space to
yield communities as semantic and independent n-grams emerging from the corpus
and use them as an input to Wordfish method, instead of considering the word
space. This strategy emphasizes the interpretability of the results, since
communities have a non-overlapping structure, hence a crucial informative power
in discriminating parties or speakers, in addition to allowing a faster
execution of the Poisson scaling model. Aside from yielding communities,
assumed to be subtopic proxies, the application of this technique outperforms
the classic Wordfish model by highlighting historical developments in the U.S.
State of the Union addresses and was found to replicate the prevailing
political stance in Germany when using the corpus of parties' legislative
manifestos.