The Inner Sentiments of a Thought
Chris Gagne Peter Dayan
Transformer-based large-scale language models (LLMs) are able to generate
highly realistic text. They are duly able to express, and at least implicitly
represent, a wide range of sentiments and color, from the obvious, such as
valence and arousal to the subtle, such as determination and admiration. We
provide a first exploration of these representations and how they can be used
for understanding the inner sentimental workings of single sentences. We train
predictors of the quantiles of the distributions of final sentiments of
sentences from the hidden representations of an LLM applied to prefixes of
increasing lengths. After showing that predictors of distributions of valence,
determination, admiration, anxiety and annoyance are well calibrated, we
provide examples of using these predictors for analyzing sentences,
illustrating, for instance, how even ordinary conjunctions (e.g., "but") can
dramatically alter the emotional trajectory of an utterance. We then show how
to exploit the distributional predictions to generate sentences with sentiments
in the tails of distributions. We discuss the implications of our results for
the inner workings of thoughts, for instance for psychiatric dysfunction.