Investigating the Utility of Surprisal from Large Language Models for
  Speech Synthesis Prosody
Sofoklis Kakouros Juraj Šimko Martti Vainio Antti Suni
This paper investigates the use of word surprisal, a measure of the
predictability of a word in a given context, as a feature to aid speech
synthesis prosody. We explore how word surprisal extracted from large language
models (LLMs) correlates with word prominence, a signal-based measure of the
salience of a word in a given discourse. We also examine how context length and
LLM size affect the results, and how a speech synthesizer conditioned with
surprisal values compares with a baseline system. To evaluate these factors, we
conducted experiments using a large corpus of English text and LLMs of varying
sizes. Our results show that word surprisal and word prominence are moderately
correlated, suggesting that they capture related but distinct aspects of
language use. We find that length of context and size of the LLM impact the
correlations, but not in the direction anticipated, with longer contexts and
larger LLMs generally underpredicting prominent words in a nearly linear
manner. We demonstrate that, in line with these findings, a speech synthesizer
conditioned with surprisal values provides a minimal improvement over the
baseline with the results suggesting a limited effect of using surprisal values
for eliciting appropriate prominence patterns.