Speaker Embeddings as Individuality Proxy for Voice Stress Detection
Zihan Wu Neil Scheidwasser-Clow Karl El Hajal Milos Cernak
Since the mental states of the speaker modulate speech, stress introduced by
cognitive or physical loads could be detected in the voice. The existing voice
stress detection benchmark has shown that the audio embeddings extracted from
the Hybrid BYOL-S self-supervised model perform well. However, the benchmark
only evaluates performance separately on each dataset, but does not evaluate
performance across the different types of stress and different languages.
Moreover, previous studies found strong individual differences in stress
susceptibility. This paper presents the design and development of voice stress
detection, trained on more than 100 speakers from 9 language groups and five
different types of stress. We address individual variabilities in voice stress
analysis by adding speaker embeddings to the hybrid BYOL-S features. The
proposed method significantly improves voice stress detection performance with
an input audio length of only 3-5 seconds.