Rhythm Modeling for Voice Conversion
Benjamin van Niekerk Marc-André Carbonneau Herman Kamper
Voice conversion aims to transform source speech into a different target
voice. However, typical voice conversion systems do not account for rhythm,
which is an important factor in the perception of speaker identity. To bridge
this gap, we introduce Urhythmic-an unsupervised method for rhythm conversion
that does not require parallel data or text transcriptions. Using
self-supervised representations, we first divide source audio into segments
approximating sonorants, obstruents, and silences. Then we model rhythm by
estimating speaking rate or the duration distribution of each segment type.
Finally, we match the target speaking rate or rhythm by time-stretching the
speech segments. Experiments show that Urhythmic outperforms existing
unsupervised methods in terms of quality and prosody. Code and checkpoints:
https://github.com/bshall/urhythmic. Audio demo page:
https://ubisoft-laforge.github.io/speech/urhythmic.