Mono-to-stereo through parametric stereo generation
Joan Serrà Davide Scaini Santiago Pascual Daniel Arteaga Jordi Pons Jeroen Breebaart Giulio Cengarle
Generating a stereophonic presentation from a monophonic audio signal is a
challenging open task, especially if the goal is to obtain a realistic spatial
imaging with a specific panning of sound elements. In this work, we propose to
convert mono to stereo by means of predicting parametric stereo (PS) parameters
using both nearest neighbor and deep network approaches. In combination with
PS, we also propose to model the task with generative approaches, allowing to
synthesize multiple and equally-plausible stereo renditions from the same mono
signal. To achieve this, we consider both autoregressive and masked token
modelling approaches. We provide evidence that the proposed PS-based models
outperform a competitive classical decorrelation baseline and that, within a PS
prediction framework, modern generative models outshine equivalent
non-generative counterparts. Overall, our work positions both PS and generative
modelling as strong and appealing methodologies for mono-to-stereo upmixing. A
discussion of the limitations of these approaches is also provided.