Learning Music Sequence Representation from Text Supervision
Tianyu Chen Yuan Xie Shuai Zhang Shaohan Huang Haoyi Zhou Jianxin Li
Music representation learning is notoriously difficult for its complex
human-related concepts contained in the sequence of numerical signals. To
excavate better MUsic SEquence Representation from labeled audio, we propose a
novel text-supervision pre-training method, namely MUSER. MUSER adopts an
audio-spectrum-text tri-modal contrastive learning framework, where the text
input could be any form of meta-data with the help of text templates while the
spectrum is derived from an audio sequence. Our experiments reveal that MUSER
could be more flexibly adapted to downstream tasks compared with the current
data-hungry pre-training method, and it only requires 0.056% of pre-training
data to achieve the state-of-the-art performance.