MERTech: Instrument Playing Technique Detection Using Self-Supervised
  Pretrained Model With Multi-Task Finetuning
Dichucheng Li Yinghao Ma Weixing Wei Qiuqiang Kong Yulun Wu Mingjin Che Fan Xia Emmanouil Benetos Wei Li
Instrument playing techniques (IPTs) constitute a pivotal component of
musical expression. However, the development of automatic IPT detection methods
suffers from limited labeled data and inherent class imbalance issues. In this
paper, we propose to apply a self-supervised learning model pre-trained on
large-scale unlabeled music data and finetune it on IPT detection tasks. This
approach addresses data scarcity and class imbalance challenges. Recognizing
the significance of pitch in capturing the nuances of IPTs and the importance
of onset in locating IPT events, we investigate multi-task finetuning with
pitch and onset detection as auxiliary tasks. Additionally, we apply a
post-processing approach for event-level prediction, where an IPT activation
initiates an event only if the onset output confirms an onset in that frame.
Our method outperforms prior approaches in both frame-level and event-level
metrics across multiple IPT benchmark datasets. Further experiments demonstrate
the efficacy of multi-task finetuning on each IPT class.