Serenade: A Model for Human-in-the-loop Automatic Chord Estimation
Hendrik Vincent Koops Gianluca Micchi Ilaria Manco Elio Quinton
Computational harmony analysis is important for MIR tasks such as automatic
segmentation, corpus analysis and automatic chord label estimation. However,
recent research into the ambiguous nature of musical harmony, causing limited
inter-rater agreement, has made apparent that there is a glass ceiling for
common metrics such as accuracy. Commonly, these issues are addressed either in
the training data itself by creating majority-rule annotations or during the
training phase by learning soft targets. We propose a novel alternative
approach in which a human and an autoregressive model together co-create a
harmonic annotation for an audio track. After automatically generating harmony
predictions, a human sparsely annotates parts with low model confidence and the
model then adjusts its predictions following human guidance. We evaluate our
model on a dataset of popular music and we show that, with this
human-in-the-loop approach, harmonic analysis performance improves over a
model-only approach. The human contribution is amplified by the second,
constrained prediction of the model.