Practical End-to-End Optical Music Recognition for Pianoform Music
Jiří Mayer Milan Straka Jan Hajič jr. Pavel Pecina
The majority of recent progress in Optical Music Recognition (OMR) has been
achieved with Deep Learning methods, especially models following the end-to-end
paradigm, reading input images and producing a linear sequence of tokens.
Unfortunately, many music scores, especially piano music, cannot be easily
converted to a linear sequence. This has led OMR researchers to use custom
linearized encodings, instead of broadly accepted structured formats for music
notation. Their diversity makes it difficult to compare the performance of OMR
systems directly. To bring recent OMR model progress closer to useful results:
(a) We define a sequential format called Linearized MusicXML, allowing to train
an end-to-end model directly and maintaining close cohesion and compatibility
with the industry-standard MusicXML format. (b) We create a dev and test set
for benchmarking typeset OMR with MusicXML ground truth based on the OpenScore
Lieder corpus. They contain 1,438 and 1,493 pianoform systems, each with an
image from IMSLP. (c) We train and fine-tune an end-to-end model to serve as a
baseline on the dataset and employ the TEDn metric to evaluate the model. We
also test our model against the recently published synthetic pianoform dataset
GrandStaff and surpass the state-of-the-art results.