Adapting the NICT-JLE Corpus for Disfluency Detection Models
Lucy Skidmore Roger K. Moore
The detection of disfluencies such as hesitations, repetitions and false
starts commonly found in speech is a widely studied area of research. With a
standardised process for evaluation using the Switchboard Corpus, model
performance can be easily compared across approaches. This is not the case for
disfluency detection research on learner speech, however, where such datasets
have restricted access policies, making comparison and subsequent development
of improved models more challenging. To address this issue, this paper
describes the adaptation of the NICT-JLE corpus, containing approximately 300
hours of English learners' oral proficiency tests, to a format that is suitable
for disfluency detection model training and evaluation. Points of difference
between the NICT-JLE and Switchboard corpora are explored, followed by a
detailed overview of adaptations to the tag set and meta-features of the
NICT-JLE corpus. The result of this work provides a standardised train, heldout
and test set for use in future research on disfluency detection for learner
speech.