Automatic Assessment of Oral Reading Accuracy for Reading Diagnostics
Bo Molenaar Cristian Tejedor-Garcia Helmer Strik Catia Cucchiarini
Automatic assessment of reading fluency using automatic speech recognition
(ASR) holds great potential for early detection of reading difficulties and
subsequent timely intervention. Precise assessment tools are required,
especially for languages other than English. In this study, we evaluate six
state-of-the-art ASR-based systems for automatically assessing Dutch oral
reading accuracy using Kaldi and Whisper. Results show our most successful
system reached substantial agreement with human evaluations (MCC = .63). The
same system reached the highest correlation between forced decoding confidence
scores and word correctness (r = .45). This system's language model (LM)
consisted of manual orthographic transcriptions and reading prompts of the test
data, which shows that including reading errors in the LM improves assessment
performance. We discuss the implications for developing automatic assessment
systems and identify possible avenues of future research.