Mi-Go: Test Framework which uses YouTube as Data Source for Evaluating
  Speech Recognition Models like OpenAI's Whisper
Tomasz Wojnar Jaroslaw Hryszko Adam Roman
This article introduces Mi-Go, a novel testing framework aimed at evaluating
the performance and adaptability of general-purpose speech recognition machine
learning models across diverse real-world scenarios. The framework leverages
YouTube as a rich and continuously updated data source, accounting for multiple
languages, accents, dialects, speaking styles, and audio quality levels. To
demonstrate the effectiveness of the framework, the Whisper model, developed by
OpenAI, was employed as a test object. The tests involve using a total of 124
YouTube videos to test all Whisper model versions. The results underscore the
utility of YouTube as a valuable testing platform for speech recognition
models, ensuring their robustness, accuracy, and adaptability to diverse
languages and acoustic conditions. Additionally, by contrasting the
machine-generated transcriptions against human-made subtitles, the Mi-Go
framework can help pinpoint potential misuse of YouTube subtitles, like Search
Engine Optimization.