Reproducibility in Machine Learning-Driven Research
Harald Semmelrock Simone Kopeinik Dieter Theiler Tony Ross-Hellauer Dominik Kowald
Research is facing a reproducibility crisis, in which the results and
findings of many studies are difficult or even impossible to reproduce. This is
also the case in machine learning (ML) and artificial intelligence (AI)
research. Often, this is the case due to unpublished data and/or source-code,
and due to sensitivity to ML training conditions. Although different solutions
to address this issue are discussed in the research community such as using ML
platforms, the level of reproducibility in ML-driven research is not increasing
substantially. Therefore, in this mini survey, we review the literature on
reproducibility in ML-driven research with three main aims: (i) reflect on the
current situation of ML reproducibility in various research fields, (ii)
identify reproducibility issues and barriers that exist in these research
fields applying ML, and (iii) identify potential drivers such as tools,
practices, and interventions that support ML reproducibility. With this, we
hope to contribute to decisions on the viability of different solutions for
supporting ML reproducibility.