Lessons Learned from Mining the Hugging Face Repository
Joel Castaño Silverio Martínez-Fernández Xavier Franch
The rapidly evolving fields of Machine Learning (ML) and Artificial
Intelligence have witnessed the emergence of platforms like Hugging Face (HF)
as central hubs for model development and sharing. This experience report
synthesizes insights from two comprehensive studies conducted on HF, focusing
on carbon emissions and the evolutionary and maintenance aspects of ML models.
Our objective is to provide a practical guide for future researchers embarking
on mining software repository studies within the HF ecosystem to enhance the
quality of these studies. We delve into the intricacies of the replication
package used in our studies, highlighting the pivotal tools and methodologies
that facilitated our analysis. Furthermore, we propose a nuanced stratified
sampling strategy tailored for the diverse HF Hub dataset, ensuring a
representative and comprehensive analytical approach. The report also
introduces preliminary guidelines, transitioning from repository mining to
cohort studies, to establish causality in repository mining studies,
particularly within the ML model of HF context. This transition is inspired by
existing frameworks and is adapted to suit the unique characteristics of the HF
model ecosystem. Our report serves as a guiding framework for researchers,
contributing to the responsible and sustainable advancement of ML, and
fostering a deeper understanding of the broader implications of ML models.