Deep Learning Model Reuse in the HuggingFace Community: Challenges,
  Benefit and Trends
Mina Taraghi Gianolli Dorcelus Armstrong Foundjem Florian Tambon Foutse Khomh
The ubiquity of large-scale Pre-Trained Models (PTMs) is on the rise,
sparking interest in model hubs, and dedicated platforms for hosting PTMs.
Despite this trend, a comprehensive exploration of the challenges that users
encounter and how the community leverages PTMs remains lacking. To address this
gap, we conducted an extensive mixed-methods empirical study by focusing on
discussion forums and the model hub of HuggingFace, the largest public model
hub. Based on our qualitative analysis, we present a taxonomy of the challenges
and benefits associated with PTM reuse within this community. We then conduct a
quantitative study to track model-type trends and model documentation evolution
over time. Our findings highlight prevalent challenges such as limited guidance
for beginner users, struggles with model output comprehensibility in training
or inference, and a lack of model understanding. We also identified interesting
trends among models where some models maintain high upload rates despite a
decline in topics related to them. Additionally, we found that despite the
introduction of model documentation tools, its quantity has not increased over
time, leading to difficulties in model comprehension and selection among users.
Our study sheds light on new challenges in reusing PTMs that were not reported
before and we provide recommendations for various stakeholders involved in PTM
reuse.