Balancing Transparency and Risk: The Security and Privacy Risks of
  Open-Source Machine Learning Models
Dominik Hintersdorf Lukas Struppek Kristian Kersting
The field of artificial intelligence (AI) has experienced remarkable progress
in recent years, driven by the widespread adoption of open-source machine
learning models in both research and industry. Considering the
resource-intensive nature of training on vast datasets, many applications opt
for models that have already been trained. Hence, a small number of key players
undertake the responsibility of training and publicly releasing large
pre-trained models, providing a crucial foundation for a wide range of
applications. However, the adoption of these open-source models carries
inherent privacy and security risks that are often overlooked. To provide a
concrete example, an inconspicuous model may conceal hidden functionalities
that, when triggered by specific input patterns, can manipulate the behavior of
the system, such as instructing self-driving cars to ignore the presence of
other vehicles. The implications of successful privacy and security attacks
encompass a broad spectrum, ranging from relatively minor damage like service
interruptions to highly alarming scenarios, including physical harm or the
exposure of sensitive user data. In this work, we present a comprehensive
overview of common privacy and security threats associated with the use of
open-source models. By raising awareness of these dangers, we strive to promote
the responsible and secure use of AI systems.