Don't Push the Button! Exploring Data Leakage Risks in Machine Learning
  and Transfer Learning
Andrea Apicella Francesco Isgrò Roberto Prevete
Machine Learning (ML) has revolutionized various domains, offering predictive
capabilities in several areas. However, with the increasing accessibility of ML
tools, many practitioners, lacking deep ML expertise, adopt a "push the button"
approach, utilizing user-friendly interfaces without a thorough understanding
of underlying algorithms. While this approach provides convenience, it raises
concerns about the reliability of outcomes, leading to challenges such as
incorrect performance evaluation. This paper addresses a critical issue in ML,
known as data leakage, where unintended information contaminates the training
data, impacting model performance evaluation. Users, due to a lack of
understanding, may inadvertently overlook crucial steps, leading to optimistic
performance estimates that may not hold in real-world scenarios. The
discrepancy between evaluated and actual performance on new data is a
significant concern. In particular, this paper categorizes data leakage in ML,
discussing how certain conditions can propagate through the ML workflow.
Furthermore, it explores the connection between data leakage and the specific
task being addressed, investigates its occurrence in Transfer Learning, and
compares standard inductive ML with transductive ML frameworks. The conclusion
summarizes key findings, emphasizing the importance of addressing data leakage
for robust and reliable ML applications.