Quantifying Overfitting: Evaluating Neural Network Performance through
  Analysis of Null Space
Hossein Rezaei Mohammad Sabokrou
Machine learning models that are overfitted/overtrained are more vulnerable
to knowledge leakage, which poses a risk to privacy. Suppose we download or
receive a model from a third-party collaborator without knowing its training
accuracy. How can we determine if it has been overfitted or overtrained on its
training data? It's possible that the model was intentionally over-trained to
make it vulnerable during testing. While an overfitted or overtrained model may
perform well on testing data and even some generalization tests, we can't be
sure it's not over-fitted. Conducting a comprehensive generalization test is
also expensive. The goal of this paper is to address these issues and ensure
the privacy and generalization of our method using only testing data. To
achieve this, we analyze the null space in the last layer of neural networks,
which enables us to quantify overfitting without access to training data or
knowledge of the accuracy of those data. We evaluated our approach on various
architectures and datasets and observed a distinct pattern in the angle of null
space when models are overfitted. Furthermore, we show that models with poor
generalization exhibit specific characteristics in this space. Our work
represents the first attempt to quantify overfitting without access to training
data or knowing any knowledge about the training samples.