Understanding Generalization via Set Theory
Shiqi Liu
Generalization is at the core of machine learning models. However, the
definition of generalization is not entirely clear. We employ set theory to
introduce the concepts of algorithms, hypotheses, and dataset generalization.
We analyze the properties of dataset generalization and prove a theorem on
surrogate generalization procedures. This theorem leads to our generalization
method. Through a generalization experiment on the MNIST dataset, we obtain
13,541 sample bases. When we use the entire training set to evaluate the
model's performance, the models achieve an accuracy of 99.945%. However, if we
shift the sample bases or modify the neural network structure, the performance
experiences a significant decline. We also identify consistently mispredicted
samples and find that they are all challenging examples. The experiments
substantiated the accuracy of the generalization definition and the
effectiveness of the proposed methods. Both the set-theoretic deduction and the
experiments help us better understand generalization.