How many views does your deep neural network use for prediction?
Keisuke Kawano Takuro Kutsuna Keisuke Sano
The generalization ability of Deep Neural Networks (DNNs) is still not fully
understood, despite numerous theoretical and empirical analyses. Recently,
Allen-Zhu & Li (2023) introduced the concept of multi-views to explain the
generalization ability of DNNs, but their main target is ensemble or distilled
models, and no method for estimating multi-views used in a prediction of a
specific input is discussed. In this paper, we propose Minimal Sufficient Views
(MSVs), which is similar to multi-views but can be efficiently computed for
real images. MSVs is a set of minimal and distinct features in an input, each
of which preserves a model's prediction for the input. We empirically show that
there is a clear relationship between the number of MSVs and prediction
accuracy across models, including convolutional and transformer models,
suggesting that a multi-view like perspective is also important for
understanding the generalization ability of (non-ensemble or non-distilled)
DNNs.