Deep Support Vectors
Junhoo Lee Hyunho Lee Kyomin Hwang Nojun Kwak
While the success of deep learning is commonly attributed to its theoretical
equivalence with Support Vector Machines (SVM), the practical implications of
this relationship have not been thoroughly explored. This paper pioneers an
exploration in this domain, specifically focusing on the identification of Deep
Support Vectors (DSVs) within deep learning models. We introduce the concept of
DeepKKT conditions, an adaptation of the traditional Karush-Kuhn-Tucker (KKT)
conditions tailored for deep learning. Through empirical investigations, we
illustrate that DSVs exhibit similarities to support vectors in SVM, offering a
tangible method to interpret the decision-making criteria of models.
Additionally, our findings demonstrate that models can be effectively
reconstructed using DSVs, resembling the process in SVM. The code will be
available.