A Note on Dimensionality Reduction in Deep Neural Networks using
  Empirical Interpolation Method
Harbir Antil Madhu Gupta Randy Price
Empirical interpolation method (EIM) is a well-known technique to efficiently
approximate parameterized functions. This paper proposes to use EIM algorithm
to efficiently reduce the dimension of the training data within supervised
machine learning. This is termed as DNN-EIM. Applications in data science
(e.g., MNIST) and parameterized (and time-dependent) partial differential
equations (PDEs) are considered. The proposed DNNs in case of classification
are trained in parallel for each class. This approach is sequential, i.e., new
classes can be added without having to retrain the network. In case of PDEs, a
DNN is designed corresponding to each EIM point. Again, these networks can be
trained in parallel, for each EIM point. In all cases, the parallel networks
require fewer than ten times the number of training weights. Significant gains
are observed in terms of training times, without sacrificing accuracy.