Using Cooperative Game Theory to Prune Neural Networks
Mauricio Diaz-Ortiz Jr Benjamin Kempinski Daphne Cornelisse Yoram Bachrach Tal Kachman
We show how solution concepts from cooperative game theory can be used to
tackle the problem of pruning neural networks.
  The ever-growing size of deep neural networks (DNNs) increases their
performance, but also their computational requirements. We introduce a method
called Game Theory Assisted Pruning (GTAP), which reduces the neural network's
size while preserving its predictive accuracy. GTAP is based on eliminating
neurons in the network based on an estimation of their joint impact on the
prediction quality through game theoretic solutions. Specifically, we use a
power index akin to the Shapley value or Banzhaf index, tailored using a
procedure similar to Dropout (commonly used to tackle overfitting problems in
machine learning).
  Empirical evaluation of both feedforward networks and convolutional neural
networks shows that this method outperforms existing approaches in the achieved
tradeoff between the number of parameters and model accuracy.