Pruning Convolutional Filters via Reinforcement Learning with Entropy
  Minimization
Bogdan Musat Razvan Andonie
Structural pruning has become an integral part of neural network
optimization, used to achieve architectural configurations which can be
deployed and run more efficiently on embedded devices. Previous results showed
that pruning is possible with minimum performance loss by utilizing a
reinforcement learning agent which makes decisions about the sparsity level of
each neural layer by maximizing as a reward the accuracy of the network. We
introduce a novel information-theoretic reward function which minimizes the
spatial entropy of convolutional activations. This minimization ultimately acts
as a proxy for maintaining accuracy, although these two criteria are not
related in any way. Our method shows that there is another possibility to
preserve accuracy without the need to directly optimize it in the agent's
reward function. In our experiments, we were able to reduce the total number of
FLOPS of multiple popular neural network architectures by 5-10x, incurring
minimal or no performance drop and being on par with the solution found by
maximizing the accuracy.