Binary domain generalization for sparsifying binary neural networks
Riccardo Schiavone Francesco Galati Maria A. Zuluaga
Binary neural networks (BNNs) are an attractive solution for developing and
deploying deep neural network (DNN)-based applications in resource constrained
devices. Despite their success, BNNs still suffer from a fixed and limited
compression factor that may be explained by the fact that existing pruning
methods for full-precision DNNs cannot be directly applied to BNNs. In fact,
weight pruning of BNNs leads to performance degradation, which suggests that
the standard binarization domain of BNNs is not well adapted for the task. This
work proposes a novel more general binary domain that extends the standard
binary one that is more robust to pruning techniques, thus guaranteeing
improved compression and avoiding severe performance losses. We demonstrate a
closed-form solution for quantizing the weights of a full-precision network
into the proposed binary domain. Finally, we show the flexibility of our
method, which can be combined with other pruning strategies. Experiments over
CIFAR-10 and CIFAR-100 demonstrate that the novel approach is able to generate
efficient sparse networks with reduced memory usage and run-time latency, while
maintaining performance.