  There are many applications scenarios for which the computational
  performance and memory footprint of the prediction phase of Deep
  Neural Networks (DNNs) need to be optimized. Binary Deep Neural
  Networks (BDNNs) have been shown to be an effective way of achieving
  this objective. In this paper, we show how Convolutional Neural
  Networks (CNNs) can be implemented using binary
  representations. Espresso is a compact, yet powerful
  library written in C/CUDA that features all the functionalities
  required for the forward propagation of CNNs, in a binary file less
  than 400KB, without any external dependencies. Although it is mainly
  designed to take advantage of massive GPU parallelism, Espresso also
  provides an equivalent CPU implementation for CNNs. Espresso
  provides special convolutional and dense layers for BCNNs,
  leveraging bit-packing and bit-wise computations
  for efficient execution. These techniques provide a speed-up of
  matrix-multiplication routines, and at the same time, reduce memory
  usage when storing parameters and activations. We experimentally
  show that Espresso is significantly faster than existing
  implementations of optimized binary neural networks (~ 2
  orders of magnitude). Espresso is released under the Apache 2.0
  license and is available at http://github.com/organization/project. Convolutional Neural Networks have revolutionized computer vision, pushing the task of object recognition beyond human capabilities BID18 BID25 BID27 . Deep Neural Networks (DNN), have also been successfully applied in other fields, such as speech recognition BID9 and automated translation BID1 BID26 . Despite achieving impressive classification accuracy results, DNNs require too much memory and power to be used effectively on embedded or low-power devices. Many networks consume a considerable amount of memory. Memory remains a very limited resource on mobile platforms making harder the usage of trained DNNs 1 . Even when memory is not an issue, DNNs remain very computationally intensive, and can quickly drain the battery. Reducing the computational load does not only improve energy efficiency, but can also enable further applications. For example, when processing real-time object classification on mobile, being able to perform faster predictions frees up computational resources that can be spent on tasks such as speech recognition and analysis. Therefore, there is a substantial interest in reducing the computational and memory requirements of DNNs.Efficient deep neural networks One way to achieve this target is to use specialized hardware for DNNs. Another strategy is to reduce the network's memory footprint and associated computation, hence increasing its efficiency. Such solutions are preferable as they can be implemented in software without requiring specialized hardware. In our research we follow the software approach, and focus our attention to quantized networks. In this case, the parameters are stored as "small" integers (typically less than 8-bit) instead of single precision floating point numbers (32-bit). In particular, we consider the binary deep neural networks (BDNN) proposed by where parameters and activations are 1-bit integers: {−1, +1}. At the expense of a relatively small decrease in accuracy, BDNNs can considerably reduce memory usage, and result in faster execution time (i.e. forward propagation). Further, note that potential hardware implementation of BDNNs would also be cheaper due to the reduced number of required FPUs. While these results are highly promising, currently only proof-of-concept implementations of BinaryNets have been published . Therefore, the availability of a flexible end-to-end framework, with particular emphasis placed on computational efficiency, can enable further research on BDNNs, as well as its application to practical scenarios.Contributions With Espresso we provide an optimized framework for BDNNs capable of achieving state-of-the-art run-time performance with minimal memory footprint while being numerical equivalent to their non-optimized binary counterpart. Espresso provides a complete optimized framework for BDNNs supporting both the dense and the convolutional layer. Current state-ofthe-art optimized BDNNs implementations are limited to the fully connected layer, with the serious drawback of not being able to run optimized state-of-art convolutional BDNNs (BCNNs). While our work is a necessary stepping stone towards optimization of training routines, in this paper we focus on the optimization of forward-propagation (i.e. testing), rather than back-propagation (i.e. training). Espresso is designed to have no external dependencies. This not only results in a highly optimized implementation of BDNNs, but also substantially simplifies its deployment in practical applications, such as those executing on mobile or embedded devices. In this paper we presented Espresso, a highly optimized forward-propagation framework for both traditional DNNs as well as BCNNs, that supports heterogeneous deployment on CPU and GPU. While BinaryNet and Nervana/neon BDNN implementations are limited to MLP networks, our framework also supports the popular CNN while simultaneously outperforming state-of-the-art implementations of MLP networks. Espresso is highly-efficient, light-weight and self-contained. Computation on the GPU side is done though specifically designed CUDA kernels, which, combined with a more careful handling of memory allocation and bit-packing, allows us to obtain considerable performance improvements. In future work we would like to add training capabilities, and perform additional performance comparisons on larger standard datasets.
state-of-the-art computational performance implementation of binary neural networks The paper presents a library written in C/CUDA that features all the functionalities required for the forward propagation of BCNNs This paper builds on Binary-NET and expands it to CNN architectures, provides optimizations that improve the speed of the forward pass, and provides optimized code for Binary CNN.