The huge size of deep networks hinders their use in small computing devices. In this paper, we consider compressing the network by weight quantization. We extend a recently proposed loss-aware weight binarization scheme to ternarization, with possibly different scaling parameters for the positive and negative weights, and m-bit (where m > 2) quantization. Experiments on feedforward and recurrent neural networks show that the proposed scheme outperforms state-of-the-art weight quantization algorithms, and is as accurate (or even more accurate) than the full-precision network. The last decade has witnessed huge success of deep neural networks in various domains. Examples include computer vision, speech recognition, and natural language processing BID16 . However, their huge size often hinders deployment to small computing devices such as cell phones and the internet of things. Many attempts have been recently made to reduce the model size. One common approach is to prune a trained dense network BID5 BID29 . However, most of the pruned weights may come from the fully-connected layers where computations are cheap, and the resultant time reduction is insignificant. BID21 and Molchanov et al. (2017) proposed to prune filters in the convolutional neural networks based on their magnitudes or significance to the loss. However, the pruned network has to be retrained, which is again expensive.Another direction is to use more compact models. GoogleNet (Szegedy et al., 2015) and ResNet BID7 replace the fully-connected layers with simpler global average pooling. However, they are also deeper. SqueezeNet BID11 reduces the model size by replacing most of the 3 × 3 filters with 1 × 1 filters. This is less efficient on smaller networks because the dense 1 × 1 convolutions are costly. MobileNet BID10 compresses the model using separable depth-wise convolution. ShuffleNet (Zhang et al., 2017) utilizes pointwise group convolution and channel shuffle to reduce the computation cost while maintaining accuracy. However, highly optimized group convolution and depth-wise convolution implementations are required. Alternatively, Novikov et al. (2015) compressed the model by using a compact multilinear format to represent the dense weight matrix. The CP and Tucker decompositions have also been used on the kernel tensor in CNNs BID15 BID13 . However, they often need expensive fine-tuning.Another effective approach to compress the network and accelerate training is by quantizing each full-precision weight to a small number of bits. This can be further divided to two sub-categories, depending on whether pre-trained models are used BID22 BID24 or the quantized model is trained from scratch BID1 BID20 . Some of these also directly learn with low-precision weights, but they usually suffer from severe accuracy deterioration BID20 BID26 . By keeping the full-precision weights during learning, BID1 pioneered the BinaryConnect algorithm, which uses only one bit for each weight while still achieving state-of-the-art classification results. Rastegari et al. (2016) further incorporated weight scaling, and obtained better results. Instead of simply finding the closest binary approximation of the full-precision weights, a loss-aware scheme is proposed in . Beyond binarization, TernaryConnect BID23 quantizes each weight to {−1, 0, 1}. BID19 and added scaling to the ternarized weights, and DoReFa-Net (Zhou et al., 2016) further extended quantization to more than three levels. However, these methods do not consider the effect of quantization on the loss, and rely on heuristics in their procedures (Zhou et al., 2016; . Recently, a loss-aware low-bit quantized neural network is proposed in BID18 . However, it uses full-precision weights in the forward pass and the extra-gradient method (Vasilyev et al., 2010) for update, both of which are expensive.In this paper, we propose an efficient and disciplined ternarization scheme for network compression. Inspired by , we explicitly consider the effect of ternarization on the loss. This is formulated as an optimization problem which is then solved efficiently by the proximal Newton algorithm. When the loss surface's curvature is ignored, the proposed method reduces to that of BID19 , and is also related to the projection step of BID18 . Next, we extend it to (i) allow the use of different scaling parameters for the positive and negative weights; and (ii) the use of m bits (where m > 2) for weight quantization. Experiments on both feedforward and recurrent neural networks show that the proposed quantization scheme outperforms state-of-the-art algorithms.Notations: For a vector x, √ x denotes the element-wise square root (i.e., [ DISPLAYFORM0 p is its p-norm, and Diag(x) returns a diagonal matrix with x on the diagonal. For two vectors x and y, x y denotes the element-wise multiplication and x y the element-wise division. Given a threshold ∆, I ∆ (x) returns a vector such DISPLAYFORM1 In this paper, we proposed a loss-aware weight quantization algorithm that directly considers the effect of quantization on the loss. The problem is solved using the proximal Newton algorithm. Each iteration consists of a preconditioned gradient descent step and a quantization step that projects fullprecision weights onto a set of quantized values. For ternarization, an exact solution and an efficient approximate solution are provided. The procedure is also extended to the use of different scaling parameters for the positive and negative weights, and to m-bit (where m > 2) quantization. Experiments on both feedforward and recurrent networks show that the proposed quantization scheme outperforms the current state-of-the-art.
A loss-aware weight quantization algorithm that directly considers its effect on the loss is proposed. Proposes a method of compressing network by means of weight ternarization.  The paper proposes a new method to train DNNs with quantized weights, by including the quantization as a constraint in a proximal quasi-Newton algorithm, which simultaneously learns a scaling for the quantized values. The paper extends the loss-aware weight binarization scheme to terarization and arbitrary m-bit quantization and demonstrate its promising performance.