Optimizing Convolutional Neural Network Architecture
Luis Balderas Miguel Lastra José M. Benítez
Convolutional Neural Networks (CNN) are widely used to face challenging tasks
like speech recognition, natural language processing or computer vision. As CNN
architectures get larger and more complex, their computational requirements
increase, incurring significant energetic costs and challenging their
deployment on resource-restricted devices. In this paper, we propose Optimizing
Convolutional Neural Network Architecture (OCNNA), a novel CNN optimization and
construction method based on pruning and knowledge distillation designed to
establish the importance of convolutional layers. The proposal has been
evaluated though a thorough empirical study including the best known datasets
(CIFAR-10, CIFAR-100 and Imagenet) and CNN architectures (VGG-16, ResNet-50,
DenseNet-40 and MobileNet), setting Accuracy Drop and Remaining Parameters
Ratio as objective metrics to compare the performance of OCNNA against the
other state-of-art approaches. Our method has been compared with more than 20
convolutional neural network simplification algorithms obtaining outstanding
results. As a result, OCNNA is a competitive CNN constructing method which
could ease the deployment of neural networks into IoT or resource-limited
devices.