Efficient ResNets: Residual Network Design
Aditya Thakur Harish Chauhan Nikunj Gupta
ResNets (or Residual Networks) are one of the most commonly used models for
image classification tasks. In this project, we design and train a modified
ResNet model for CIFAR-10 image classification. In particular, we aimed at
maximizing the test accuracy on the CIFAR-10 benchmark while keeping the size
of our ResNet model under the specified fixed budget of 5 million trainable
parameters. Model size, typically measured as the number of trainable
parameters, is important when models need to be stored on devices with limited
storage capacity (e.g. IoT/edge devices). In this article, we present our
residual network design which has less than 5 million parameters. We show that
our ResNet achieves a test accuracy of 96.04% on CIFAR-10 which is much higher
than ResNet18 (which has greater than 11 million trainable parameters) when
equipped with a number of training strategies and suitable ResNet
hyperparameters. Models and code are available at
https://github.com/Nikunj-Gupta/Efficient_ResNets.