Flexible Channel Dimensions for Differentiable Architecture Search
Ahmet Caner Yüzügüler Nikolaos Dimitriadis Pascal Frossard
Finding optimal channel dimensions (i.e., the number of filters in DNN
layers) is essential to design DNNs that perform well under computational
resource constraints. Recent work in neural architecture search aims at
automating the optimization of the DNN model implementation. However, existing
neural architecture search methods for channel dimensions rely on fixed search
spaces, which prevents achieving an efficient and fully automated solution. In
this work, we propose a novel differentiable neural architecture search method
with an efficient dynamic channel allocation algorithm to enable a flexible
search space for channel dimensions. We show that the proposed framework is
able to find DNN architectures that are equivalent to previous methods in task
accuracy and inference latency for the CIFAR-10 dataset with an improvement of
$1.3-1.7\times$ in GPU-hours and $1.5-1.7\times$ in the memory requirements
during the architecture search stage. Moreover, the proposed frameworks do not
require a well-engineered search space a priori, which is an important step
towards fully automated design of DNN architectures.