SimQ-NAS: Simultaneous Quantization Policy and Neural Architecture
  Search
Sharath Nittur Sridhar Maciej Szankin Fang Chen Sairam Sundaresan Anthony Sarah
Recent one-shot Neural Architecture Search algorithms rely on training a
hardware-agnostic super-network tailored to a specific task and then extracting
efficient sub-networks for different hardware platforms. Popular approaches
separate the training of super-networks from the search for sub-networks, often
employing predictors to alleviate the computational overhead associated with
search. Additionally, certain methods also incorporate the quantization policy
within the search space. However, while the quantization policy search for
convolutional neural networks is well studied, the extension of these methods
to transformers and especially foundation models remains under-explored. In
this paper, we demonstrate that by using multi-objective search algorithms
paired with lightly trained predictors, we can efficiently search for both the
sub-network architecture and the corresponding quantization policy and
outperform their respective baselines across different performance objectives
such as accuracy, model size, and latency. Specifically, we demonstrate that
our approach performs well across both uni-modal (ViT and BERT) and multi-modal
(BEiT-3) transformer-based architectures as well as convolutional architectures
(ResNet). For certain networks, we demonstrate an improvement of up to $4.80x$
and $3.44x$ for latency and model size respectively, without degradation in
accuracy compared to the fully quantized INT8 baselines.