Augmented Random Search for Multi-Objective Bayesian Optimization of
  Neural Networks
Mark Deutel Georgios Kontes Christopher Mutschler Jürgen Teich
Deploying Deep Neural Networks (DNNs) on tiny devices is a common trend to
process the increasing amount of sensor data being generated. Multi-objective
optimization approaches can be used to compress DNNs by applying network
pruning and weight quantization to minimize the memory footprint (RAM), the
number of parameters (ROM) and the number of floating point operations (FLOPs)
while maintaining the predictive accuracy. In this paper, we show that existing
multi-objective Bayesian optimization (MOBOpt) approaches can fall short in
finding optimal candidates on the Pareto front and propose a novel solver based
on an ensemble of competing parametric policies trained using an Augmented
Random Search Reinforcement Learning (RL) agent. Our methodology aims at
finding feasible tradeoffs between a DNN's predictive accuracy, memory
consumption on a given target system, and computational complexity. Our
experiments show that we outperform existing MOBOpt approaches consistently on
different data sets and architectures such as ResNet-18 and MobileNetV3.