A Survey on Multi-Objective Neural Architecture Search
Seyed Mahdi Shariatzadeh Mahmood Fathy Reza Berangi Mohammad Shahverdy
Recently, the expert-crafted neural architectures is increasing overtaken by
the utilization of neural architecture search (NAS) and automatic generation
(and tuning) of network structures which has a close relation to the
Hyperparameter Optimization and Auto Machine Learning (AutoML). After the
earlier NAS attempts to optimize only the prediction accuracy, Multi-Objective
Neural architecture Search (MONAS) has been attracting attentions which
considers more goals such as computational complexity, power consumption, and
size of the network for optimization, reaching a trade-off between the accuracy
and other features like the computational cost. In this paper, we present an
overview of principal and state-of-the-art works in the field of MONAS.
Starting from a well-categorized taxonomy and formulation for the NAS, we
address and correct some miscategorizations in previous surveys of the NAS
field. We also provide a list of all known objectives used and add a number of
new ones and elaborate their specifications. We have provides analyses about
the most important objectives and shown that the stochastic properties of some
the them should be differed from deterministic ones in the multi-objective
optimization procedure of NAS. We finalize this paper with a number of future
directions and topics in the field of MONAS.