A Survey on Multi-Objective based Parameter Optimization for Deep
  Learning
Mrittika Chakraborty Wreetbhas Pal Sanghamitra Bandyopadhyay Ujjwal Maulik
Deep learning models form one of the most powerful machine learning models
for the extraction of important features. Most of the designs of deep neural
models, i.e., the initialization of parameters, are still manually tuned.
Hence, obtaining a model with high performance is exceedingly time-consuming
and occasionally impossible. Optimizing the parameters of the deep networks,
therefore, requires improved optimization algorithms with high convergence
rates. The single objective-based optimization methods generally used are
mostly time-consuming and do not guarantee optimum performance in all cases.
Mathematical optimization problems containing multiple objective functions that
must be optimized simultaneously fall under the category of multi-objective
optimization sometimes referred to as Pareto optimization. Multi-objective
optimization problems form one of the alternatives yet useful options for
parameter optimization. However, this domain is a bit less explored. In this
survey, we focus on exploring the effectiveness of multi-objective optimization
strategies for parameter optimization in conjunction with deep neural networks.
The case studies used in this study focus on how the two methods are combined
to provide valuable insights into the generation of predictions and analysis in
multiple applications.