Python Tool for Visualizing Variability of Pareto Fronts over Multiple
  Runs
Shuhei Watanabe
Hyperparameter optimization is crucial to achieving high performance in deep
learning. On top of the performance, other criteria such as inference time or
memory requirement often need to be optimized due to some practical reasons.
This motivates research on multi-objective optimization (MOO). However, Pareto
fronts of MOO methods are often shown without considering the variability
caused by random seeds and this makes the performance stability evaluation
difficult. Although there is a concept named empirical attainment surface to
enable the visualization with uncertainty over multiple runs, there is no major
Python package for empirical attainment surface. We, therefore, develop a
Python package for this purpose and describe the usage. The package is
available at https://github.com/nabenabe0928/empirical-attainment-func.