Deep-ELA: Deep Exploratory Landscape Analysis with Self-Supervised
  Pretrained Transformers for Single- and Multi-Objective Continuous
  Optimization Problems
Moritz Vinzent Seiler Pascal Kerschke Heike Trautmann
In many recent works, the potential of Exploratory Landscape Analysis (ELA)
features to numerically characterize, in particular, single-objective
continuous optimization problems has been demonstrated. These numerical
features provide the input for all kinds of machine learning tasks on
continuous optimization problems, ranging, i.a., from High-level Property
Prediction to Automated Algorithm Selection and Automated Algorithm
Configuration. Without ELA features, analyzing and understanding the
characteristics of single-objective continuous optimization problems would be
impossible.
  Yet, despite their undisputed usefulness, ELA features suffer from several
drawbacks. These include, in particular, (1.) a strong correlation between
multiple features, as well as (2.) its very limited applicability to
multi-objective continuous optimization problems. As a remedy, recent works
proposed deep learning-based approaches as alternatives to ELA. In these works,
e.g., point-cloud transformers were used to characterize an optimization
problem's fitness landscape. However, these approaches require a large amount
of labeled training data.
  Within this work, we propose a hybrid approach, Deep-ELA, which combines (the
benefits of) deep learning and ELA features. Specifically, we pre-trained four
transformers on millions of randomly generated optimization problems to learn
deep representations of the landscapes of continuous single- and
multi-objective optimization problems. Our proposed framework can either be
used out-of-the-box for analyzing single- and multi-objective continuous
optimization problems, or subsequently fine-tuned to various tasks focussing on
algorithm behavior and problem understanding.