A Review of Surrogate Assisted Multiobjective Evolutionary Algorithms
---
paper_title: Evolutionary Algorithms for Solving Multi-Objective Problems

paper_content:

List of Figures. List of Tables. Preface. Foreword. 1. Basic Concepts. 2. Evolutionary Algorithm MOP Approaches. 3. MOEA Test Suites. 4. MOEA Testing and Analysis. 5. MOEA Theory and Issues. 3. MOEA Theoretical Issues. 6. Applications. 7. MOEA Parallelization. 8. Multi-Criteria Decision Making. 9. Special Topics. 10. Epilog. Appendix A: MOEA Classification and Technique Analysis. Appendix B: MOPs in the Literature. Appendix C: Ptrue & PFtrue for Selected Numeric MOPs. Appendix D: Ptrue & PFtrue for Side-Constrained MOPs. Appendix E: MOEA Software Availability. Appendix F: MOEA-Related Information. Index. References.
---
paper_title: Knowledge Incorporation in Multi-objective Evolutionary Algorithms

paper_content:

This chapter presents a survey of techniques used to incorporate knowledge into evolutionary algorithms, with a particular emphasis on multi-objective optimization. We focus on two main groups of techniques: those that incorporate knowledge into the fitness evaluation, and those that incorporate knowledge in the initialization process and the operators of an evolutionary algorithm. Several techniques representative of each of these groups are briefly discussed, together with some examples found in the specialized literature. In the last part of the chapter, we provide some research ideas that are worth exploring in the future by researchers interested in this topic.
---
paper_title: A Survey of Fitness Approximation Methods Applied in Evolutionary Algorithms

paper_content:

Evolutionary algorithms (EAs) used in complex optimization domains usually need to perform a large number of fitness function evaluations in order to get near-optimal solutions. In real world application domains such as engineering design problems, such evaluations can be extremely computationally expensive. In some extreme cases there is no clear definition of the fitness function or the fitness function is too ambiguous to be deterministically evaluated. It is therefore common to estimate or approximate the fitness. A popular method is to construct a so-called surrogate or meta-model, which can simulate the behavior of the original fitness function, but can be evaluated much faster. An interesting trend is to use multiple surrogates to gain better performance in fitness approximation. In this chapter, an up-to-date survey of fitness approximation applied in evolutionary algorithms is presented. The main focus areas are the methods of fitness approximation, the working styles of fitness approximation, and the management of the approximation during the optimization process. To conclude, some open questions in this area are discussed.
---
paper_title: A Review of Techniques for Handling Expensive Functions in Evolutionary Multi-Objective Optimization

paper_content:

Evolutionary algorithms have been very popular for solving multiobjective optimization problems, mainly because of their ease of use, and their wide applicability. However, multi-objective evolutionary algorithms (MOEAs) tend to consume an important number of objective function evaluations, in order to achieve a reasonably good approximation of the Pareto front. This is a major concern when attempting to use MOEAs for real-world applications, since we can normally afford only a fairly limited number of fitness function evaluations in such cases. Despite these concerns, relatively few efforts have been reported in the literature to reduce the computational cost of MOEAs. It has been until relatively recently, that researchers have developed techniques to achieve an effective reduction of fitness function evaluations by exploiting knowledge acquired during the search. In this chapter, we analyze different proposals currently available in the specialized literature to deal with expensive functions in evolutionary multi-objective optimization. Additionally, we review some real-world applications of these methods, which can be seen as case studies in which such techniques led to a substantial reduction in the computational cost of the MOEA adopted. Finally, we also indicate some of the potential paths for future research in this area.
---
paper_title: Multiquadric equations of topography and other irregular surfaces

paper_content:

A new analytical method of representing irregular surfaces that involves the summation of equations of quadric surfaces having unknown coefficients is described. The quadric surfaces are located at significant points throughout the region to be mapped. Procedures are given for solving multiquadric equations of topography that are based on coordinate data. Contoured multiquadric surfaces are compared with topography and other irregular surfaces from which the multiquadric equation was derived.
---
paper_title: Principles of geostatistics

paper_content:

Knowledge of ore grades and ore reserves as well as error estimation of these values, is fundamental for mining engineers and mining geologists. Until now no appropriate scientific approach to those estimation problems has existed: geostatistics, the principles of which are summarized in this paper, constitutes a new science leading to such an approach. The author criticizes classical statistical methods still in use, and shows some of the main results given by geostatistics. Any ore deposit evaluation as well as proper decision of starting mining operations should be preceded by a geostatistical investigation which may avoid economic failures.
---
paper_title: Multiquadric equations of topography and other irregular surfaces

paper_content:

A new analytical method of representing irregular surfaces that involves the summation of equations of quadric surfaces having unknown coefficients is described. The quadric surfaces are located at significant points throughout the region to be mapped. Procedures are given for solving multiquadric equations of topography that are based on coordinate data. Contoured multiquadric surfaces are compared with topography and other irregular surfaces from which the multiquadric equation was derived.
---
paper_title: A Survey of Fitness Approximation Methods Applied in Evolutionary Algorithms

paper_content:

Evolutionary algorithms (EAs) used in complex optimization domains usually need to perform a large number of fitness function evaluations in order to get near-optimal solutions. In real world application domains such as engineering design problems, such evaluations can be extremely computationally expensive. In some extreme cases there is no clear definition of the fitness function or the fitness function is too ambiguous to be deterministically evaluated. It is therefore common to estimate or approximate the fitness. A popular method is to construct a so-called surrogate or meta-model, which can simulate the behavior of the original fitness function, but can be evaluated much faster. An interesting trend is to use multiple surrogates to gain better performance in fitness approximation. In this chapter, an up-to-date survey of fitness approximation applied in evolutionary algorithms is presented. The main focus areas are the methods of fitness approximation, the working styles of fitness approximation, and the management of the approximation during the optimization process. To conclude, some open questions in this area are discussed.
---
paper_title: MOPSO: a proposal for multiple objective particle swarm optimization

paper_content:

This paper introduces a proposal to extend the heuristic called "particle swarm optimization" (PSO) to deal with multiobjective optimization problems. Our approach uses the concept of Pareto dominance to determine the flight direction of a particle and it maintains previously found nondominated vectors in a global repository that is later used by other particles to guide their own flight. The approach is validated using several standard test functions from the specialized literature. Our results indicate that our approach is highly competitive with current evolutionary multiobjective optimization techniques.
---
paper_title: Advances in Hierarchical, Parallel Evolutionary Algorithms for Aerodynamic Shape Optimisation

paper_content:

The paper presents the recent developments in Hierarchical Parallel Evolutionary Algorithms to speed up optimisation of aerodynamic shapes. One is the implementation of different models in different layers of a Parallel Genetic Algorithm. The other is Asynchronous Hierarchical Evolution Strategy. These methods are employed to reconstruct a one-dimensional transonic nozzle and a two-dimensional aerofoil shape. Considerable speed up is achieved as a result.
---
paper_title: Multi-Objective Optimization of Transonic Compressor Blade Using Evolutionary Algorithm

paper_content:

In this work we perform multi-objective optimization of the NASA rotor67 transonic compressor blade. Our objectives are to maximize the stage pressure ratio as well as to minimize the compressor weight. The backbones of the optimization approach consist of a genetic algorithm, a gradient-based method, and a response surface model. The genetic algorithm is used to facilitate the multi-objective optimization and to flnd the global optima of high-dimensional problems. The gradient-based method accelerates the optimization convergence rate. The response surface model, constructed to replace the computationally expensive analysis tool, reduces the computational cost. Representative solutions are selected from the Pareto-optimal front to verify against the CFD tool. Comparing with the baseline design some optimal solutions increase the stage pressure ratio by 1.8% and decrease the weight by 5.4%. A detailed study of ∞ow structure near peak e‐ciency is presented by means of pressure distribution and streamlines inside boundary layers. Our results show that the optimized blade favors a lighter weight by a thinner blade shape. The stage pressure rise is attributed to a reduced separation zone and a weaken shock wave.
---
paper_title: Combining Convergence and Diversity in Evolutionary Multiobjective Optimization

paper_content:

Over the past few years, the research on evolutionary algorithms has demonstrated their niche in solving multiobjective optimization problems, where the goal is to find a number of Pareto-optimal solutions in a single simulation run. Many studies have depicted different ways evolutionary algorithms can progress towards the Pareto-optimal set with a widely spread distribution of solutions. However, none of the multiobjective evolutionary algorithms (MOEAs) has a proof of convergence to the true Pareto-optimal solutions with a wide diversity among the solutions. In this paper, we discuss why a number of earlier MOEAs do not have such properties. Based on the concept of e-dominance, new archiving strategies are proposed that overcome this fundamental problem and provably lead to MOEAs that have both the desired convergence and distribution properties. A number of modifications to the baseline algorithm are also suggested. The concept of e-dominance introduced in this paper is practical and should make the proposed algorithms useful to researchers and practitioners alike.
---
paper_title: Pareto simulated annealing—a metaheuristic technique for multiple‐objective combinatorial optimization

paper_content:

This paper presents a multiple-objective metaheuristic procedure—Pareto simulated annealing. The goal of the procedure is to find in a relatively short time a good approximation of the set of efficient solutions of a multiple-objective combinatorial optimization problem. The procedure uses a sample, of so-called generating solutions. Each solution explores its neighbourhood in a way similar to that of classical simulated annealing. Weights of the objectives, used for their local aggregation, are tuned in each iteration in order to assure a tendency for approaching the efficient solutions set while maintaining a uniform distribution of the generating solutions over this set. A computational experiment shows that the method is a better tool for approximating the efficient set than some previous proposals. © 1998 John Wiley & Sons, Ltd.
---
paper_title: Handling multiple objectives with particle swarm optimization

paper_content:

This paper presents an approach in which Pareto dominance is incorporated into particle swarm optimization (PSO) in order to allow this heuristic to handle problems with several objective functions. Unlike other current proposals to extend PSO to solve multiobjective optimization problems, our algorithm uses a secondary (i.e., external) repository of particles that is later used by other particles to guide their own flight. We also incorporate a special mutation operator that enriches the exploratory capabilities of our algorithm. The proposed approach is validated using several test functions and metrics taken from the standard literature on evolutionary multiobjective optimization. Results indicate that the approach is highly competitive and that can be considered a viable alternative to solve multiobjective optimization problems.
---
paper_title: PDE: a Pareto-frontier differential evolution approach for multi-objective optimization problems

paper_content:

The use of evolutionary algorithms (EAs) to solve problems with multiple objectives (known as multi-objective optimization problems (MOPs)) has attracted much attention. Being population based approaches, EAs offer a means to find a group of Pareto-optimal solutions in a single run. Differential evolution (DE) is an EA that was developed to handle optimization problems over continuous domains. The objective of this paper is to introduce a novel Pareto-frontier differential evolution (PDE) algorithm to solve MOPs. The solutions provided by the proposed algorithm for two standard test problems, outperform the Strength Pareto Evolutionary Algorithm, one of the state-of-the-art evolutionary algorithms for solving MOPs.
---
paper_title: Enhanced multi-objective optimization of a microchannel heat sink through evolutionary algorithm coupled with multiple surrogate models

paper_content:

A liquid flow microchannel heat sink has been optimized with the help of three-dimensional numerical analysis and multiple surrogate methods. Two objective functions, thermal resistance and pumping power have been selected to assess the performance of the microchannel heat sink. The design variables related to the width of the microchannel at the top and bottom, depth of the microchannel, and width of fin, which contribute to objective functions, have been identified and a three-level full factorial design was selected to exploit the design space. The numerical solutions obtained at these design points were utilized to construct surrogate models, namely Response Surface Approximations, Kriging and Radial Basis Neural Network. A hybrid multi-objective evolutionary algorithm coupled with surrogate models is applied to find out global Pareto-optimal solutions. The accuracy of the surrogate models has been discussed in view of their predictions and trade-off analysis was performed in view of conflicting nature of the two objectives. The Pareto-optimal sensitivity of the design variables has been found out to economically compromise with the design variables. The application of the multiple surrogate methods not only improves quality of multi-objective optimization but also gives the feedback of the fidelity of the optimization model.
---
paper_title: The self-adaptive Pareto differential evolution algorithm

paper_content:

The Pareto differential evolution (PDE) algorithm was introduced and showed competitive results. The behavior of PDE, as in many other evolutionary multiobjective optimization (EMO) methods, varies according to the crossover and mutation rates. In this paper, we present a new version of PDE with self-adaptive crossover and mutation. We call the new version self-adaptive Pareto differential evolution (SPDE). The emphasis of this paper is to analyze the dynamics and behavior of SPDE. The experiments also show that the algorithm is very competitive with other EMO algorithms.
---
paper_title: Multiobjective optimization for crash safety design of vehicles using stepwise regression model

paper_content:

In automotive industry, structural optimization for crashworthiness criteria is of special importance. Due to the high nonlinearities, however, there exists substantial difficulty to obtain accurate continuum or discrete sensitivities. For this reason, metamodel or surrogate model methods have been extensively employed in vehicle design with industry interest. This paper presents a multiobjective optimization procedure for the vehicle design, where the weight, acceleration characteristics and toe-board intrusion are considered as the design objectives. The response surface method with linear and quadratic basis functions is employed to formulate these objectives, in which optimal Latin hypercube sampling and stepwise regression techniques are implemented. In this study, a nondominated sorting genetic algorithm is employed to search for Pareto solution to a full-scale vehicle design problem that undergoes both the full frontal and 40% offset-frontal crashes. The results demonstrate the capability and potential of this procedure in solving the crashworthiness design of vehicles.
---
paper_title: Response surface approximation of Pareto optimal front in multi-objective optimization

paper_content:

A systematic approach is presented to approximate the Pareto optimal front (POF) by a response surface approximation. The data for the POF is obtained by multi-objective evolutionary algorithm. Improvements to address drift in the POF are also presented. The approximated POF can help visualize and quantify trade-offs among objectives to select compromise designs. The bounds of this approximate POF are obtained using multiple convex-hulls. The proposed approach is applied to study trade-offs among objectives of a rocket injector design problem where performance and life objectives compete. The POF is approximated using a quintic polynomial. The compromise region quantifies trade-offs among objectives. � 2006 Elsevier B.V. All rights reserved.
---
paper_title: Application of Particle Swarm to Multiobjective Optimization

paper_content:

A new and distinct cultivar of impatiens plant named Bora Bora, characterized by its large, purple-violet flowers, compact growth habit with excellent self-branching without pinching, medium green foliage, very early and uniform flowering, floriferous habit, and by its suitability to pot or hanging basket culture.
---
paper_title: Indicator-Based Selection in Multiobjective Search

paper_content:

This paper discusses how preference information of the decision maker can in general be integrated into multiobjective search. The main idea is to first define the optimization goal in terms of a binary performance measure (indicator) and then to directly use this measure in the selection process. To this end, we propose a general indicator-based evolutionary algorithm (IBEA) that can be combined with arbitrary indicators. In contrast to existing algorithms, IBEA can be adapted to the preferences of the user and moreover does not require any additional diversity preservation mechanism such as fitness sharing to be used. It is shown on several continuous and discrete benchmark problems that IBEA can substantially improve on the results generated by two popular algorithms, namely NSGA-II and SPEA2, with respect to different performance measures.
---
paper_title: On Similarity-Based Surrogate Models for Expensive Single- and Multi-objective Evolutionary Optimization

paper_content:

In this chapter we propose a surrogate-assisted framework for expensive single- and multi-objective evolutionary optimization, under a fixed budget of computationally intensive evaluations. The framework uses similarity-based surrogate models and an individual-based model management with pre-selection. Instead of existing frameworks where the surrogates are used to improve the performance of evolutionary operators or as local search tools, here we use them to allow for an augmented number of generations to evolve solutions. The introduction of the surrogates into the evolutionary cycle is controlled by a single parameter, which is related with the number of generations performed by the evolutionary algorithm.Numerical experiments are conducted in order to assess the applicability and the performance in constrained and unconstrained, single- and multi-objective optimization problems. The results show that the present framework arises as an attractive alternative to improve the final solutions with a fixed budget of expensive evaluations.
---
paper_title: A novel multi-objective optimization method based on an approximation model management technique

paper_content:

In this paper, a novel multi-objective optimization method is suggested based on an approximation model management technique. It is a sequential approximation method, in which a multi-objective optimization with approximation models subject to design variable move limits is iterated until convergence. In each iteration step, the approximation models are constructed by the response surface approximations with the samples which are obtained from the design of experiments, and a Pareto optimal set predicted by the approximations is identified through a multi-objective genetic algorithm. According to the prediction of the approximation models, a move limits updating strategy is employed to determine the design variable move limits for the next iteration. At the end of each iteration step, some uniform distributed points chosen from the predictive Pareto optimal frontier are verified by the high fidelity models and the obtained actual Pareto optimal set is stored in an external archive. The high efficiency of the present method is demonstrated by four different test functions and two engineering applications.
---
paper_title: Multi-objective evolutionary optimization of subsonic airfoils by kriging approximation and evolution control

paper_content:

This work focuses on multi-objective evolutionary optimization by approximation function. It uses the new general concept of evolution control to online enriching the database of correct solutions, which are the basis of the learning procedure for the kriging approximators. Substantially, given an initial very poor model approximation (small size of the database), the database, and consequently the models, is enriched by evaluating part of the individuals of the optimization process. The technique showed being efficient for the considered aerodynamic problem, by requiring few hundreds of true computations when the dimensionality of the problem is 5.
---
paper_title: Accelerating the Convergence of Evolutionary Algorithms by Fitness Landscape Approximation

paper_content:

A new algorithm is presented for accelerating the convergence of evolutionary optimization methods through a reduction in the number of fitness function calls. Such a reduction is obtained by 1) creating an approximate model of the fitness landscape using kriging interpolation, and 2) using this model instead of the original fitness function for evaluating some of the next generations. The main interest of the presented approach lies in problems for which the computational costs associated with fitness function evaluation is very high, such as in the case of most engineering design problems. Numerical results presented for a test case show that the reconstruction algorithm can effectively reduces the number of fitness function calls for simple problems as well as for difficult multidimensional ones.
---
paper_title: Modified Efficient Global Optimization for a Hat-Stiffened Composite Panel with Buckling Constraint

paper_content:

The optimization method for composite structural components described herein uses modified efficient global optimization with a multi-objective genetic algorithm and a kriging response surface. For efficient global optimization using kriging, the kriging response surface is used as a representative of the function value. The stochastic distribution of the kriging is used to improve the estimation error of the kriging surrogate model. Using efficient global optimization, a hat-stiffened composite panel was optimized to reduce the weight with the buckling load constraint. The expected improvement was used as a single objective function of a particle swarm optimization. Nevertheless, it is difficult to obtain a feasible solution that satisfies buckling load constraints with the progress of optimization. Using a multi-objective genetic algorithm, we obtain the feasible optimal structure satisfying the constraints. The expected improvement objective function is divided into two objective functions: weight reduction and the uncertainty of satisfaction of the buckling load constraint. Kriging approximation, which is improved with the selected Pareto optimal frontier, reduces the computational cost. Also, a genetic algorithm is used to optimize the stiffened panel configuration. The fractal branch-and-bound method is used for stacking sequence optimizations. This method obtained a feasible optimal structure at a low computational cost.
---
paper_title: Multi-Objective Optimization Using Surrogates

paper_content:

Until recently, optimization was regarded as a discipline of rather theoretical interest, with limited real-life applicability due to the computational or experimental expense involved. Practical multiobjective optimization was considered almost as a utopia even in academic studies due to the multiplication of this expense. This paper discusses the idea of using surrogate models for multiobjective optimization. With recent advances in grid and parallel computing more companies are buying inexpensive computing clusters that work in parallel. This allows, for example, efficient fusion of surrogates and finite element models into a multiobjective optimization cycle. The research presented here demonstrates this idea using several response surface methods on a pre-selected set of test functions. We aim to show that there are number of techniques which can be used to tackle difficult problems and we also demonstrate that a careful choice of response surface methods is important when carrying out surrogate assisted multiobjective search.
---
paper_title: MOEA/D assisted by rbf networks for expensive multi-objective optimization problems

paper_content:

The development of multi-objective evolutionary algorithms assisted by surrogate models has increased in the last few years. However, in real-world applications, the high modality and dimensionality that functions have, often causes problems to such models. In fact, if the Pareto optimal set of a multi-objective optimization problem is located in a search space in which the surrogate model is not able to shape the corresponding region, the search could be misinformed and thus converge to wrong regions. Because of this, a considerable amount of research has focused on improving the prediction of the surrogate models by adding the new solutions to the training set and retraining the model. However, when the size of the training set increases, the training complexity can significantly increase. In this paper, we present a surrogate model which maintains the size of the training set, and in which the prediction of the function is improved by using radial basis function networks in a cooperative way. Preliminary results indicate that our proposed approach can produce good quality results when it is restricted to performing only 200, 1,000 and 5,000 fitness function evaluations. Our proposed approach is validated using a set of standard test problems and an airfoil design problem.
---
paper_title: MOPED: A multi-objective parzen-based estimation of distribution algorithm for continuous problems

paper_content:

An evolutionary multi-objective optimization tool based on an estimation of distribution algorithm is proposed. The algorithm uses the ranking method of non-dominated sorting genetic algorithm-II and the Parzen estimator to approximate the probability density of solutions lying on the Pareto front. The proposed algorithm has been applied to different types of test case problems and results show good performance of the overall optimization procedure in terms of the number of function evaluations. An alternative spreading technique that uses the Parzen estimator in the objective function space is proposed as well. When this technique is used, achieved results appear to be qualitatively equivalent to those previously obtained by adopting the crowding distance described in non-dominated sorting genetic algorithm-II.
---
paper_title: A hybrid surrogate-based approach for evolutionary multi-objective optimization

paper_content:

Evolutionary algorithms have gained popularity as an alternative for dealing with multi-objective optimization problems. However, these algorithms require to perform a relatively high number of fitness function evaluations in order to generate a reasonably good approximation of the Pareto front. This can be a shortcoming when fitness evaluations are computationally expensive. In this paper, we propose an approach that combines an evolutionary algorithm with an ensemble of surrogate models based on support vector machines (SVM), which are used to approximate the fitness functions of a problem. The proposed approach performs a model selection process for determining the appropriate hyperparameters values for each SVM in the ensemble. The ensemble is constructed in an incremental fashion, such that the models are updated with the knowledge gained during the evolutionary process, but the information from previous evaluated regions is also preserved. A criterion based on surrogate fidelity is also proposed for determining when should the surrogates be updated. We evaluate the performance of our proposal using a benchmark of test problems widely used in the literature and we compare our results with respect to those obtained by the NSGA-II. Our proposed approach is able to significantly reduce the number of fitness function evaluations performed, while producing solutions which are close to the true Pareto front.
---
paper_title: A Multi-Objective Evolutionary Algorithm Using Neural Networks to Approximate Fitness Evaluations

paper_content:

Two different methods to accelerate the search of a Multi-Objective Evolutionary Algorithm (MOEA) using Artificial Neural Networks are presented. Two different methods are proposed. One using ANN to approximate the fitness of the solutions alternated with the real fitness evaluation, being the ANN approximation used only when the estimated error of the neural network was lower than a pre-defined value. In the second method, the ANN is used as a local search strategy by defining new better solutions from the precedent generation. These methods can substantially reduce the number of fitness evaluations on computational expensive problems while not compromise the good search capabilities of MOEA. The efficiency of the methods proposed is tested on several benchmark functions as well on a real multi-optimization problem of polymer extrusion.
---
paper_title: Multi-objective management of saltwater intrusion in coastal aquifers using genetic programming and modular neural network based surrogate models

paper_content:

Surrogate model based methodologies are developed for evolving multi-objective management strategies for saltwater intrusion in coastal aquifers. Two different surrogate models based on genetic programming (GP) and modular neural network (MNN) are developed and linked to a multi-objective genetic algorithm (MOGA) to derive the optimal pumping strategies for coastal aquifer management, considering two objectives. Trained and tested surrogate models are used to predict the salinity concentrations at different locations resulting due to groundwater extraction. A two-stage training strategy is implemented for training the surrogate models. Surrogate models are initially trained with input patterns selected uniformly from the entire search space and optimal management strategies based on the model predictions are derived from the management model. A search space adaptation and model retraining is performed by identifying a modified search space near the initial optimal solutions based on the relative importance of the variables in salinity prediction. Retraining of the surrogate models is performed using input–output samples generated in the modified search space. Performance of the methodologies using GP and MNN based surrogate models are compared for an illustrative study area. The capability of GP to identify the impact of input variables and the resulting parsimony of the input variables helps in developing efficient surrogate models. The developed GP models have lesser uncertainty compared to MNN models as the number of parameters used in GP is much lesser than that in MNN models. Also GP based model was found to be better suited for optimization using adaptive search space.
---
paper_title: ASM-MOMA: Multiobjective memetic algorithm with aggregate surrogate model

paper_content:

Evolutionary algorithms generally require a large number of objective function evaluations which can be costly in practice. These evaluations can be replaced by evaluations of a cheaper meta-model (surrogate model) of the objective functions. In this paper we present a novel distance based aggregate surrogate model for multiobjective optimization and describe a memetic multiobjective algorithm based on this model. Various variants of the models are tested and discussed and the algorithm is compared to standard multiobjective evolutionary algorithms. We show that our algorithm greatly reduces the number of required objective function evaluations.
---
paper_title: A mono surrogate for multiobjective optimization

paper_content:

Most surrogate approaches to multi-objective optimization build a surrogate model for each objective. These surrogates can be used inside a classical Evolutionary Multiobjective Optimization Algorithm (EMOA) in lieu of the actual objectives, without modifying the underlying EMOA; or to filter out points that the models predict to be uninteresting. In contrast, the proposed approach aims at building a global surrogate model defined on the decision space and tightly characterizing the current Pareto set and the dominated region, in order to speed up the evolution progress toward the true Pareto set. This surrogate model is specified by combining a One-class Support Vector Machine (SVMs) to characterize the dominated points, and a Regression SVM to clamp the Pareto front on a single value. The resulting surrogate model is then used within state-of-the-art EMOAs to pre-screen the individuals generated by application of standard variation operators. Empirical validation on classical MOO benchmark problems shows a significant reduction of the number of evaluations of the actual objective functions.
---
paper_title: Combining Convergence and Diversity in Evolutionary Multiobjective Optimization

paper_content:

Over the past few years, the research on evolutionary algorithms has demonstrated their niche in solving multiobjective optimization problems, where the goal is to find a number of Pareto-optimal solutions in a single simulation run. Many studies have depicted different ways evolutionary algorithms can progress towards the Pareto-optimal set with a widely spread distribution of solutions. However, none of the multiobjective evolutionary algorithms (MOEAs) has a proof of convergence to the true Pareto-optimal solutions with a wide diversity among the solutions. In this paper, we discuss why a number of earlier MOEAs do not have such properties. Based on the concept of e-dominance, new archiving strategies are proposed that overcome this fundamental problem and provably lead to MOEAs that have both the desired convergence and distribution properties. A number of modifications to the baseline algorithm are also suggested. The concept of e-dominance introduced in this paper is practical and should make the proposed algorithms useful to researchers and practitioners alike.
---
paper_title: Kriging-Assisted Multi-Objective Particle Swarm Optimization of permanent magnet synchronous machine for hybrid and electric cars

paper_content:

In this paper a novel combination of a multiobjective particle swarm optimization algorithm and a Kriging metamodel is presented, which is used for the design optimization of a permanent magnet synchronous machine (PMSM) for electric and hybrid cars. As the evaluation of mostly conflicting objective functions requires computational expensive numerical field simulations, a fast converging optimization algorithm which depends on only a few exact evaluations is needed. Instead of running exact calculations on every design the objective values are estimated by means of a Kriging model in advance and only the most promising design candidates are being computed in a more accurate way. Furthermore, good solutions are searched for directly on the metamodel and particles are spawned at the corresponding positions in the parameter space. The performance of the proposed algorithm is investigated with the help of an analytical test case and used for design optimization of a 10 poles PMSM with 11 parameters and 3 objectives.
---
paper_title: An informed convergence accelerator for evolutionary multiobjective optimiser

paper_content:

A novel optimisation accelerator deploying neural network predictions and objective space direct manipulation strategies is presented. The concept of directing the search through the use of 'mirage' solutions is introduced and investigated. The accelerator is meant to be a portable component that can be plugged into any stochastic optimisation algorithm, such as genetic algorithms. The purpose of the new component termed as the Informed Convergence Accelerator (ICA) is to enhance the search capability, convergence extent and most especially the speed of convergence of the hosting stochastic global optimisation technique. ICA was hybridized with the Non-Dominated Sorting Genetic Algorithm (NSGA-II). Enhanced results were achieved demonstrating the utility of the introduced component.
---
paper_title: A Surrogate Based Multiobjective Evolution Strategy with Different Models for Local Search and Pre-selection

paper_content:

In this paper we present a multiobjective evolutionary algorithm which uses surrogate models in two different ways -- during a local search and during pre-selection. Two different approaches to surrogate modeling are used, and the algorithm provides multiple individuals in each generation to enable easy parallelization. The algorithm is tested and compared to standard multiobjective evolutionary algorithms and to our previously developed surrogate evolution strategy. We also discuss the importance of the use of two different approaches and show that it improves the convergence speed significantly.
---
paper_title: Informed operators: Speeding up genetic-algorithm-based design optimization using reduced models

paper_content:

In this paper we describe a method for improving genetic-algorithm-based optimization using informed genetic operators. The idea is to make the genetic operators such as mutation and crossover more informed using reduced models. In every place where a random choice is made, for example when a point is mutated, instead of generating just one random mutation we generate several, rank them using a reduced model, then take the best to be the result of the mutation. The proposed method is particularly suitable for search spaces with expensive evaluation functions, such as arise in engineering design. Empirical results in several engineering design domains demonstrate that the proposed method can significantly speed up the GA optimizer.
---
paper_title: MODE-LD+SS: A novel Differential Evolution algorithm incorporating local dominance and scalar selection mechanisms for multi-objective optimization

paper_content:

In this paper, we present a novel Multi-Objective Evolutionary Algorithm (MOEA) called MODE-LD+SS, which combines Differential Evolution with local dominance and a scalar selection mechanism for improving both its convergence rate and its distribution of solutions along the Pareto front. In order to assess the performance of the proposed approach, we use a set of standard test functions and performance measures taken from the specialized literature. Results are compared with respect to three MOEAs representative of the state-of-the-art in the area: NSGA-II, SPEA2, and MOEA/D.
---
paper_title: Efficient Global Optimization of Expensive Black-Box Functions

paper_content:

In many engineering optimization problems, the number of function evaluations is severely limited by time or cost. These problems pose a special challenge to the field of global optimization, since existing methods often require more function evaluations than can be comfortably afforded. One way to address this challenge is to fit response surfaces to data collected by evaluating the objective and constraint functions at a few points. These surfaces can then be used for visualization, tradeoff analysis, and optimization. In this paper, we introduce the reader to a response surface methodology that is especially good at modeling the nonlinear, multimodal functions that often occur in engineering. We then show how these approximating functions can be used to construct an efficient global optimization algorithm with a credible stopping rule. The key to using response surfaces for global optimization lies in balancing the need to exploit the approximating surface (by sampling where it is minimized) with the need to improve the approximation (by sampling where prediction error may be high). Striking this balance requires solving certain auxiliary problems which have previously been considered intractable, but we show how these computational obstacles can be overcome.
---
paper_title: A Multi-Objective Evolutionary Algorithm Using Neural Networks to Approximate Fitness Evaluations

paper_content:

Two different methods to accelerate the search of a Multi-Objective Evolutionary Algorithm (MOEA) using Artificial Neural Networks are presented. Two different methods are proposed. One using ANN to approximate the fitness of the solutions alternated with the real fitness evaluation, being the ANN approximation used only when the estimated error of the neural network was lower than a pre-defined value. In the second method, the ANN is used as a local search strategy by defining new better solutions from the precedent generation. These methods can substantially reduce the number of fitness evaluations on computational expensive problems while not compromise the good search capabilities of MOEA. The efficiency of the methods proposed is tested on several benchmark functions as well on a real multi-optimization problem of polymer extrusion.
---
paper_title: Multi-objective airfoil shape optimization using a multiple-surrogate approach

paper_content:

In this paper, we present a surrogate-based multi-objective evolutionary optimization approach to optimize airfoil aerodynamic designs. Our approach makes use of multiple surrogate models which operate in parallel with the aim of combining their features when solving a costly multi-objective optimization problem. The proposed approach is used to solve five multiobjective airfoil aerodynamic optimization problems. We compare the performance of a multi-objective evolutionary algorithm with surrogates with respect to the same approach without using surrogates. Our preliminary results indicate that our proposal can achieve a substantial reduction in the number of objective function evaluations, which has obvious advantages for dealing with expensive objective functions such as those involved in aeronautical optimization problems.
---
paper_title: ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems

paper_content:

This paper concerns multiobjective optimization in scenarios where each solution evaluation is financially and/or temporally expensive. We make use of nine relatively low-dimensional, nonpathological, real-valued functions, such as arise in many applications, and assess the performance of two algorithms after just 100 and 250 (or 260) function evaluations. The results show that NSGA-II, a popular multiobjective evolutionary algorithm, performs well compared with random search, even within the restricted number of evaluations used. A significantly better performance (particularly, in the worst case) is, however, achieved on our test set by an algorithm proposed herein-ParEGO-which is an extension of the single-objective efficient global optimization (EGO) algorithm of Jones et al. ParEGO uses a design-of-experiments inspired initialization procedure and learns a Gaussian processes model of the search landscape, which is updated after every function evaluation. Overall, ParEGO exhibits a promising performance for multiobjective optimization problems where evaluations are expensive or otherwise restricted in number.
---
paper_title: Aggregate meta-models for evolutionary multiobjective and many-objective optimization

paper_content:

Abstract Evolutionary algorithms are among the best multiobjective optimizers. However, they need a large number of function evaluations. In this paper a meta-model based approach to the reduction in the needed number of function evaluations is presented. Local aggregate meta-models are used in a memetic operator. The algorithm is first discussed from a theoretical point of view and then it is shown that the meta-models greatly reduce the number of function evaluations. The approach is compared to a similar one with a single global meta-model as well as to more traditional NSGA-II and ϵ - IBEA . Moreover, it is shown that aggregate meta-models work even for a larger number of objectives and therefore should be considered when designing many-objective evolutionary algorithms.
---
paper_title: Indicator-Based Selection in Multiobjective Search

paper_content:

This paper discusses how preference information of the decision maker can in general be integrated into multiobjective search. The main idea is to first define the optimization goal in terms of a binary performance measure (indicator) and then to directly use this measure in the selection process. To this end, we propose a general indicator-based evolutionary algorithm (IBEA) that can be combined with arbitrary indicators. In contrast to existing algorithms, IBEA can be adapted to the preferences of the user and moreover does not require any additional diversity preservation mechanism such as fitness sharing to be used. It is shown on several continuous and discrete benchmark problems that IBEA can substantially improve on the results generated by two popular algorithms, namely NSGA-II and SPEA2, with respect to different performance measures.
---
paper_title: Expensive Multiobjective Optimization by MOEA/D With Gaussian Process Model

paper_content:

In some expensive multiobjective optimization problems (MOPs), several function evaluations can be carried out in a batch way. Therefore, it is very desirable to develop methods which can generate multipler test points simultaneously. This paper proposes such a method, called MOEA/D-EGO, for dealing with expensive multiobjective optimization. MOEA/D-EGO decomposes an MOP in question into a number of single-objective optimization subproblems. A predictive model is built for each subproblem based on the points evaluated so far. Effort has been made to reduce the overhead for modeling and to improve the prediction quality. At each generation, MOEA/D is used for maximizing the expected improvement metric values of all the subproblems, and then several test points are selected for evaluation. Extensive experimental studies have been carried out to investigate the ability of the proposed algorithm.
---
paper_title: Multiobjective Metamodel–Assisted Memetic Algorithms

paper_content:

The hybridization of global metaheuristics, such as evolutionary algorithms (EAs), and gradient-based methods for local search, in the framework of the so-called memetic algorithms (MAs) can be used to solve multi-objective optimization problems, either in the Lamarckian or Baldwinian spirit. Reducing the CPU cost of MAs is necessary for problems with computationally demanding evaluations. For the same purpose, in EAs, metamodels are in widespread use, giving rise to various kinds of metamodelassisted EAs (MAEAs). Metamodels are surrogate evaluation models of various types: multilayer perceptrons, radial basis function networks, polynomial regressions models, kriging, etc. A good practice is to use local metamodels, trained on the fly for each new individual, using selected entries from a database where all the previously evaluated offspring are recorded. The selection of suitable training patterns is important in order to minimize the prediction error of the metamodel. The MAEA developed by the authors in the past uses the inexact pre-evaluation (IPE) technique which starts after running a conventional EA for just a few generations on the exact evaluation model. The exactly evaluated offspring are all stored in the database. For the subsequent generations, local metamodels are trained for each new offspring to get an approximation to the objective functions so that, based on it, a few top individuals (in the Pareto front sense) are selected for exact re-evaluation.
---
paper_title: Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels

paper_content:

This paper presents and analyzes in detail an efficient search method based on evolutionary algorithms (EA) assisted by local Gaussian random field metamodels (GRFM). It is created for the use in optimization problems with one (or many) computationally expensive evaluation function(s). The role of GRFM is to predict objective function values for new candidate solutions by exploiting information recorded during previous evaluations. Moreover, GRFM are able to provide estimates of the confidence of their predictions. Predictions and their confidence intervals predicted by GRFM are used by the metamodel assisted EA. It selects the promising members in each generation and carries out exact, costly evaluations only for them. The extensive use of the uncertainty information of predictions for screening the candidate solutions makes it possible to significantly reduce the computational cost of singleand multiobjective EA. This is adequately demonstrated in this paper by means of mathematical test cases and a multipoint airfoil design in aerodynamics
---
paper_title: Multiobjective GA optimization using reduced models

paper_content:

In this paper, we propose a novel method for solving multiobjective optimization problems using reduced models. Our method, called objective exchange genetic algorithm for design optimization (OEGADO), is intended for solving real-world application problems. For such problems, the number of objective evaluations performed is a critical factor as a single objective evaluation can be quite expensive. The aim of our research is to reduce the number of objective evaluations needed to find a well-distributed sampling of the Pareto-optimal region by applying reduced models to steady-state multiobjective GAs. OEGADO runs several GAs concurrently with each GA optimizing one objective and forming a reduced model of its objective. At regular intervals, each GA exchanges its reduced model with the others. The GAs use these reduced models to bias their search toward compromise solutions. Empirical results in several engineering and benchmark domains comparing OEGADO with two state-of-the-art multiobjective evolutionary algorithms show that OEGADO outperformed them for difficult problems.
---
paper_title: An Evolutionary Strategy for Surrogate-Based Multiobjective Optimization

paper_content:

The paper presents a surrogate-based evolutionary strategy for multiobjective optimization. The evolutionary strategy uses distance based aggregate surrogate models in two ways: as a part of memetic search and as way to pre-select individuals in order to avoid evaluation of bad individuals. The model predicts the distance of individuals to the currently known Pareto set. The newly proposed algorithm is compared to other algorithms which use similar surrogate models on a set of benchmark functions.
---
paper_title: Aggregate meta-models for evolutionary multiobjective and many-objective optimization

paper_content:

Abstract Evolutionary algorithms are among the best multiobjective optimizers. However, they need a large number of function evaluations. In this paper a meta-model based approach to the reduction in the needed number of function evaluations is presented. Local aggregate meta-models are used in a memetic operator. The algorithm is first discussed from a theoretical point of view and then it is shown that the meta-models greatly reduce the number of function evaluations. The approach is compared to a similar one with a single global meta-model as well as to more traditional NSGA-II and ϵ - IBEA . Moreover, it is shown that aggregate meta-models work even for a larger number of objectives and therefore should be considered when designing many-objective evolutionary algorithms.
---

Title: A Review of Surrogate Assisted Multiobjective Evolutionary Algorithms

Section 1: Introduction
Description 1: Write about the overall context and motivation for surrogate-assisted multiobjective evolutionary algorithms, highlighting their significance and challenges.

Section 2: Surrogate Model Techniques
Description 2: Discuss various surrogate modeling techniques used in MOEAs, including polynomial approximation models, Kriging-DACE, radial basis function network, and support vector regression.

Section 3: Taxonomy of Surrogate-Based MOEAs
Description 3: Present the classification of surrogate-based MOEAs according to their working styles, including direct fitness replacement and indirect fitness replacement methods.

Section 3.1: No Evolution Control (NEC)
Description 3.1: Review MOEAs that completely replace real fitness evaluations with surrogate models without using the original fitness function during the evolutionary process.

Section 3.2: Fixed Evolution Control (FEC)
Description 3.2: Review MOEAs that alternate evaluations between real fitness functions and surrogate models following a fixed strategy.

Section 3.3: Adaptive Evolution Control (AEC)
Description 3.3: Review MOEAs that dynamically adjust the frequency of control based on adaptive criteria such as the accuracy of the surrogate model.

Section 4: Indirect Fitness Replacement (IFR) Methods
Description 4: Discuss MOEAs that use surrogate models in an auxiliary way during the evolutionary process, particularly in variation operators or during the generation of initial populations.

Section 5: Discussion
Description 5: Summarize the findings of the review, emphasizing the advantages and disadvantages of each class of surrogate-based MOEAs and the trends observed in the reviewed approaches.

Section 6: Conclusions
Description 6: Conclude with insights on current practices in surrogate-assisted MOEAs, suggesting areas for future research and potential improvements.