Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy
---
paper_title: A Unified View on Hybrid Metaheuristics ?

paper_content:

Manifold possibilities of hybridizing individual metaheuristics with each other and/or with algorithms from other fields exist. A large number of publications documents the benefits and great success of such hybrids. This article overviews several popular hybridization approaches and classifies them based on various characteristics. In particular with respect to low-level hybrids of different metaheuristics, a unified view based on a common pool template is described. It helps in making similarities and different key components of existing metaheuristics explicit. We then consider these key components as a toolbox for building new, effective hybrid metaheuristics. This approach of thinking seems to be superior to sticking too strongly to the philosophies and historical backgrounds behind the different metaheuristic paradigms. Finally, particularly promising possibilities of combining metaheuristics with constraint programming and integer programming techniques are highlighted.
---
paper_title: Improved particle swarm optimization combined with chaos

paper_content:

Abstract As a novel optimization technique, chaos has gained much attention and some applications during the past decade. For a given energy or cost function, by following chaotic ergodic orbits, a chaotic dynamic system may eventually reach the global optimum or its good approximation with high probability. To enhance the performance of particle swarm optimization (PSO), which is an evolutionary computation technique through individual improvement plus population cooperation and competition, hybrid particle swarm optimization algorithm is proposed by incorporating chaos. Firstly, adaptive inertia weight factor (AIWF) is introduced in PSO to efficiently balance the exploration and exploitation abilities. Secondly, PSO with AIWF and chaos are hybridized to form a chaotic PSO (CPSO), which reasonably combines the population-based evolutionary searching ability of PSO and chaotic searching behavior. Simulation results and comparisons with the standard PSO and several meta-heuristics show that the CPSO can effectively enhance the searching efficiency and greatly improve the searching quality.
---
paper_title: Hybrid methods using genetic algorithms for global optimization

paper_content:

This paper discusses the trade-off between accuracy, reliability and computing time in global optimization. Particular compromises provided by traditional methods (Quasi-Newton and Nelder-Mead's simplex methods) and genetic algorithms are addressed and illustrated by a particular application in the field of nonlinear system identification. Subsequently, new hybrid methods are designed, combining principles from genetic algorithms and "hill-climbing" methods in order to find a better compromise to the trade-off. Inspired by biology and especially by the manner in which living beings adapt themselves to their environment, these hybrid methods involve two interwoven levels of optimization, namely evolution (genetic algorithms) and individual learning (Quasi-Newton), which cooperate in a global process of optimization. One of these hybrid methods appears to join the group of state-of-the-art global optimization methods: it combines the reliability properties of the genetic algorithms with the accuracy of Quasi-Newton method, while requiring a computation time only slightly higher than the latter.
---
paper_title: A multi-objective genetic local search algorithm and its application to flowshop scheduling

paper_content:

We propose a hybrid algorithm for finding a set of nondominated solutions of a multi objective optimization problem. In the proposed algorithm, a local search procedure is applied to each solution (i.e., each individual) generated by genetic operations. Our algorithm uses a weighted sum of multiple objectives as a fitness function. The fitness function is utilized when a pair of parent solutions are selected for generating a new solution by crossover and mutation operations. A local search procedure is applied to the new solution to maximize its fitness value. One characteristic feature of our algorithm is to randomly specify weight values whenever a pair of parent solutions are selected. That is, each selection (i.e., the selection of two parent solutions) is performed by a different weight vector. Another characteristic feature of our algorithm is not to examine all neighborhood solutions of a current solution in the local search procedure. Only a small number of neighborhood solutions are examined to prevent the local search procedure from spending almost all available computation time in our algorithm. High performance of our algorithm is demonstrated by applying it to multi objective flowshop scheduling problems.
---
paper_title: DEPSO: hybrid particle swarm with differential evolution operator

paper_content:

A hybrid particle swarm with differential evolution operator, termed DEPSO, which provide the bell-shaped mutations with consensus on the population diversity along with the evolution, while keeping the self-organized particle swarm dynamics, is proposed. Then it is applied to a set of benchmark functions, and the experimental results illustrate its efficiency.
---
paper_title: Evolutionary Programming Made Faster

paper_content:

Evolutionary programming (EP) has been applied with success to many numerical and combinatorial optimization problems in recent years. EP has rather slow convergence rates, however, on some function optimization problems. In the paper, a "fast EP" (FEP) is proposed which uses a Cauchy instead of Gaussian mutation as the primary search operator. The relationship between FEP and classical EP (CEP) is similar to that between fast simulated annealing and the classical version. Both analytical and empirical studies have been carried out to evaluate the performance of FEP and CEP for different function optimization problems. The paper shows that FEP is very good at search in a large neighborhood while CEP is better at search in a small local neighborhood. For a suite of 23 benchmark problems, FEP performs much better than CEP for multimodal functions with many local minima while being comparable to CEP in performance for unimodal and multimodal functions with only a few local minima. The paper also shows the relationship between the search step size and the probability of finding a global optimum and thus explains why FEP performs better than CEP on some functions but not on others. In addition, the importance of the neighborhood size and its relationship to the probability of finding a near-optimum is investigated. Based on these analyses, an improved FEP (IFEP) is proposed and tested empirically. This technique mixes different search operators (mutations). The experimental results show that IFEP performs better than or as well as the better of FEP and CEP for most benchmark problems tested.
---
paper_title: Optimization by simulated annealing

paper_content:

There is a deep and useful connection between statistical mechanics (the behavior of systems with many degrees of freedom in thermal equilibrium at a finite temperature) and multivariate or combinatorial optimization (finding the minimum of a given function depending on many parameters). A detailed analogy with annealing in solids provides a framework for optimization of the properties of very large and complex systems. This connection to statistical mechanics exposes new information and provides an unfamiliar perspective on traditional optimization problems and methods.
---
paper_title: A hybrid of genetic algorithm and particle swarm optimization for recurrent network design

paper_content:

An evolutionary recurrent network which automates the design of recurrent neural/fuzzy networks using a new evolutionary learning algorithm is proposed in this paper. This new evolutionary learning algorithm is based on a hybrid of genetic algorithm (GA) and particle swarm optimization (PSO), and is thus called HGAPSO. In HGAPSO, individuals in a new generation are created, not only by crossover and mutation operation as in GA, but also by PSO. The concept of elite strategy is adopted in HGAPSO, where the upper-half of the best-performing individuals in a population are regarded as elites. However, instead of being reproduced directly to the next generation, these elites are first enhanced. The group constituted by the elites is regarded as a swarm, and each elite corresponds to a particle within it. In this regard, the elites are enhanced by PSO, an operation which mimics the maturing phenomenon in nature. These enhanced elites constitute half of the population in the new generation, whereas the other half is generated by performing crossover and mutation operation on these enhanced elites. HGAPSO is applied to recurrent neural/fuzzy network design as follows. For recurrent neural network, a fully connected recurrent neural network is designed and applied to a temporal sequence production problem. For recurrent fuzzy network design, a Takagi-Sugeno-Kang-type recurrent fuzzy network is designed and applied to dynamic plant control. The performance of HGAPSO is compared to both GA and PSO in these recurrent networks design problems, demonstrating its superiority.
---
paper_title: Particle swarm optimization

paper_content:

A concept for the optimization of nonlinear functions using particle swarm methodology is introduced. The evolution of several paradigms is outlined, and an implementation of one of the paradigms is discussed. Benchmark testing of the paradigm is described, and applications, including nonlinear function optimization and neural network training, are proposed. The relationships between particle swarm optimization and both artificial life and genetic algorithms are described.
---
paper_title: Special Issue on Memetic Algorithms

paper_content:

The ten papers in this special section are devoted to memetic algorithms. The papers are loosely grouped into two categories: memetic algorithm methodologies and domain-specific memetic algorithms. Briefly summarizes the articles included in this section.
---
paper_title: An effective hybrid optimization strategy for job-shop scheduling problems

paper_content:

Abstract Simulated annealing is a naturally serial algorithm, but its behavior can be controlled by the cooling schedule. Genetic algorithm exhibits implicit parallelism and can retain useful redundant information about what is learned from previous searches by its representation in individuals in the population, but GA may lose solutions and substructures due to the disruptive effects of genetic operators and is not easy to regulate GA's convergence. By reasonably combining these two global probabilistic search algorithms, we develop a general, parallel and easily implemented hybrid optimization framework, and apply it to job-shop scheduling problems. Based on effective encoding scheme and some specific optimization operators, some benchmark job-shop scheduling problems are well solved by the hybrid optimization strategy, and the results are competitive with the best literature results. Besides the effectiveness and robustness of the hybrid strategy, the combination of different search mechanisms and structures can relax the parameter-dependence of GA and SA. Scope and purpose Job-shop scheduling problem (JSP) is one of the most well-known machine scheduling problems and one of the strongly NP-hard combinatorial optimization problems. Developing effective search methods is always an important and valuable work. The scope and purpose of this paper is to present a parallel and easily implemented hybrid optimization framework, which reasonably combines genetic algorithm with simulated annealing. Based on effective encoding scheme and some specific optimization operators, the job-shop scheduling problems are well solved by the hybrid optimization strategy.
---
paper_title: Differential Evolution – A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces

paper_content:

A new heuristic approach for minimizing possibly nonlinear and non-differentiable continuous space functions is presented. By means of an extensive testbed it is demonstrated that the new method converges faster and with more certainty than many other acclaimed global optimization methods. The new method requires few control variables, is robust, easy to use, and lends itself very well to parallel computation.
---
paper_title: Meta-Lamarckian learning in memetic algorithms

paper_content:

Over the last decade, memetic algorithms (MAs) have relied on the use of a variety of different methods as the local improvement procedure. Some recent studies on the choice of local search method employed have shown that this choice significantly affects the efficiency of problem searches. Given the restricted theoretical knowledge available in this area and the limited progress made on mitigating the effects of incorrect local search method choice, we present strategies for MA control that decide, at runtime, which local method is chosen to locally improve the next chromosome. The use of multiple local methods during a MA search in the spirit of Lamarckian learning is here termed Meta-Lamarckian learning. Two adaptive strategies for Meta-Lamarckian learning are proposed in this paper. Experimental studies with Meta-Lamarckian learning strategies on continuous parametric benchmark problems are also presented. Further, the best strategy proposed is applied to a real-world aerodynamic wing design problem and encouraging results are obtained. It is shown that the proposed approaches aid designers working on complex engineering problems by reducing the probability of employing inappropriate local search methods in a MA, while at the same time, yielding robust and improved design search performance.
---
paper_title: Linear Programming and Extensions

paper_content:

In real-world problems related to finance, business, and management, mathematicians and economists frequently encounter optimization problems. In this classic book, George Dantzig looks at a wealth of examples and develops linear programming methods for their solutions. He begins by introducing the basic theory of linear inequalities and describes the powerful simplex method used to solve them. Treatments of the price concept, the transportation problem, and matrix methods are also given, and key mathematical concepts such as the properties of convex sets and linear vector spaces are covered."The author of this book was the main force in establishing a new mathematical discipline, and he has contributed to its further development at every stage and from every angle. This volume ... is a treasure trove for those who work in this field--teachers, students, and users alike. Its encyclopaedic coverage, due in part to collaboration with other experts, makes it an absolute must."--S. Vajda, Zentralblatt fYr Mathematik und ihre Grenzgebiete
---
paper_title: A Taxonomy of Hybrid Metaheuristics

paper_content:

Hybrid metaheuristics have received considerable interest these recent years in the field of combinatorial optimization. A wide variety of hybrid approaches have been proposed in the literature. In this paper, a taxonomy of hybrid metaheuristics is presented in an attempt to provide a common terminology and classification mechanisms. The taxonomy, while presented in terms of metaheuristics, is also applicable to most types of heuristics and exact optimization algorithms. ::: ::: As an illustration of the usefulness of the taxonomy an annoted bibliography is given which classifies a large number of hybrid approaches according to the taxonomy.
---
paper_title: Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation

paper_content:

List of Figures. List of Tables. Preface. Contributing Authors. Series Foreword. Part I: Foundations. 1. An Introduction to Evolutionary Algorithms J.A. Lozano. 2. An Introduction to Probabilistic Graphical Models P. Larranaga. 3. A Review on Estimation of Distribution Algorithms P. Larranaga. 4. Benefits of Data Clustering in Multimodal Function Optimization via EDAs J.M. Pena, et al. 5. Parallel Estimation of Distribution Algorithms J.A. Lozano, et al. 6. Mathematical Modeling of Discrete Estimation of Distribution Algorithms C. Gonzalez, et al. Part II: Optimization. 7. An Empiricial Comparison of Discrete Estimation of Distribution Algorithms R. Blanco., J.A. Lozano. 8. Results in Function Optimization with EDAs in Continuous Domain E. Bengoetxea, et al. 9. Solving the 0-1 Knapsack Problem with EDAs R. Sagarna, P. Larranaga. 10. Solving the Traveling Salesman Problem with EDAs V. Robles, et al. 11. EDAs Applied to the Job Shop Scheduling Problem J.A. Lozano, A. Mendiburu. 12. Solving Graph Matching with EDAs Using a Permutation-Based Representation E. Bengoetxea, et al. Part III: Machine Learning. 13. Feature Subset Selection by Estimation of Distribution Algorithms I. Inza, et al. 14. Feature Weighting for Nearest Neighbor by EDAs I. Inza, et al. 15. Rule Induction by Estimation of Distribution Algorithms B. Sierra, et al. 16. Partial Abductive Inference in Bayesian Networks: An Empirical Comparison Between GAs and EDAs L.M. de Campos, et al.17. Comparing K-Means, GAs and EDAs in Partitional Clustering J. Roure, et al. 18. Adjusting Weights in Artificial Neural Networks using Evolutionary Algorithms C. Cotta, et al. Index.
---
paper_title: A tutorial for competent memetic algorithms: model, taxonomy, and design issues

paper_content:

The combination of evolutionary algorithms with local search was named "memetic algorithms" (MAs) (Moscato, 1989). These methods are inspired by models of natural systems that combine the evolutionary adaptation of a population with individual learning within the lifetimes of its members. Additionally, MAs are inspired by Richard Dawkin's concept of a meme, which represents a unit of cultural evolution that can exhibit local refinement (Dawkins, 1976). In the case of MA's, "memes" refer to the strategies (e.g., local refinement, perturbation, or constructive methods, etc.) that are employed to improve individuals. In this paper, we review some works on the application of MAs to well-known combinatorial optimization problems, and place them in a framework defined by a general syntactic model. This model provides us with a classification scheme based on a computable index D, which facilitates algorithmic comparisons and suggests areas for future research. Also, by having an abstract model for this class of metaheuristics, it is possible to explore their design space and better understand their behavior from a theoretical standpoint. We illustrate the theoretical and practical relevance of this model and taxonomy for MAs in the context of a discussion of important design issues that must be addressed to produce effective and efficient MAs.
---
paper_title: Hybrid Metaheuristics: An Emerging Approach to Optimization

paper_content:

Optimization problems are of great importance in many fields. They can be tackled, for example, by approximate algorithms such as metaheuristics. Examples of metaheuristics are simulated annealing, tabu search, evolutionary computation, iterated local search, variable neighborhood search, and ant colony optimization. In recent years it has become evident that a skilled combination of a metaheuristic with other optimization techniques, a so called hybrid metaheuristic, can provide a more efficient behavior and a higher flexibility. This is because hybrid metaheuristics combine their advantages with the complementary strengths of, for example, more classical optimization techniques such as branch and bound or dynamic programming. The authors involved in this book are among the top researchers in their domain. The book is intended both to provide an overview of hybrid metaheuristics to novices of the field, and to provide researchers from the field with a collection of some of the most interesting recent developments.
---
paper_title: Ant Colony Optimization

paper_content:

Swarm intelligence is a relatively new approach to problem solving that takes inspiration from the social behaviors of insects and of other animals. In particular, ants have inspired a number of methods and techniques among which the most studied and the most successful is the general purpose optimization technique known as ant colony optimization. Ant colony optimization (ACO) takes inspiration from the foraging behavior of some ant species. These ants deposit pheromone on the ground in order to mark some favorable path that should be followed by other members of the colony. Ant colony optimization exploits a similar mechanism for solving optimization problems. From the early nineties, when the first ant colony optimization algorithm was proposed, ACO attracted the attention of increasing numbers of researchers and many successful applications are now available. Moreover, a substantial corpus of theoretical results is becoming available that provides useful guidelines to researchers and practitioners in further applications of ACO. The goal of this article is to introduce ant colony optimization and to survey its most notable applications
---
paper_title: DE/EDA: A new evolutionary algorithm for global optimization

paper_content:

Differential evolution (DE) was very successful in solving the global continuous optimization problem. It mainly uses the distance and direction information from the current population to guide its further search. Estimation of distribution algorithm (EDA) samples new solutions from a probability model which characterizes the distribution of promising solutions. This paper proposes a combination of DE and EDA (DE/EDA) for the global continuous optimization problem. DE/EDA combines global information extracted by EDA with differential information obtained by DE to create promising solutions. DE/EDA has been compared with the best version of the DE algorithm and an EDA on several commonly utilized test problems. Experimental results demonstrate that DE/EDA outperforms the DE algorithm and the EDA. The effect of the parameters of DE/EDA to its performance is investigated experimentally.
---
paper_title: Metaheuristics in combinatorial optimization: Overview and conceptual comparison

paper_content:

The field of metaheuristics for the application to combinatorial optimization problems is a rapidly growing field of research. This is due to the importance of combinatorial optimization problems for the scientific as well as the industrial world. We give a survey of the nowadays most important metaheuristics from a conceptual point of view. We outline the different components and concepts that are used in the different metaheuristics in order to analyze their similarities and differences. Two very important concepts in metaheuristics are intensification and diversification. These are the two forces that largely determine the behavior of a metaheuristic. They are in some way contrary but also complementary to each other. We introduce a framework, that we call the I&D frame, in order to put different intensification and diversification components into relation with each other. Outlining the advantages and disadvantages of different metaheuristic approaches we conclude by pointing out the importance of hybridization of metaheuristics as well as the integration of metaheuristics and other methods for optimization.
---
paper_title: Differential Evolution: A Survey of the State-of-the-Art

paper_content:

Differential evolution (DE) is arguably one of the most powerful stochastic real-parameter optimization algorithms in current use. DE operates through similar computational steps as employed by a standard evolutionary algorithm (EA). However, unlike traditional EAs, the DE-variants perturb the current-generation population members with the scaled differences of randomly selected and distinct population members. Therefore, no separate probability distribution has to be used for generating the offspring. Since its inception in 1995, DE has drawn the attention of many researchers all over the world resulting in a lot of variants of the basic algorithm with improved performance. This paper presents a detailed review of the basic concepts of DE and a survey of its major variants, its application to multiobjective, constrained, large scale, and uncertain optimization problems, and the theoretical studies conducted on DE so far. Also, it provides an overview of the significant engineering applications that have benefited from the powerful nature of DE.
---
paper_title: Timetable Synchronization of Mass Rapid Transit System Using Multiobjective Evolutionary Approach

paper_content:

Users of mass rapid transit are often required to make transfers between different train lines to reach their destinations. Timetable synchronization minimizes delays during such transfers. This paper formulates a novel measure for timetable synchronization by means of a total passenger dissatisfaction index (TPDI); and the impact of such synchronization on the original unsynchronized timetable is accounted using a total deviation index (TDV) that assigns penalties when deviations from the original timetable are incurred. Pareto fronts displaying the relationship between TPDI and TDV are generated using the state-of-the-art nondominated sorting genetic algorithm 2 (NSGA 2). To further improve NSGA 2, three schemes-the use of a variant of the NSGA2 with differential evolution, a process we termed "seeding," and finally a hybrid combination with local search techniques like heuristic hill climbing, tabu search, and simulated annealing - are proposed. Simulation results demonstrate that the "seeded" NSGA2-DE combined with the hill climbing heuristic produce the best results for the application. Solutions from the Pareto fronts are chosen for implementation to describe the different operating regions. A discussion section details the advantages and drawbacks of the proposed schemes.
---
paper_title: Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series)

paper_content:

Problems demanding globally optimal solutions are ubiquitous, yet many are intractable when they involve constrained functions having many local optima and interacting, mixed-type variables.The differential evolution (DE) algorithm is a practical approach to global numerical optimization which is easy to understand, simple to implement, reliable, and fast. Packed with illustrations, computer code, new insights, and practical advice, this volume explores DE in both principle and practice. It is a valuable resource for professionals needing a proven optimizer and for students wanting an evolutionary perspective on global numerical optimization.
---
paper_title: Automatic Clustering Using an Improved Differential Evolution Algorithm

paper_content:

Differential evolution (DE) has emerged as one of the fast, robust, and efficient global search heuristics of current interest. This paper describes an application of DE to the automatic clustering of large unlabeled data sets. In contrast to most of the existing clustering techniques, the proposed algorithm requires no prior knowledge of the data to be classified. Rather, it determines the optimal number of partitions of the data "on the run." Superiority of the new method is demonstrated by comparing it with two recently developed partitional clustering techniques and one popular hierarchical clustering algorithm. The partitional clustering algorithms are based on two powerful well-known optimization algorithms, namely the genetic algorithm and the particle swarm optimization. An interesting real-world application of the proposed method to automatic segmentation of images is also reported.
---
paper_title: Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization

paper_content:

Differential evolution (DE) is an efficient and powerful population-based stochastic search technique for solving optimization problems over continuous space, which has been widely applied in many scientific and engineering fields. However, the success of DE in solving a specific problem crucially depends on appropriately choosing trial vector generation strategies and their associated control parameter values. Employing a trial-and-error scheme to search for the most suitable strategy and its associated parameter settings requires high computational costs. Moreover, at different stages of evolution, different strategies coupled with different parameter settings may be required in order to achieve the best performance. In this paper, we propose a self-adaptive DE (SaDE) algorithm, in which both trial vector generation strategies and their associated control parameter values are gradually self-adapted by learning from their previous experiences in generating promising solutions. Consequently, a more suitable generation strategy along with its parameter settings can be determined adaptively to match different phases of the search process/evolution. The performance of the SaDE algorithm is extensively evaluated (using codes available from P. N. Suganthan) on a suite of 26 bound-constrained numerical optimization problems and compares favorably with the conventional DE and several state-of-the-art parameter adaptive DE variants.
---
paper_title: Differential Evolution – A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces

paper_content:

A new heuristic approach for minimizing possibly nonlinear and non-differentiable continuous space functions is presented. By means of an extensive testbed it is demonstrated that the new method converges faster and with more certainty than many other acclaimed global optimization methods. The new method requires few control variables, is robust, easy to use, and lends itself very well to parallel computation.
---
paper_title: A Particle Swarm Optimization Algorithm with Differential Evolution

paper_content:

Differential evolution (DE) is a simple evolutionary algorithm that has shown superior performance in the global continuous optimization. It mainly utilizes the differential information to guide its further search. But the differential information also results in instability of performance. Particle swarm optimization (PSO) has been developing rapidly and has been applied widely since it is introduced, as it can converge quickly. But PSO easily got stuck in local optima because it easily loses the diversity of swarm. This paper proposes a combination of DE and PSO (termed DEPSO) that makes up their disadvantages. DEPSO combines the differential information obtained by DE with the memory information extracted by PSO to create the promising solutions. Finally, DEPSO is tested to solve several benchmark optimization problems. The experimental results show the effectiveness of DEPSO algorithm for the multimodal function, and also verify that DEPSO can perform better than other algorithms (DE, CPSO) in solving the benchmark problems.
---
paper_title: Nonlinear System Control Using Adaptive Neural Fuzzy Networks Based on a Modified Differential Evolution

paper_content:

This study presents an adaptive neural fuzzy network (ANFN) controller based on a modified differential evolution (MODE) for solving control problems. The proposed ANFN controller adopts a functional link neural network as the consequent part of the fuzzy rules. Thus, the consequent part of the ANFN controller is a nonlinear combination of input variables. The proposed MODE learning algorithm adopts an evolutionary learning method to optimize the controller parameters. For design optimization, a new criterion is introduced. A hardware-in-the loop control technique is developed and applied to the designed ANFN controller using the MODE learning algorithm. The proposed ANFN controller with the MODE learning algorithm (ANFN-MODE) is used in two practical applications-the planetary-train-type inverted pendulum system and the magnetic levitation system. The experiment is developed in a real-time visual simulation environment. Experimental results of this study have demonstrated the robustness and effectiveness of the proposed ANFN-MODE controller.
---
paper_title: Self-Adapting Control Parameters in Differential Evolution: A Comparative Study on Numerical Benchmark Problems

paper_content:

We describe an efficient technique for adapting control parameter settings associated with differential evolution (DE). The DE algorithm has been used in many practical cases and has demonstrated good convergence properties. It has only a few control parameters, which are kept fixed throughout the entire evolutionary process. However, it is not an easy task to properly set control parameters in DE. We present an algorithm-a new version of the DE algorithm-for obtaining self-adaptive control parameter settings that show good performance on numerical benchmark problems. The results show that our algorithm with self-adaptive control parameter settings is better than, or at least comparable to, the standard DE algorithm and evolutionary algorithms from literature when considering the quality of the solutions obtained
---
paper_title: Advances in Differential Evolution

paper_content:

Differential evolution is arguably one of the hottest topics in today's computational intelligence research. This book seeks to present a comprehensive study of the state of the art in this technology and also directions for future research. The fourteen chapters of this book have been written by leading experts in the area. The first seven chapters focus on algorithm design, while the last seven describe real-world applications. Chapter 1 introduces the basic differential evolution (DE) algorithm and presents a broad overview of the field. Chapter 2 presents a new, rotationally invariant DE algorithm. The role of self-adaptive control parameters in DE is investigated in Chapter 3. Chapters 4 and 5 address constrained optimization; the former develops suitable stopping conditions for the DE run, and the latter presents an improved DE algorithm for problems with very small feasible regions. A novel DE algorithm, based on the concept of "opposite" points, is the topic of Chapter 6. Chapter 7 provides a survey of multi-objective differential evolution algorithms. A review of the major application areas of differential evolution is presented in Chapter 8. Chapter 9 discusses the application of differential evolution in two important areas of applied electromagnetics. Chapters 10 and 11 focus on applications of hybrid DE algorithms to problems in power system optimization. Chapter 12 applies the DE algorithm to computer chess. The use of DE to solve a problem in bioprocess engineering is discussed in Chapter 13. Chapter 14 describes the application of hybrid differential evolution to a problem in control engineering.
---
paper_title: Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance

paper_content:

The study manipulated the neighborhood topologies of particle swarms optimizing four test functions. Several social network structures were tested, with "small-world" randomization of a specified number of links. Sociometric structure and the small-world manipulation interacted with function to produce a significant effect on performance.
---
paper_title: Evolutionary Tristate PSO for Strategic Bidding of Pumped-Storage Hydroelectric Plant

paper_content:

This paper develops bidding strategy for operating multiunit pumped-storage power plant in a day-ahead electricity market. Based on forecasted hourly market clearing price, the objective is to self-schedule and maximize the expected profit of the pumped-storage plant, considering both spinning and nonspinning reserve bids and meeting the technical operating constraints. Evolutionary tristate particle swarm optimization (ETPSO) based approach is proposed to solve the problem, combining basic particle swarm optimization (PSO) with tristate coding technique and genetics-based mutation operation. The discrete characteristic of a pumped-storage plant is modeled using tristate coding technique and mutation operation is used for faster convergence. The proposed model is adaptive for nonlinear 3-D relationship between the power produced, the energy stored, and the head of the associated reservoir. The proposed approach is applied for a practical utility consisting of four units. Simulation results for different operating cycles of the storage plant indicate the attractive properties of ETPSO approach with highly optimal solution and robust convergence behavior.
---
paper_title: Comparing inertia weights and constriction factors in particle swarm optimization

paper_content:

The performance of particle swarm optimization using an inertia weight is compared with performance using a constriction factor. Five benchmark functions are used for the comparison. It is concluded that the best approach is to use the constriction factor while limiting the maximum velocity Vmax to the dynamic range of the variable Xmax on each dimension. This approach provides performance on the benchmark functions superior to any other published results known by the authors.
---
paper_title: Neighborhood topologies in fully informed and best-of-neighborhood particle swarms

paper_content:

We vary the way an individual in the particle swarm interacts with its neighbors. Performance depends on population topology as well as algorithm version.
---
paper_title: A Hybrid of Cooperative Particle Swarm Optimization and Cultural Algorithm for Neural Fuzzy Networks and Its Prediction Applications

paper_content:

This study presents an evolutionary neural fuzzy network, designed using the functional-link-based neural fuzzy network (FLNFN) and a new evolutionary learning algorithm. This new evolutionary learning algorithm is based on a hybrid of cooperative particle swarm optimization and cultural algorithm. It is thus called cultural cooperative particle swarm optimization (CCPSO). The proposed CCPSO method, which uses cooperative behavior among multiple swarms, can increase the global search capacity using the belief space. Cooperative behavior involves a collection of multiple swarms that interact by exchanging information to solve a problem. The belief space is the information repository in which the individuals can store their experiences such that other individuals can learn from them indirectly. The proposed FLNFN model uses functional link neural networks as the consequent part of the fuzzy rules. This study uses orthogonal polynomials and linearly independent functions in a functional expansion of the functional link neural networks. The FLNFN model can generate the consequent part of a nonlinear combination of input variables. Finally, the proposed FLNFN with CCPSO (FLNFN-CCPSO) is adopted in several predictive applications. Experimental results have demonstrated that the proposed CCPSO method performs well in predicting the time series problems.
---
paper_title: Particle swarm optimization

paper_content:

A concept for the optimization of nonlinear functions using particle swarm methodology is introduced. The evolution of several paradigms is outlined, and an implementation of one of the paradigms is discussed. Benchmark testing of the paradigm is described, and applications, including nonlinear function optimization and neural network training, are proposed. The relationships between particle swarm optimization and both artificial life and genetic algorithms are described.
---
paper_title: Population structure and particle swarm performance

paper_content:

The effects of various population topologies on the particle swarm algorithm were systematically investigated. Random graphs were generated to specifications, and their performance on several criteria was compared. What makes a good population structure? We discovered that previous assumptions may not have been correct.
---
paper_title: Comprehensive learning particle swarm optimizer for global optimization of multimodal functions

paper_content:

This paper presents a variant of particle swarm optimizers (PSOs) that we call the comprehensive learning particle swarm optimizer (CLPSO), which uses a novel learning strategy whereby all other particles' historical best information is used to update a particle's velocity. This strategy enables the diversity of the swarm to be preserved to discourage premature convergence. Experiments were conducted (using codes available from http://www.ntu.edu.sg/home/epnsugan) on multimodal test functions such as Rosenbrock, Griewank, Rastrigin, Ackley, and Schwefel and composition functions both with and without coordinate rotation. The results demonstrate good performance of the CLPSO in solving multimodal problems when compared with eight other recent variants of the PSO.
---
paper_title: A Cooperative approach to particle swarm optimization

paper_content:

The particle swarm optimizer (PSO) is a stochastic, population-based optimization technique that can be applied to a wide range of problems, including neural network training. This paper presents a variation on the traditional PSO algorithm, called the cooperative particle swarm optimizer, or CPSO, employing cooperative behavior to significantly improve the performance of the original algorithm. This is achieved by using multiple swarms to optimize different components of the solution vector cooperatively. Application of the new PSO algorithm on several benchmark optimization problems shows a marked improvement in performance over the traditional PSO.
---
paper_title: A Unified View on Hybrid Metaheuristics ?

paper_content:

Manifold possibilities of hybridizing individual metaheuristics with each other and/or with algorithms from other fields exist. A large number of publications documents the benefits and great success of such hybrids. This article overviews several popular hybridization approaches and classifies them based on various characteristics. In particular with respect to low-level hybrids of different metaheuristics, a unified view based on a common pool template is described. It helps in making similarities and different key components of existing metaheuristics explicit. We then consider these key components as a toolbox for building new, effective hybrid metaheuristics. This approach of thinking seems to be superior to sticking too strongly to the philosophies and historical backgrounds behind the different metaheuristic paradigms. Finally, particularly promising possibilities of combining metaheuristics with constraint programming and integer programming techniques are highlighted.
---
paper_title: An Effective PSO-Based Hybrid Algorithm for Multiobjective Permutation Flow Shop Scheduling

paper_content:

This paper proposes a hybrid algorithm based on particle swarm optimization (PSO) for a multiobjective permutation flow shop scheduling problem, which is a typical NP-hard combinatorial optimization problem with strong engineering backgrounds. Not only does the proposed multiobjective algorithm (named MOPSO) apply the parallel evolution mechanism of PSO characterized by individual improvement, population cooperation, and competition to effectively perform exploration but it also utilizes several adaptive local search methods to perform exploitation. First, to make PSO suitable for solving scheduling problems, a ranked-order value (ROV) rule based on a random key technique to convert the continuous position values of particles to job permutations is presented. Second, a multiobjective local search based on the Nawaz-Enscore-Ham heuristic is applied to good solutions with a specified probability to enhance the exploitation ability. Third, to enrich the searching behavior and to avoid premature convergence, a multiobjective local search based on simulated annealing with multiple different neighborhoods is designed, and an adaptive meta-Lamarckian learning strategy is employed to decide which neighborhood will be used. Due to the fusion of multiple different searching operations, good solutions approximating the real Pareto front can be obtained. In addition, MOPSO adopts a random weighted linear sum function to aggregate multiple objectives to a single one for solution evaluation and for guiding the evolution process in the multiobjective sense. Due to the randomness of weights, searching direction can be enriched, and solutions with good diversity can be obtained. Simulation results and comparisons based on a variety of instances demonstrate the effectiveness, efficiency, and robustness of the proposed hybrid algorithm.
---
paper_title: Self-adaptive population sizing for a tune-free differential evolution

paper_content:

The study and research of evolutionary algorithms (EAs) is getting great attention in recent years. Although EAs have earned extensive acceptance through numerous successful applications in many fields, the problem of finding the best combination of evolutionary parameters especially for population size that need the manual settings by the user is still unresolved. In this paper, our system is focusing on differential evolution (DE) and its control parameters. To overcome the problem, two new systems were carried out for the self-adaptive population size to test two different methodologies (absolute encoding and relative encoding) in DE and compared their performances against the original DE. Fifty runs are conducted for every 20 well-known benchmark problems to test on every proposed algorithm in this paper to achieve the function optimization without explicit parameter tuning in DE. The empirical testing results showed that DE with self-adaptive population size using relative encoding performed well in terms of the average performance as well as stability compared to absolute encoding version as well as the original DE.
---
paper_title: An effective hybrid DE-based algorithm for multi-objective flow shop scheduling with limited buffers

paper_content:

This paper proposes an effective hybrid algorithm based on differential evolution (DE), namely HDE, to solve multi-objective permutation flow shop scheduling problem (MPFSSP) with limited buffers between consecutive machines, which is a typical NP-hard combinatorial optimization problem with strong engineering background. Firstly, to make DE suitable for solving scheduling problems, a largest-order-value (LOV) rule is presented to convert the continuous values of individuals in DE to job permutations. Secondly, after the DE-based exploration, an efficient local search, which is designed based on the landscape of MPFSSP with limited buffers, is applied to emphasize exploitation. Thus, not only does the HDE apply the parallel evolution mechanism of DE to perform effective exploration (global search) in the whole solution space, but it also adopts problem-dependent local search to perform thorough exploitation (local search) in the promising sub-regions. In addition, the concept of Pareto dominance is used to handle the updating of solutions in sense of multi-objective optimization. Moreover, the convergence property of HDE is analyzed by using the theory of finite Markov chain. Finally, simulations and comparisons based on benchmarks demonstrate the effectiveness and efficiency of the proposed HDE.
---
paper_title: A Differential Free Point Generation Scheme in the Differential Evolution Algorithm

paper_content:

In this paper we derive the probability distribution of trial points in the differential evolution (de) algorithm, in particular the probability distribution of points generated by mutation. We propose a point generation scheme that uses an approximation to this distribution. The scheme can dispense with the differential vector used in the mutation of de. We propose a de algorithm that replaces the differential based mutation scheme with a probability distribution based point generation scheme. We also propose a de algorithm that uses a probabilistic combination of the point generation by the probability distribution and the point generation by mutation. A numerical study is carried out using a set of 50 test problems, many of which are inspired by practical applications. Numerical results suggest that the new algorithms are superior to the original version both in terms of the number of function evaluations and cpu times.
---
paper_title: Adaptive memory programming: A unified view of metaheuristics

paper_content:

The paper analyses recent developments of a number of memory-based metaheuristics such as taboo search, scatter search, genetic algorithms and ant colonies. Its shows that the implementations of these general solving methods are more and more similar. So, a unified presentation is proposed under the name of Adaptive Memory Programming (AMP). A number of methods recently developed for the quadratic assignment, vehicle routing and graph colouring problems are reviewed and presented under the adaptive memory programming point of view. AMP presents a number of interesting aspects such as a high parallelization potential and theability of dealing with real and dynamic applications.
---
paper_title: Efficient Population Utilization Strategy for Particle Swarm Optimizer

paper_content:

The particle swarm optimizer (PSO) is a population-based optimization technique that can be applied to a wide range of problems. This paper presents a variation on the traditional PSO algorithm, called the efficient population utilization strategy for PSO (EPUS-PSO), adopting a population manager to significantly improve the efficiency of PSO. This is achieved by using variable particles in swarms to enhance the searching ability and drive particles more efficiently. Moreover, sharing principals are constructed to stop particles from falling into the local minimum and make the global optimal solution easier found by particles. Experiments were conducted on unimodal and multimodal test functions such as quadric, griewanks, rastrigin, ackley, and weierstrass, with and without coordinate rotation. The results show good performance of the EPUS-PSO in solving most benchmark problems as compared to other recent variants of the PSO.
---
paper_title: PSO-Based Multiobjective Optimization With Dynamic Population Size and Adaptive Local Archives

paper_content:

Recently, various multiobjective particle swarm optimization (MOPSO) algorithms have been developed to efficiently and effectively solve multiobjective optimization problems. How ever, the existing MOPSO designs generally adopt a notion to "estimate" a fixed population size sufficiently to explore the search space without incurring excessive computational complexity. To address the issue, this paper proposes the integration of a dynamic population strategy within the multiple-swarm MOPSO. The proposed algorithm is named dynamic population multiple-swarm MOPSO. An additional feature, adaptive local archives, is designed to improve the diversity within each swarm. Performance metrics and benchmark test functions are used to examine the performance of the proposed algorithm compared with that of five selected MOPSOs and two selected multiobjective evolutionary algorithms. In addition, the computational cost of the proposed algorithm is quantified and compared with that of the selected MOPSOs. The proposed algorithm shows competitive results with improved diversity and convergence and demands less computational cost.
---
paper_title: A unified optimization framework for population-based methods

paper_content:

Combinatorial optimization problems arise in many applications such as task assignment, facility location, and elevator scheduling. A wide variety of population-based solution methods have been developed, either instance-based (e.g., genetic algorithm (GA) and particle swarm optimization (PSO)) or model-based (e.g., ant colony optimization(ACO) and estimation of distribution algorithms (EDAs)). Their various mechanisms make it difficult to analyze and compare these methods and to extend the advancement in one method to another. To this end, a unified optimization framework towards representing these seemingly different methods is established as iteratively sampling and updating of a population distribution. This framework is then innovatively instantiated with PSO from the instance-based category and EDA from the model-based category. Finally, the possible use and the finite time performance analysis of the unified framework are discussed.
---
paper_title: Accelerating Differential Evolution Using an Adaptive Local Search

paper_content:

We propose a crossover-based adaptive local search (LS) operation for enhancing the performance of standard differential evolution (DE) algorithm. Incorporating LS heuristics is often very useful in designing an effective evolutionary algorithm for global optimization. However, determining a single LS length that can serve for a wide range of problems is a critical issue. We present a LS technique to solve this problem by adaptively adjusting the length of the search, using a hill-climbing heuristic. The emphasis of this paper is to demonstrate how this LS scheme can improve the performance of DE. Experimenting with a wide range of benchmark functions, we show that the proposed new version of DE, with the adaptive LS, performs better, or at least comparably, to classic DE algorithm. Performance comparisons with other LS heuristics and with some other well-known evolutionary algorithms from literature are also presented.
---
paper_title: Exploring dynamic self-adaptive populations in differential evolution

paper_content:

Although the Differential Evolution (DE) algorithm has been shown to be a simple yet powerful evolutionary algorithm for optimizing continuous functions, users are still faced with the problem of preliminary testing and hand-tuning of the evolutionary parameters prior to commencing the actual optimization process. As a solution, self-adaptation has been found to be highly beneficial in automatically and dynamically adjusting evolutionary parameters such as crossover rates and mutation rates. In this paper, we present a first attempt at self-adapting the population size parameter in addition to self-adapting crossover and mutation rates. Firstly, our main objective is to demonstrate the feasibility of self-adapting the population size parameter in DE. Using De Jong's F1–F5 benchmark test problems, we showed that DE with self-adaptive populations produced highly competitive results compared to a conventional DE algorithm with static populations. In addition to reducing the number of parameters used in DE, the proposed algorithm actually outperformed the conventional DE algorithm for one of the test problems. It was also found that that an absolute encoding methodology for self-adapting population size in DE produced results with greater optimization reliability compared to a relative encoding methodology.
---
paper_title: SWAF: Swarm Algorithm Framework for Numerical Optimization

paper_content:

A swarm algorithm framework (SWAF), realized by agent-based modeling, is presented to solve numerical optimization problems. Each agent is a bare bones cognitive architecture, which learns knowledge by appropriately deploying a set of simple rules in fast and frugal heuristics. Two essential categories of rules, the generate-and-test and the problem-formulation rules, are implemented, and both of the macro rules by simple combination and subsymbolic deploying of multiple rules among them are also studied. Experimental results on benchmark problems are presented, and performance comparison between SWAF and other existing algorithms indicates that it is efficiently.
---
paper_title: Bare bones particle swarms

paper_content:

The particle swarm algorithm has just enough moving parts to make it hard to understand. The formula is very simple, it is even easy to describe the working of the algorithm verbally, yet it is very difficult to grasp in one's mind how the particles oscillate around centers that are constantly changing; how they influence one another; how the various parameters affect the trajectory of the particle; how the topology of the swarm affects its performance; and so on. This paper strips away some traditional features of the particle swarm in the search for the properties that make it work. The particle swarm algorithm is modified by eliminating the velocity formula. Variations are compared. In the process some of the mysteries of the algorithm are revealed, we discover its similarity to other stochastic population-based problem solving methods, and new avenues of investigation are suggested or implied.
---
paper_title: Population size reduction for the differential evolution algorithm

paper_content:

This paper studies the efficiency of a recently defined population-based direct global optimization method called Differential Evolution with self-adaptive control parameters. The original version uses fixed population size but a method for gradually reducing population size is proposed in this paper. It improves the efficiency and robustness of the algorithm and can be applied to any variant of a Differential Evolution algorithm. The proposed modification is tested on commonly used benchmark problems for unconstrained optimization and compared with other optimization methods such as Evolutionary Algorithms and Evolution Strategies.
---
paper_title: Mean and Variance of the Sampling Distribution of Particle Swarm Optimizers During Stagnation

paper_content:

Several theoretical analyses of the dynamics of particle swarms have been offered in the literature over the last decade. Virtually all rely on substantial simplifications, often including the assumption that the particles are deterministic. This has prevented the exact characterization of the sampling distribution of the particle swarm optimizer (PSO). In this paper we introduce a novel method that allows us to exactly determine all the characteristics of a PSO sampling distribution and explain how it changes over any number of generations, in the presence stochasticity. The only assumption we make is stagnation, i.e., we study the sampling distribution produced by particles in search for a better personal best. We apply the analysis to the PSO with inertia weight, but the analysis is also valid for the PSO with constriction and other forms of PSO.
---
paper_title: CONTROLLED POOL MAINTENANCE FOR METAHEURISTICS

paper_content:

Recent metaheuristic developments have proved to be successful especially in cases where their fundamental concepts are complemented with pool-oriented approaches such as scatter search. These algorithms maintain a reference set of high quality solutions which are repeatedly used during the search in order to guarantee a fruitful balance between diversification and intensification. However, maintaining such a pool is not a trivial undertaking. The main problem is to find a balance between the attempt to collect a number of high quality solutions, which often results in similar solution properties, and the need to guarantee a certain degree of diversity in the pool. In the present study we highlight the advantage of pool-oriented design while focusing on controlled input and output operations. We analyze the main pool components for metaheuristics and present a review of the literature on relevant strategies in tabu search, scatter search, and path relinking. Additionally, we provide a synthesis of the literature that highlights the critical aspects of effective pool maintenance.
---
paper_title: An Effective PSO-Based Memetic Algorithm for Flow Shop Scheduling

paper_content:

This paper proposes an effective particle swarm optimization (PSO)-based memetic algorithm (MA) for the permutation flow shop scheduling problem (PFSSP) with the objective to minimize the maximum completion time, which is a typical non-deterministic polynomial-time (NP) hard combinatorial optimization problem. In the proposed PSO-based MA (PSOMA), both PSO-based searching operators and some special local searching operators are designed to balance the exploration and exploitation abilities. In particular, the PSOMA applies the evolutionary searching mechanism of PSO, which is characterized by individual improvement, population cooperation, and competition to effectively perform exploration. On the other hand, the PSOMA utilizes several adaptive local searches to perform exploitation. First, to make PSO suitable for solving PFSSP, a ranked-order value rule based on random key representation is presented to convert the continuous position values of particles to job permutations. Second, to generate an initial swarm with certain quality and diversity, the famous Nawaz-Enscore-Ham (NEH) heuristic is incorporated into the initialization of population. Third, to balance the exploration and exploitation abilities, after the standard PSO-based searching operation, a new local search technique named NEH_1 insertion is probabilistically applied to some good particles selected by using a roulette wheel mechanism with a specified probability. Fourth, to enrich the searching behaviors and to avoid premature convergence, a simulated annealing (SA)-based local search with multiple different neighborhoods is designed and incorporated into the PSOMA. Meanwhile, an effective adaptive meta-Lamarckian learning strategy is employed to decide which neighborhood to be used in SA-based local search. Finally, to further enhance the exploitation ability, a pairwise-based local search is applied after the SA-based search. Simulation results based on benchmarks demonstrate the effectiveness of the PSOMA. Additionally, the effects of some parameters on optimization performances are also discussed
---
paper_title: A Taxonomy of Evolutionary Algorithms in Combinatorial Optimization

paper_content:

This paper shows how evolutionary algorithms can be described in a concise, yet comprehensive and accurate way. A classification scheme is introduced and presented in a tabular form called TEA (Table of Evolutionary Algorithms). It distinguishes between different classes of evolutionary algorithms (e.g., genetic algorithms, ant systems) by enumerating the fundamental ingredients of each of these algorithms. At the end, possible uses of the TEA are illustrated on classical evolutionary algorithms.
---
paper_title: THE PARTICLE SWARM AS COLLABORATIVE SAMPLING OF THE SEARCH SPACE

paper_content:

The particle swarm algorithm uses principles derived from social psychology to find optimal points in a search space. The present paper decomposes and reinterprets the particle swarm in order to discover new ways of implementing the algorithm. Some essential characteristics of the method are illuminated, and some inessential features are discarded. Various new forms are tested and found to perform well on a suite of test functions. In particular, it is shown that the traditional trajectory formulas can be replaced with random number generators sampling from various symmetrical probability distributions. The excellent performance of these new versions demonstrates that the strength of the algorithm is in the interactions of the particles, rather than in their behavior as individuals.
---
paper_title: Classification of adaptive memetic algorithms: a comparative study

paper_content:

Adaptation of parameters and operators represents one of the recent most important and promising areas of research in evolutionary computations; it is a form of designing self-configuring algorithms that acclimatize to suit the problem in hand. Here, our interests are on a recent breed of hybrid evolutionary algorithms typically known as adaptive memetic algorithms (MAs). One unique feature of adaptive MAs is the choice of local search methods or memes and recent studies have shown that this choice significantly affects the performances of problem searches. In this paper, we present a classification of memes adaptation in adaptive MAs on the basis of the mechanism used and the level of historical knowledge on the memes employed. Then the asymptotic convergence properties of the adaptive MAs considered are analyzed according to the classification. Subsequently, empirical studies on representatives of adaptive MAs for different type-level meme adaptations using continuous benchmark problems indicate that global-level adaptive MAs exhibit better search performances. Finally we conclude with some promising research directions in the area.
---
paper_title: A Combined Swarm Differential Evolution Algorithm for Optimization Problems

paper_content:

An algorithm that is a combination of the particle swarm and differential evolution algorithms is introduced. The results of testing this on a graduated set of trial problems is given. It is shown that the combined algorithm out performs both of the component algorithms under most conditions, in both absolute and computational load weighted terms.
---
paper_title: The self-adaptive Pareto differential evolution algorithm

paper_content:

The Pareto differential evolution (PDE) algorithm was introduced and showed competitive results. The behavior of PDE, as in many other evolutionary multiobjective optimization (EMO) methods, varies according to the crossover and mutation rates. In this paper, we present a new version of PDE with self-adaptive crossover and mutation. We call the new version self-adaptive Pareto differential evolution (SPDE). The emphasis of this paper is to analyze the dynamics and behavior of SPDE. The experiments also show that the algorithm is very competitive with other EMO algorithms.
---
paper_title: Hybrid differential evolution - Particle Swarm Optimization algorithm for solving global optimization problems

paper_content:

This paper presents a simple, hybrid two phase global optimization algorithm called DE-PSO for solving global optimization problems. DE-PSO consists of alternating phases of differential evolution (DE) and Particle Swarm Optimization (PSO). The algorithm is designed so as to preserve the strengths of both the algorithms. Empirical results show that the proposed DE-PSO is quite competent for solving the considered test functions as well as real life problems.
---
paper_title: Hybrid particle swarm with differential evolution for multimodal image registration

paper_content:

In this paper, we propose an algorithm of hybrid particle swarm with differential evolution (DE) operator, termed DEPSO, to solve the problem of multimodal image registration. This algorithm combines the robustness of entropy based measures and the search power of the DEPSO which provides the bell-shaped mutations with consensus on the population diversity, while keeps the particle swarm dynamics. The main idea is to find the best transformation that superimposes two multimodal images by maximizing the mutual information value through the DEPSO. We show that this algorithm, besides its simplicity, provides a robust and efficient way to rigidly register multimodal images in various situations.
---
paper_title: Inserting information sharing mechanism of PSO to improve the convergence of DE

paper_content:

in the present study we propose a new hybrid version of Differential Evolution (DE) and Particle Swarm Optimization (PSO) algorithms. In the proposed algorithm named as Hybrid Differential Evolution (HDE) a ‘switchover constant’ called α is defined. HDE starts as the basic DE algorithm which switches over to PSO when α is activated. The constant α on the other hand is activated at a point where the DE procedure usually slows down which is usually in the proximity of global optimal i.e. to say when the search domain is contracted around the global optimal. Experimenting with a test bed of 12 benchmark functions, we show the promising nature of the proposed HDE algorithm.
---
paper_title: Chaotic co-evolutionary algorithm based on differential evolution and particle swarm optimization

paper_content:

A chaotic co-evolutionary algorithm based on differential evolution and particle swarm optimization (CCDEPSO) is proposed. CCDEPSO is a two-population co-evolutionary algorithm, in which the individuals of one population are evolved according to differential evolution algorithm (DE) and the other individuals are evolved according to particle swarm optimization algorithm (PSO). In order to realize co-evolving of the two sub-populations, an information sharing scheme by sharing the fitness value and the corresponding position value of the two sub-populations is introduced. Furthermore, in order to improve the searching speed and avoid the results trapping in local optima prematurely, chaotic initialization and chaotic perturbation based on Tent map are introduced in CCDEPSO. The comparative testing experiments are performed by means of CCDEPSO, DE, CPSO and PSO algorithms on six benchmark functions. Experimental results demonstrate that the global optimization ability of CCDEPSO is better than other algorithms above mentioned.
---
paper_title: Design of T-S Fuzzy Model Based on PSODE Algorithm

paper_content:

This paper presents a novel fuzzy modeling strategy based on the hybrid of particle swarm optimization (PSO) and differential evolution (DE), and the proposed hybrid algorithm is referred to as PSODE. PSODE is based on a two-population scheme, in which the individuals in one population is enhanced by PSO and the individuals in the other population is evolved by DE. The individuals both in PSO and DE are co-evolved during the algorithm execution by employing an information sharing mechanisms. To further improve the proposed PSODE algorithm a nonlinear inertia weight approach and a mutation mechanism are presented respectively. In the simulation part, the PSODE is used to automatic design of fuzzy identifier for a nonlinear dynamic system. The performance of the suggested method is compared to PSO, DE and some other methods in the fuzzy identifier design to demonstrate its superiority.
---
paper_title: Super-fit control adaptation in memetic differential evolution frameworks

paper_content:

This paper proposes the super-fit memetic differential evolution (SFMDE). This algorithm employs a differential evolution (DE) framework hybridized with three meta-heuristics, each having different roles and features. Particle Swarm Optimization assists the DE in the beginning of the optimization process by helping to generate a super-fit individual. The two other meta-heuristics are local searchers adaptively coordinated by means of an index measuring quality of the super-fit individual with respect to the rest of the population. The choice of the local searcher and its application is then executed by means of a probabilistic scheme which makes use of the generalized beta distribution. These two local searchers are the Nelder mead algorithm and the Rosenbrock Algorithm. The SFMDE has been tested on two engineering problems; the first application is the optimal control drive design for a direct current (DC) motor, the second is the design of a digital filter for image processing purposes. Numerical results show that the SFMDE is a flexible and promising approach which has a high performance standard in terms of both final solutions detected and convergence speed.
---
paper_title: DEPSO: hybrid particle swarm with differential evolution operator

paper_content:

A hybrid particle swarm with differential evolution operator, termed DEPSO, which provide the bell-shaped mutations with consensus on the population diversity along with the evolution, while keeping the self-organized particle swarm dynamics, is proposed. Then it is applied to a set of benchmark functions, and the experimental results illustrate its efficiency.
---
paper_title: Differential Evolution Based Particle Swarm Optimization

paper_content:

A new, almost parameter-free optimization algorithm is developed in this paper as a hybrid of the barebones particle swarm optimizer (PSO) and differential evolution (DE). The DE is used to mutate, for each particle, the attractor associated with that particle, defined as a weighted average of its personal and neighborhood best positions. Results of this algorithm are compared to that of the barebones PSO, Von Neumann PSO, a DE PSO, and DE/rand/1/bin. These results show that the new algorithm provides excellent results with the added advantage that no parameter tuning is needed
---
paper_title: A hybrid optimization method of Particle Swarm Optimization and Cultural Algorithm

paper_content:

A novel hybrid optimization algorithm, PSOCCA, based on the fusion of the Particle Swarm Optimization (PSO) and Cultural Algorithm (CA), is proposed in this paper to enhance the convergence characteristics of the original PSO. In the PSOCCA, the particle swarm is considered to be a population component of the CA, and three knowledge sources are stored in the belief space to update the population space and establish the relationship between the two CA spaces. A unique mutation operator inspired by the Differential Evolution (DE) is used to mutate those particles with unsatisfactory performances in the population space. A new kind of knowledge, average value knowledge, that is stored in the belief space, is also introduced to regulate the variation ratio in the swarm population. The optimization performance of our PSOCCA is investigated using ten high-dimension and multi-peak functions. Simulation results demonstrate that it can be superior to the regular PSO.
---
paper_title: RNN based MIMO channel prediction

paper_content:

A new hybrid PSO-EA-DEPSO algorithm based on particle swarm optimization (PSO), evolutionary algorithm (EA), and differential evolution (DE) is presented for training a recurrent neural network (RNN) for multiple-input multiple-output (MIMO) channel prediction. This algorithm is shown to outperform RNN predictors trained off-line by PSO, EA, and DEPSO as well as a linear predictor trained by the Levinson-Durbin algorithm. To explore the effects of channel prediction error at the receiver, new expressions for the received SNR, array gain, and average probability of error are derived and analyzed. These expressions differ from previous results which assume the prediction error is Gaussian and/or independent of the true CSI. The array gain decays with increasing signal-to-noise ratio and is slightly larger for spatially correlated systems. As the prediction error increases in the non-saturation region, the coding gain decreases and the diversity gain remains unaffected.
---
paper_title: Cultural algorithm-based quantum-behaved particle swarm optimization

paper_content:

A hybrid quantum-behaved particle swarm optimization (QPSO) based on cultural algorithm (CA), which we call cultural QPSO, is proposed. Although QPSO is a promising algorithm for many optimization problems, it is apt to lose the diversity of the swarm in the later period of the search and prematurely converges to the local optimum. Inspired by the structure of human society, this paper uses the CA model to diversify the QPSO population and improve the QPSO's performance. In this model, the swarm is divided into two sub-swarms: the common particle and the elite particle sub-swarm. If a particle comes from a common sub-swarm, it will evolve according to the QPSO method, and during the evolvement, it will be affected not only by the other common particles but also by the elites. For the elites, the differential evolution (DE) method is adopted for evolvement. After each generation, the elites will be re-elected from the whole swarm according to fitness values. The simulation results on benchmark functions demonstrate that cultural QPSO outperforms the original QPSO for many problems.
---
paper_title: Clustering with differential evolution particle swarm optimization

paper_content:

The applications of recently developed meta-heuristics in cluster analysis, such as particle swarm optimization (PSO) and differential evolution (DE), have increasingly attracted attention and popularity in a wide variety of communities owing to their effectiveness in solving complicated combinatorial optimization problems. Here, we propose to use a hybrid of PSO and DE, known as differential evolution particle swarm optimization (DEPSO), in order to further improve search capability and achieve higher flexibility in exploring the natural while hidden data structures of data of interest. Empirical results show that the DEPSO-based clustering algorithm achieves better performance in terms of the number of epochs required to reach a pre-specified cutoff value of the fitness function than either of the other approaches used. Further experimental studies on both synthetic and real data sets demonstrate the effectiveness of the proposed method in finding meaningful clustering solutions.
---
paper_title: An evolutionary SPDE breeding-based hybrid particle swarm optimizer: application in coordination of robot ants for camera coverage area optimization

paper_content:

In this paper we propose a new Hybrid Particle Swarm Optimizer model based on particle swarm, with breeding concepts from novel evolutionary algorithms. The hybrid PSO combines traditional velocity and position update rules of RANDIW-PSO and ideas from Self Adaptive Pareto Differential Evolution Algorithm (SPDE). The hybrid model is tested and compared with some high quality PSO models like the RANDIW-PSO and TVIW-PSO. The results indicate two good prospects of our proposed hybrid PSO model: potential to achieve faster convergence as well as potential to find a better solution. The hybrid PSO model, with the abovementioned features, is then efficiently utilized to coordinate robot ants in order to help them to probe as much camera coverage area of some planetary surface or working field as possible with minimum common area coverage.
---
paper_title: Population-Based Algorithm Portfolios for Numerical Optimization

paper_content:

In this paper, we consider the scenario that a population-based algorithm is applied to a numerical optimization problem and a solution needs to be presented within a given time budget. Although a wide range of population-based algorithms, such as evolutionary algorithms, particle swarm optimizers, and differential evolution, have been developed and studied under this scenario, the performance of an algorithm may vary significantly from problem to problem. This implies that there is an inherent risk associated with the selection of algorithms. We propose that, instead of choosing an existing algorithm and investing the entire time budget in it, it would be less risky to distribute the time among multiple different algorithms. A new approach named population-based algorithm portfolio (PAP), which takes multiple algorithms as its constituent algorithms, is proposed based upon this idea. PAP runs each constituent algorithm with a part of the given time budget and encourages interaction among the constituent algorithms with a migration scheme. As a general framework rather than a specific algorithm, PAP is easy to implement and can accommodate any existing population-based search algorithms. In addition, a metric is also proposed to compare the risks of any two algorithms on a problem set. We have comprehensively evaluated PAP via investigating 11 instantiations of it on 27 benchmark functions. Empirical results have shown that PAP outperforms its constituent algorithms in terms of solution quality, risk, and probability of finding the global optimum. Further analyses have revealed that the advantages of PAP are mostly credited to the synergy between constituent algorithms, which should complement each other either over a set of problems, or during different stages of an optimization process.
---
paper_title: Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization

paper_content:

Differential evolution (DE) is an efficient and powerful population-based stochastic search technique for solving optimization problems over continuous space, which has been widely applied in many scientific and engineering fields. However, the success of DE in solving a specific problem crucially depends on appropriately choosing trial vector generation strategies and their associated control parameter values. Employing a trial-and-error scheme to search for the most suitable strategy and its associated parameter settings requires high computational costs. Moreover, at different stages of evolution, different strategies coupled with different parameter settings may be required in order to achieve the best performance. In this paper, we propose a self-adaptive DE (SaDE) algorithm, in which both trial vector generation strategies and their associated control parameter values are gradually self-adapted by learning from their previous experiences in generating promising solutions. Consequently, a more suitable generation strategy along with its parameter settings can be determined adaptively to match different phases of the search process/evolution. The performance of the SaDE algorithm is extensively evaluated (using codes available from P. N. Suganthan) on a suite of 26 bound-constrained numerical optimization problems and compares favorably with the conventional DE and several state-of-the-art parameter adaptive DE variants.
---
paper_title: Multistage Inventory Hybrid Intelligent Optimization Under Grey Fuzzy Uncertainty

paper_content:

The customer demand and replenishment lead-time can be considered as grey fuzzy variables combining grey and fuzzy twofold uncertain factors. The multistage inventory model under periodical review policy was presented based on the chance measure of grey fuzzy variable. The inventory would be replenished to certain level when the inventory level drops to the re-order point. The optimal re-order point and the inventory replenishment level of every stage can be obtained by minimizing the multistage inventory cost. The grey fuzzy simulation technology can generate input-output data for the uncertain functions. The neural network trained from the input-output data can approximate the uncertain functions. The particle swarm optimization (PSO) algorithm was improved with the differential evolution algorithm. The improved PSO algorithm was combined with neural network to optimize the inventory model. A numerical example was given to illustrate the feasibility of the model and algorithm.
---
paper_title: MIMO beam-forming with neural network channel prediction trained by a novel PSO-EA-DEPSO algorithm

paper_content:

A new hybrid algorithm based on particle swarm optimization (PSO), evolutionary algorithm (EA), and differential evolution (DE) is presented for training a recurrent neural network (RNN) for multiple-input multiple-output (MIMO) channel prediction. The hybrid algorithm is shown to be superior in performance to PSO and differential evolution PSO (DEPSO) for different channel environments. The received signal-to-noise ratio is derived for un-coded and beam-forming MIMO systems to see how the RNN error affects the performance. This error is shown to deteriorate the accuracy of the weak singular modes, making beam-forming more desirable. Bit error rate simulations are performed to validate these results.
---
paper_title: Research on Hybrid PSODE with Triple Populations Based on Multiple Differential Evolutionary Models

paper_content:

Regarding the complementarity of particle swarm optimization (PSO) and differential evolution (DE) and the complementarity of different differential evolutionary modes of DE, hybrid PSODE with three populations based on multiple differential evolutionary models is proposed. The algorithm effectively combines PSO and DE of two different evolutionary modes, improving local and global optimization ability dramatically. Hybrid PSODE with three populations, hybrid PSODE with two populations and hybrid DE with two populations are compared by 4 typical test functions, proving that hybrid PSODE with three populations has comprehensive superiority in global convergence properties and local search ability. Besides, among hybrid PSODE with three populations based on different differential evolutionary models, the algorithm with combination of rand and best evolutionary modes performs best and has perfect robustness and stability.
---
paper_title: Hybridizing particle swarm optimization with differential evolution for constrained numerical and engineering optimization

paper_content:

We propose a novel hybrid algorithm named PSO-DE, which integrates particle swarm optimization (PSO) with differential evolution (DE) to solve constrained numerical and engineering optimization problems. Traditional PSO is easy to fall into stagnation when no particle discovers a position that is better than its previous best position for several generations. DE is incorporated into update the previous best positions of particles to force PSO jump out of stagnation, because of its strong searching ability. The hybrid algorithm speeds up the convergence and improves the algorithm's performance. We test the presented method on 11 well-known benchmark test functions and five engineering optimization functions. Comparisons show that PSO-DE outperforms or performs similarly to seven state-of-the-art approaches in terms of the quality of the resulting solutions.
---
paper_title: Differential evolution based particle swarm optimizer for neural network learning

paper_content:

An improved particle swarm optimizer based on differential evolution theory is proposed. This algorithm introduces differential mutation operator into the basic particle swarm optimizer in order to solve the premature convergence problem. And this new algorithm was used to training weights and thresholds of feedforward neural network, simulation results show that this approach is effective and has an excellent convergence performance.
---
paper_title: Meta-Lamarckian learning in memetic algorithms

paper_content:

Over the last decade, memetic algorithms (MAs) have relied on the use of a variety of different methods as the local improvement procedure. Some recent studies on the choice of local search method employed have shown that this choice significantly affects the efficiency of problem searches. Given the restricted theoretical knowledge available in this area and the limited progress made on mitigating the effects of incorrect local search method choice, we present strategies for MA control that decide, at runtime, which local method is chosen to locally improve the next chromosome. The use of multiple local methods during a MA search in the spirit of Lamarckian learning is here termed Meta-Lamarckian learning. Two adaptive strategies for Meta-Lamarckian learning are proposed in this paper. Experimental studies with Meta-Lamarckian learning strategies on continuous parametric benchmark problems are also presented. Further, the best strategy proposed is applied to a real-world aerodynamic wing design problem and encouraging results are obtained. It is shown that the proposed approaches aid designers working on complex engineering problems by reducing the probability of employing inappropriate local search methods in a MA, while at the same time, yielding robust and improved design search performance.
---
paper_title: Environmental/economic power dispatch using a hybrid multi-objective optimization algorithm

paper_content:

Abstract To solve the highly constrained environmental/economic dispatch problem involving conflicting objectives, this paper presents a hybrid multi-objective optimization algorithm based on particle swarm optimization (PSO) and differential evolution (DE). In this algorithm, a PSO with time variant acceleration coefficients is designed to explore the entire search space, while a local version of DE is proposed to exploit the sub-space with sparse solutions. A crowing distance-based approach is introduced to assign the particles’ leaders and to update the external archive. Moreover, a new technique for equality constraints is proposed to hurdle the unfeasible solutions directly. Finally, several optimization trials of the proposed algorithm are carried out on the IEEE 30-bus test system. Results demonstrate superiority of the proposed approach and confirm its potential to solve the multi-objective EED problem.
---
paper_title: Differential evolution particle swarm optimization for digital filter design

paper_content:

In this paper, swarm and evolutionary algorithms have been applied for the design of digital filters. Particle swarm optimization (PSO) and differential evolution particle swarm optimization (DEPSO) have been used here for the design of linear phase finite impulse response (FIR) filters. Two different fitness functions have been studied and experimented, each having its own significance. The first study considers a fitness function based on the passband and stopband ripple, while the second study considers a fitness function based on the mean squared error between the actual and the ideal filter response. DEPSO seems to be promising tool for FIR filter design especially in a dynamic environment where filter coefficients have to be adapted and fast convergence is of importance.
---
paper_title: A hybrid particle swarm optimization approach with prior crossover differential evolution

paper_content:

Particle swarm optimization (PSO) is population-based heuristic searching algorithm. PSO has excellent ability of global optimization. However, there are some shortcomings of prematurity, low convergence accuracy and speed, similarly to other evolutionary algorithms (EA). To improve its performance, a hybrid particle swarm optimization is proposed in the paper. Firstly, the average position and velocity of particles are incorporated into basic PSO for concerning with the effect of the evolution of the whole swarm. Then a differential evolution (DE) computation, which introduces an extra population for prior crossover, is hybridized with the improved PSO to form a novel optimization algorithm, PSOPDE. The role of prior crossover is to appropriately diversify the population and increase the probability of reaching better solutions. DE component takes into account the stochastic differential variation, and enhances the exploitation in the neighborhoods of current solutions. PSOPDE is implemented on five typical benchmark functions, and compared with six other algorithms. The results indicate that PSOPDE behaves better, and greatly improve the searching efficiency and quality.
---
paper_title: A Particle Swarm Optimization Algorithm with Differential Evolution

paper_content:

Differential evolution (DE) is a simple evolutionary algorithm that has shown superior performance in the global continuous optimization. It mainly utilizes the differential information to guide its further search. But the differential information also results in instability of performance. Particle swarm optimization (PSO) has been developing rapidly and has been applied widely since it is introduced, as it can converge quickly. But PSO easily got stuck in local optima because it easily loses the diversity of swarm. This paper proposes a combination of DE and PSO (termed DEPSO) that makes up their disadvantages. DEPSO combines the differential information obtained by DE with the memory information extracted by PSO to create the promising solutions. Finally, DEPSO is tested to solve several benchmark optimization problems. The experimental results show the effectiveness of DEPSO algorithm for the multimodal function, and also verify that DEPSO can perform better than other algorithms (DE, CPSO) in solving the benchmark problems.
---
paper_title: EVOLVING DIGITAL CIRCUITS USING HYBRID PARTICLE SWARM OPTIMIZATION AND DIFFERENTIAL EVOLUTION

paper_content:

This paper presents the evolution of combinational logic circuits by a new hybrid algorithm known as the Differential Evolution Particle Swarm Optimization (DEPSO), formulated from the concepts of a modified particle swarm and differential evolution. The particle swarm in the hybrid algorithm is represented by a discrete 3-integer approach. A hybrid multi-objective fitness function is coined to achieve two goals for the evolution of circuits. The first goal is to evolve combinational logic circuits with 100% functionality, called the feasible circuits. The second goal is to minimize the number of logic gates needed to realize the feasible circuits. In addition, the paper presents modifications to enhance performance and robustness of particle swarm and evolutionary techniques for discrete optimization problems. Comparison of the performance of the hybrid algorithm to the conventional Karnaugh map and evolvable hardware techniques such as genetic algorithm, modified particle swarm, and differential evolution are presented on a number of case studies. Results show that feasible circuits are always achieved by the DEPSO algorithm unlike with other algorithms and the percentage of best solutions (minimal logic gates) is higher.
---
paper_title: Designing neural networks using hybrid particle swarm optimization

paper_content:

Evolving artificial neural network is an important issue in both evolutionary computation (EC) and neural networks (NN) fields. In this paper, a hybrid particle swarm optimization (PSO) is proposed by incorporating differential evolution (DE) and chaos into the classic PSO. By combining DE operation with PSO, the exploration and exploitation abilities can be well balanced, and the diversity of swarms can be reasonably maintained. Moreover, by hybridizing chaotic local search (CLS), DE operator and PSO operator, searching behavior can be enriched and the ability to avoid being trapped in local optima can be well enhanced. Then, the proposed hybrid PSO (named CPSODE) is applied to design multi-layer feed-forward neural network. Simulation results and comparisons demonstrate the effectiveness and efficiency of the proposed hybrid PSO.
---
paper_title: Balancing the exploration and exploitation capabilities of the Differential Evolution Algorithm

paper_content:

The hybridization and composition of different Evolutionary Algorithms to improve the quality of the solutions and to accelerate execution is a common research practice. In this paper we propose a hybrid approach that combines differential evolution mutation operators in an attempt to balance their exploration and exploitation capabilities. Additionally, a self-balancing hybrid mutation operator is presented, which favors the exploration of the search space during the first phase of the optimization, while later opts for the exploitation to aid convergence to the optimum. Extensive experimental results indicate that the proposed approaches effectively enhance DEpsilas ability to accurately locate solutions in the search space.
---
paper_title: Evolving cognitive and social experience in Particle Swarm Optimization through Differential Evolution

paper_content:

In recent years, the Particle Swarm Optimization has rapidly gained increasing popularity and many variants and hybrid approaches have been proposed to improve it. Motivated by the behavior and the proximity characteristics of the social and cognitive experience of each particle in the swarm, we develop a hybrid approach that combines the Particle Swarm Optimization and the Differential Evolution algorithm. Particle Swarm Optimization has the tendency to distribute the best personal positions of the swarm near to the vicinity of problem's optima. In an attempt to efficiently guide the evolution and enhance the convergence, we evolve the personal experience of the swarm with the Differential Evolution algorithm. Extensive experimental results on twelve high dimensional multimodal benchmark functions indicate that the hybrid variants are very promising and improve the original algorithm.
---
paper_title: Modeling of gene regulatory networks with hybrid differential evolution and particle swarm optimization.

paper_content:

In the last decade, recurrent neural networks (RNNs) have attracted more efforts in inferring genetic regulatory networks (GRNs), using time series gene expression data from microarray experiments. This is critically important for revealing fundamental cellular processes, investigating gene functions, and understanding their relations. However, RNNs are well known for training difficulty. Traditional gradient descent-based methods are easily stuck in local minima and the computation of the derivatives is also not always possible. Here, the performance of three evolutionary-swarm computation technology-based methods, known as differential evolution (DE), particle swarm optimization (PSO), and the hybrid of DE and PSO (DEPSO), in training RNNs is investigated. Furthermore, the gene networks are reconstructed via the identification of the gene interactions, which are explained through corresponding connection weight matrices. The experimental results on two data sets studied in this paper demonstrate that the DEPSO algorithm performs better in RNN training. Also, the RNN-based model can provide meaningful insight in capturing the nonlinear dynamics of genetic networks and revealing genetic regulatory interactions.
---
paper_title: CO-EVOLUTIONARY HYBRID DIFFERENTIAL EVOLUTION FOR MIXED-INTEGER OPTIMIZATION PROBLEMS

paper_content:

Abstract Evolutionary algorithms are promising candidates for obtaining the global optimum. Hybrid differential evolution is one or the evolutionary algorithms, which has been successfully applied to many real-world nonlinear programming problems. This paper proposes a co-evolutionary hybrid differential evolution to solve mixed-integer nonlinear programming (MINLP) problems. The key ingredients of the algorithm consist of an integer-valued variable evolution and a real-valued variable co-evolution, so that the algorithm can be used to solve MINLP problems or pure integer programming problems. Furthermore, the algorithm combines a local search heuristic (called acceleration) and a widespread search heuristic (called migration) to promote the search for a global optimum. Some numerical examples are tested to illustrate the performance of the proposed algorithm. Numerical examples show that the proposed algorithm converges to better solutions than the conventional MINLP optimization methods
---
paper_title: A Novel Hybrid Particle Swarm Optimization for Multi-Objective Problems

paper_content:

To solve the multi-objective problems, a novel hybrid particle swarm optimization algorithm is proposed(called HPSODE). The new algorithm includes three major improvement: (I)Population initialization is constructed by statistical method Uniform Design , (II)Regeneration method has two phases: the first phase is particles updated by adaptive PSO model with constriction factor *** , the second phase is Differential Evolution operator with archive, (III)A new accept rule called Distance/volume fitness is designed to update archive. Experiment on ZDTx and DTLZx problems by jMetal 2.1, the results show that the new hybrid algorithm significant outperforms OMOPSO, SMPSO in terms of additive Epsilon, HyperVolume, Genetic Distance, Inverted Genetic Distance.
---
paper_title: A tutorial for competent memetic algorithms: model, taxonomy, and design issues

paper_content:

The combination of evolutionary algorithms with local search was named "memetic algorithms" (MAs) (Moscato, 1989). These methods are inspired by models of natural systems that combine the evolutionary adaptation of a population with individual learning within the lifetimes of its members. Additionally, MAs are inspired by Richard Dawkin's concept of a meme, which represents a unit of cultural evolution that can exhibit local refinement (Dawkins, 1976). In the case of MA's, "memes" refer to the strategies (e.g., local refinement, perturbation, or constructive methods, etc.) that are employed to improve individuals. In this paper, we review some works on the application of MAs to well-known combinatorial optimization problems, and place them in a framework defined by a general syntactic model. This model provides us with a classification scheme based on a computable index D, which facilitates algorithmic comparisons and suggests areas for future research. Also, by having an abstract model for this class of metaheuristics, it is possible to explore their design space and better understand their behavior from a theoretical standpoint. We illustrate the theoretical and practical relevance of this model and taxonomy for MAs in the context of a discussion of important design issues that must be addressed to produce effective and efficient MAs.
---
paper_title: Hybrid Differential Evolution Particle Swarm Optimization Algorithm for Reactive Power Optimization

paper_content:

Reactive power optimization is a mixed integer nonlinear programming problem where metaheuristics techniques have proven suitable for providing optimal solutions. In this paper, swarm and evolutionary algorithm have been applied for reactive power optimization. The objective of this nonlinear optimization is minimization of system losses and improvement of voltage profiles in a power system. A hybrid differential evolution particle swarm optimization algorithm is presented to obtain the global optimum. The proposed algorithm is implemented on the IEEE 14-bus system. To validate the effectiveness of the algorithm, the simulation results are compared with other optimization algorithms'. It is shown that the approach developed is feasible and efficient.
---
paper_title: Diploid hybrid Particle swarm optimization with differential evolution for Open Vehicle Routing Problem

paper_content:

To improve the performance of particle swarm optimization and differential evolution, the diploid differential evolution particle swarm optimization was presented according to the genetics rules. The algorithm was applied into the Open Vehicle Routing Problem. In the algorithm, dominant character and recessive character were included in every individual. Particle swarm optimization was implemented by dominant character, and differential evolution was implemented by recessive character. Dominant character was replaced by recessive character if its fitness was smaller than recessive character. A real number encoding method was presented and some heuristics were modified and improved the solution. The parameters of algorithm were analyzed in the experiment. And comparison with other algorithms in the literatures showed the proposed algorithm was effective.
---
paper_title: Robust Optimization in HTS Cable Based on DEPSO and Design for Six Sigma

paper_content:

The non-uniform AC current distribution among the multi-layer conductors in a high-temperature superconducting (HTS) cable reduces the current capacity and increases the AC loss. In this paper, Particle swarm optimization coupled with differential evolution operator (DEPSO) has been applied in structural optimization of HTS cables. While the existence of fluctuation in design variables or operation conditions has a great influence on the cable quality, in order to eliminate the effects of parameter perturbations in design and improve the design efficiency, a robust design method based on design for six sigma (DFSS) is applied in this paper. The optimization solutions show that the proposed optimization procedure can not only achieve a uniform current distribution, but also improve significantly the reliability and robustness of the HTS cable quality.
---
paper_title: Hybridizing PSO and DE for improved vector evaluated multi-objective optimization

paper_content:

This paper introduces a new vector evaluated multi-objective optimization algorithm. The vector evaluated differential evolution particle swarm optimization (VEDEPSO) algorithm is a hybridization of the classical vector evaluated particle swarm optimization (VEPSO) and vector evaluated differential evolution (VEDE) algorithms of Parsopoulos et. al. [9], [10]. Comparisons of VEDEPSO with respect to VEPSO and VEDE on a well known multi-objective benchmark problem set indicated that significant performance improvements can be attributed to the VEDEPSO algorithm.
---
paper_title: Cooperative evolutionary algorithm for space trajectory optimization

paper_content:

A hybrid evolutionary algorithm which synergistically exploits differential evolution, genetic algorithms and particle swarm optimization, has been developed and applied to spacecraft trajectory optimization. The cooperative procedure runs the three basic algorithms in parallel, while letting the best individuals migrate to the other populations at prescribed intervals. Rendezvous problems and round-trip Earth–Mars missions have been considered. The results show that the hybrid algorithm has better performance compared to the basic algorithms that are employed. In particular, for the rendezvous problem, a 100% efficiency can be obtained both by differential evolution and the genetic algorithm only when particular strategies and parameter settings are adopted. On the other hand, the hybrid algorithm always attains the global optimum, even though nonoptimal strategies and parameter settings are adopted. Also the number of function evaluations, which must be performed to attain the optimum, is reduced when the hybrid algorithm is used. In the case of Earth–Mars missions, the hybrid algorithm is successfully employed to determine mission opportunities in a large search space.
---
paper_title: The Back Analysis of Mechanics Parameters Based on DEPSO Algorithm and Parallel FEM

paper_content:

The back-analysis of mechanics parameters needs iterative forward calculating, resulting in low efficiency; meanwhile the Particle Swarm Optimization (PSO) algorithm and other optimization algorithms are exposed to local optimum possibilities. In this paper, DEPSO-ParallelFEM, a system integrated of an algorithm of hybrid particle swarm with Differential Evolution (DE) operator, termed DEPSO, and parallel Finite Element Method (FEM), is proposed to solve these problems. DEPSO guarantees the particle to escape from local minima by enhancing particle’s diversity through the combination of PSO operator and DE operator; and parallel FEM is applied to improve the computing speed and precision by adopting the techniques of Cluster of Workstation (COW), MPI, Domain Decomposition Method (DDM), and Object-Oriented Programming (OOP) and so on. A computational example proves that this system is of excellent parameter exploration capability and high speed; thus it is of great academic value and significant applicable value.
---
paper_title: An adaptive hybrid optimizer based on particle swarm and differential evolution for global optimization

paper_content:

This paper presents extensive experiments on a hybrid optimization algorithm (DEPSO) we recently developed by combining the advantages of two powerful population-based metaheuristics—differential evolution (DE) and particle swarm optimization (PSO). The hybrid optimizer achieves on-the-fly adaptation of evolution methods for individuals in a statistical learning way. Two primary parameters for the novel algorithm including its learning period and population size are empirically analyzed. The dynamics of the hybrid optimizer is revealed by tracking and analyzing the relative success ratio of PSO versus DE in the optimization of several typical problems. The comparison between the proposed DEPSO and its competitors involved in our previous research is enriched by using multiple rotated functions. Benchmark tests involving scalability test validate that the DEPSO is competent for the global optimization of numerical functions due to its high optimization quality and wide applicability.
---
paper_title: Hybrid PSO-DE for solving the economic dispatch problem with generator constraints

paper_content:

This paper proposes an improved approach based on conventional particle swarm optimization (PSO) for solving an economic dispatch(ED) problem with considering the generator constraints. The mutation operators of the differential evolution (DE) are used for improving diversity exploration of PSO, which called hybrid particle swarm optimization - differential evolution (PSO-DE). The mutation operators are activated if velocity values of PSO nearly to zero or violated from the boundaries. Four scenarios of mutation operators are implemented for PSO-DE. The simulation results of all scenarios of the PSO-DE outperform over the PSO and other existing approaches which appeared in literatures.
---
paper_title: A Hybrid Differential Evolution Method for Practical Engineering Problems

paper_content:

This paper introduced an almost control parameters free modified differential evolution for global optimization problems. The modifications are derived from the mechanisms of particle swarm optimization viz., topologies, inertial weight, neighborhood best and individual best, with which each individual performed the mutation operator based on its current position, the neighborhood best and its individual best along with the inertial weight. And the crossover operator in traditional differential evolution is removed, while the selection operator was employed. The approach was employed for a tension/compression string design problem and an economic dispatch problem in power system. By comparisons with the other evolutionary algorithms, the proposed approach has shown its feasibility and effectiveness.
---
paper_title: A Hybrid of Differential Evolution and Particle Swarm Optimization for Global Optimization

paper_content:

A hybrid differential evolution (HDE) approach derived from both the differential evolution (DE) and the particle swarm optimization (PSO) is proposed. In HDE, individuals in a new generation are created, not only by crossover and mutation operation as in DE, but also by PSO operations. The concepts of inertia weight and neighbor topology are adopted in HDE. The former is employed to provide consistency and diversity by adding a weighted velocity to the trial vector. In the latter, instead of the whole population, each individual can only communicate with its neighbors, and each individual creates its trial vector based on the best individual found by its neighbors so far. The proposed approach is employed for four well-known benchmarks, and the simulation results have shown its feasibility and effectiveness.
---
paper_title: Differential Evolution Using a Neighborhood-Based Mutation Operator

paper_content:

Differential evolution (DE) is well known as a simple and efficient scheme for global optimization over continuous spaces. It has reportedly outperformed a few evolutionary algorithms (EAs) and other search heuristics like the particle swarm optimization (PSO) when tested over both benchmark and real-world problems. DE, however, is not completely free from the problems of slow and/or premature convergence. This paper describes a family of improved variants of the DE/target-to-best/1/bin scheme, which utilizes the concept of the neighborhood of each population member. The idea of small neighborhoods, defined over the index-graph of parameter vectors, draws inspiration from the community of the PSO algorithms. The proposed schemes balance the exploration and exploitation abilities of DE without imposing serious additional burdens in terms of function evaluations. They are shown to be statistically significantly better than or at least comparable to several existing DE variants as well as a few other significant evolutionary computing techniques over a test suite of 24 benchmark functions. The paper also investigates the applications of the new DE variants to two real-life problems concerning parameter estimation for frequency modulated sound waves and spread spectrum radar poly-phase code design.
---
paper_title: Improving particle swarm optimization with differentially perturbed velocity

paper_content:

This paper introduces a novel scheme of improving the performance of particle swarm optimization (PSO) by a vector differential operator borrowed from differential evolution (DE). Performance comparisons of the proposed method are provided against (a) the original DE, (b) the canonical PSO, and (c) three recent, high-performance PSO-variants. The new algorithm is shown to be statistically significantly better on a seven-function test suite for the following performance measures: solution quality, time to find the solution, frequency of finding the solution, and scalability.
---
paper_title: Spatial clustering with obstacles constraints using PSO-DV and K-Medoids

paper_content:

Spatial clustering with obstacles constraints (SCOC) has been a new topic in spatial data mining (SDM).In this paper, we propose an advanced Particle swarm optimization (PSO) and differential evolution (DE) method for SCOC. In the process of doing so,we first developed a novel spatial obstructed distance using PSO-DV(particle swarm optimization with differentially perturbed Velocity) based on grid model to obtain obstructed distance, which is named PDGSOD, and then we presented a new PDKSCOC based on PSO-DV and K-Medoids to cluster spatial data with obstacles constraints. The experimental results show that PDGSOD is effective, and PDKSCOC can not only give attention to higher local constringency speed and stronger global optimum search, but also get down to the obstacles constraints and practicalities of spatial clustering; and it performs better than Improved KMedoids SCOC (IKSCOC) in terms of quantization error and has higher constringency speed than Genetic K-Medoids SCOC (GKSCOC).
---
paper_title: A novel differential evolution scheme combined with particle swarm intelligence

paper_content:

Differential evolution (DE) and particle swarm optimization (PSO) are the evolutionary computation paradigms, and both have shown superior performance on complex nonlinear function optimization problems. This paper detects the underlying relationship between them and then qualitatively proves that the two heuristic approaches from different theoretical background are consistent in form. Within the general perspective, the PSO can be regarded as a kind of DE. Inspired by this, a novel variant of DE mixed with particle swarm intelligence (DE-SI) is presented. Comparison experiments involving ten test functions well studied in the evolutionary optimization literature are used to highlight some performance differences between the DE-SI, two versions of DE and two PSO variants. The results from our study show that DE-SI keeps the most rapid convergence rate of all techniques and obtains the global optima for most benchmark problems.
---
paper_title: A hybrid PSO-DV based intelligent method for fault diagnosis of gear-box

paper_content:

The gear box fault occur can lead to the fatal breakdown of mechanical system. Back propagation neural network (BPNN) have been proved to be of widespread utility for identifying and classifying gear box faults to prevent serious damage in a mechanical system. Some researchers have used particle swarm optimization (PSO) to train BPNN. However, because the PSO algorithm has several parameters to be adjusted by empirical approach, if these parameters are not appropriately set, the search will become very slow near the global optimum and even trap into local minima. In this paper, a novel hybrid intelligent method for classifying gear box faults based on vibration signal using the particle swarm optimization (PSO) algorithm, differential evolution (DE) algorithm and BPNN named PSO-DV based BP is presented. The proposed PSO-DV includes both faster convergence of PSO and capability escape from local optima of DE. Experiments were performed on a gear-box fault simulator. The fault samples are obtained by simulating corresponding fault on experiment gear-box. In presented work, a classical PSO based BP neural network and PSO-DV based BP neural network are used for gear box fault classification, their relative effectiveness in fault diagnosis is compared. The experimental results verified that proposed hybrid PSO-DV intelligent method can escape from local minima, so has better convergence than BP neural network and classical PSO based BP neural network. Meanwhile, it achieves also very high accuracy rate of recognition and thus provides decision support in fault classification.
---
paper_title: Optimization of Control Parameters for Genetic Algorithms

paper_content:

The task of optimizing a complex system presents at least two levels of problems for the system designer. First, a class of optimization algorithms must be chosen that is suitable for application to the system. Second, various parameters of the optimization algorithm need to be tuned for efficiency. A class of adaptive search procedures called genetic algorithms (GA) has been used to optimize a wide variety of complex systems. GA's are applied to the second level task of identifying efficient GA's for a set of numerical optimization problems. The results are validated on an image registration problem. GA's are shown to be effective for both levels of the systems optimization problem.
---
paper_title: A novel hybrid differential evolution and particle swarm optimization algorithm for unconstrained optimization

paper_content:

An algorithm called DE-PSO is proposed which incorporates concepts from DE and PSO, updating particles not only by DE operators but also by mechanisms of PSO. The proposed algorithm is tested on several benchmark functions. Numerical comparisons with different hybrid meta-heuristics demonstrate its effectiveness and efficiency.
---
paper_title: Differential Evolution Based Particle Swarm Optimization

paper_content:

A new, almost parameter-free optimization algorithm is developed in this paper as a hybrid of the barebones particle swarm optimizer (PSO) and differential evolution (DE). The DE is used to mutate, for each particle, the attractor associated with that particle, defined as a weighted average of its personal and neighborhood best positions. Results of this algorithm are compared to that of the barebones PSO, Von Neumann PSO, a DE PSO, and DE/rand/1/bin. These results show that the new algorithm provides excellent results with the added advantage that no parameter tuning is needed
---
paper_title: Choosing Leaders for Multi-objective PSO Algorithms Using Differential Evolution

paper_content:

The fast convergence of particle swarm algorithms can become a downside in multi-objective optimization problems when there are many local optimal fronts. In such a situation a multi-objective particle swarm algorithm may get stuck to a local Pareto optimal front. In this paper we propose a new approach in selecting leaders for the particles to follow, which in-turn will guide the algorithm towards the Pareto optimal front. The proposed algorithm uses a Differential Evolution operator to create the leaders. These leaders can successfully guide the other particles towards the Pareto optimal front for various types of test problems. This simple yet robust algorithm is effective compared with existing multi-objective particle swarm algorithms.
---
paper_title: Differential Evolution Using a Neighborhood-Based Mutation Operator

paper_content:

Differential evolution (DE) is well known as a simple and efficient scheme for global optimization over continuous spaces. It has reportedly outperformed a few evolutionary algorithms (EAs) and other search heuristics like the particle swarm optimization (PSO) when tested over both benchmark and real-world problems. DE, however, is not completely free from the problems of slow and/or premature convergence. This paper describes a family of improved variants of the DE/target-to-best/1/bin scheme, which utilizes the concept of the neighborhood of each population member. The idea of small neighborhoods, defined over the index-graph of parameter vectors, draws inspiration from the community of the PSO algorithms. The proposed schemes balance the exploration and exploitation abilities of DE without imposing serious additional burdens in terms of function evaluations. They are shown to be statistically significantly better than or at least comparable to several existing DE variants as well as a few other significant evolutionary computing techniques over a test suite of 24 benchmark functions. The paper also investigates the applications of the new DE variants to two real-life problems concerning parameter estimation for frequency modulated sound waves and spread spectrum radar poly-phase code design.
---
paper_title: Cultured Differential Particle Swarm Optimization for Numerical Optimization Problems

paper_content:

A cultural algorithm based on differential particle swarm optimization (DPSO) is proposed in this paper. The main novel feature of this approach is the use of DPSO as a population space. DPSO is a hybrid-optimized algorithm based principally on the introduction of the differential variation mechanism into the PSO algorithm, which will maintain swarm diversity. The knowledge sources contained in the belief space of the cultural algorithm are specifically designed according to the DPSO evolution population. Different knowledge sources are used to influence the variation operator of DPSO, in order to enhance search ability and reduce the computational cost. The simulation results of typical function optimization problems show that the proposed algorithm has good optimization quality.
---
paper_title: Bare bones particle swarms

paper_content:

The particle swarm algorithm has just enough moving parts to make it hard to understand. The formula is very simple, it is even easy to describe the working of the algorithm verbally, yet it is very difficult to grasp in one's mind how the particles oscillate around centers that are constantly changing; how they influence one another; how the various parameters affect the trajectory of the particle; how the topology of the swarm affects its performance; and so on. This paper strips away some traditional features of the particle swarm in the search for the properties that make it work. The particle swarm algorithm is modified by eliminating the velocity formula. Variations are compared. In the process some of the mysteries of the algorithm are revealed, we discover its similarity to other stochastic population-based problem solving methods, and new avenues of investigation are suggested or implied.
---
paper_title: Application of particle swarm optimization technique and its variants to generation expansion planning problem

paper_content:

Abstract This paper presents the application of particle swarm optimization (PSO) technique and its variants to least-cost generation expansion planning (GEP) problem. The GEP problem is a highly constrained, combinatorial optimization problem that can be solved by complete enumeration. PSO is one of the swarm intelligence (SI) techniques, which use the group intelligence behavior along with individual intelligence to solve the combinatorial optimization problem. A novel ‘virtual mapping procedure’ (VMP) is introduced to enhance the effectiveness of the PSO approaches. Penalty function approach (PFA) is used to reduce the number of infeasible solutions in the subsequent iterations. In addition to simple PSO, many variants such as constriction factor approach (CFA), Lbest model, hybrid PSO (HPSO), stretched PSO (SPSO) and composite PSO (C-PSO) are also applied to test systems. The differential evolution (DE) technique is used for parameter setting of C-PSO. The PSO and its variants are applied to a synthetic test system of five types of candidate units with 6- and 14-year planning horizon. The results obtained are compared with dynamic programming (DP) in terms of speed and efficiency.
---
paper_title: Bare bones differential evolution

paper_content:

The barebones differential evolution (BBDE) is a new, almost parameter-free optimization algorithm that is a hybrid of the barebones particle swarm optimizer and differential evolution. Differential evolution is used to mutate, for each particle, the attractor associated with that particle, defined as a weighted average of its personal and neighborhood best positions. The performance of the proposed approach is investigated and compared with differential evolution, a Von Neumann particle swarm optimizer and a barebones particle swarm optimizer. The experiments conducted show that the BBDE provides excellent results with the added advantage of little, almost no parameter tuning. Moreover, the performance of the barebones differential evolution using the ring and Von Neumann neighborhood topologies is investigated. Finally, the application of the BBDE to the real-world problem of unsupervised image classification is investigated. Experimental results show that the proposed approach performs very well compared to other state-of-the-art clustering algorithms in all measured criteria.
---
paper_title: A Master-Slave Particle Swarm Optimization Algorithm for Solving Constrained Optimization Problems

paper_content:

Penalty function based PSO converts constrained optimization problems into non-constrained optimization problems, but slow convergence and premature convergence easily happen because of inappropriate penalty coefficients. Modified PSO by tracking best feasible particle can not facilitate particles exploring unknown feasible region from known infeasible region, so the global exploration ability is greatly limited. Therefore, finding better unknown feasible solution by flying through infeasible region is critical to the performance of PSO. This paper proposes Master-Slave Particle Swarm Optimization (MSPSO), a novel approach for solving constrained optimization problems, in which particles in master swarm fly toward better feasible particles, particles in slave swarm fly toward better infeasible particles, and particles in two swarms help each other flying by sharing information of better feasible and infeasible particles. The proposed algorithm was tested on 11 benchmark constrained optimization problems. The test results show that MSPSO can significantly improve the globe exploration ability and effectively avoid being trapped into local optimum. By comparison with other evolutionary algorithms, MSPSO performs better for constrained optimization problems.
---
paper_title: Classification of adaptive memetic algorithms: a comparative study

paper_content:

Adaptation of parameters and operators represents one of the recent most important and promising areas of research in evolutionary computations; it is a form of designing self-configuring algorithms that acclimatize to suit the problem in hand. Here, our interests are on a recent breed of hybrid evolutionary algorithms typically known as adaptive memetic algorithms (MAs). One unique feature of adaptive MAs is the choice of local search methods or memes and recent studies have shown that this choice significantly affects the performances of problem searches. In this paper, we present a classification of memes adaptation in adaptive MAs on the basis of the mechanism used and the level of historical knowledge on the memes employed. Then the asymptotic convergence properties of the adaptive MAs considered are analyzed according to the classification. Subsequently, empirical studies on representatives of adaptive MAs for different type-level meme adaptations using continuous benchmark problems indicate that global-level adaptive MAs exhibit better search performances. Finally we conclude with some promising research directions in the area.
---
paper_title: Inserting information sharing mechanism of PSO to improve the convergence of DE

paper_content:

in the present study we propose a new hybrid version of Differential Evolution (DE) and Particle Swarm Optimization (PSO) algorithms. In the proposed algorithm named as Hybrid Differential Evolution (HDE) a ‘switchover constant’ called α is defined. HDE starts as the basic DE algorithm which switches over to PSO when α is activated. The constant α on the other hand is activated at a point where the DE procedure usually slows down which is usually in the proximity of global optimal i.e. to say when the search domain is contracted around the global optimal. Experimenting with a test bed of 12 benchmark functions, we show the promising nature of the proposed HDE algorithm.
---
paper_title: Chaotic co-evolutionary algorithm based on differential evolution and particle swarm optimization

paper_content:

A chaotic co-evolutionary algorithm based on differential evolution and particle swarm optimization (CCDEPSO) is proposed. CCDEPSO is a two-population co-evolutionary algorithm, in which the individuals of one population are evolved according to differential evolution algorithm (DE) and the other individuals are evolved according to particle swarm optimization algorithm (PSO). In order to realize co-evolving of the two sub-populations, an information sharing scheme by sharing the fitness value and the corresponding position value of the two sub-populations is introduced. Furthermore, in order to improve the searching speed and avoid the results trapping in local optima prematurely, chaotic initialization and chaotic perturbation based on Tent map are introduced in CCDEPSO. The comparative testing experiments are performed by means of CCDEPSO, DE, CPSO and PSO algorithms on six benchmark functions. Experimental results demonstrate that the global optimization ability of CCDEPSO is better than other algorithms above mentioned.
---
paper_title: Coevolutionary Particle Swarm Optimization Using Gaussian Distribution for Solving Constrained Optimization Problems

paper_content:

In this correspondence, an approach based on coevolutionary particle swarm optimization to solve constrained optimization problems formulated as min-max problems is presented. In standard or canonical particle swarm optimization (PSO), a uniform probability distribution is used to generate random numbers for the accelerating coefficients of the local and global s. We propose a Gaussian probability distribution to generate the accelerating coefficients of PSO. Two populations of PSO using Gaussian distribution are used on the optimization algorithm that is tested on a suite of well-known benchmark constrained optimization problems. Results have been compared with the canonical PSO (constriction factor) and with a coevolutionary genetic algorithm. Simulation results show the suitability of the proposed algorithm in terms of effectiveness and robustness
---
paper_title: Scale factor inheritance mechanism in distributed differential evolution

paper_content:

This article proposes a distributed differential evolution which employs a novel self-adaptive scheme, namely scale factor inheritance. In the proposed algorithm, the population is distributed over several sub-populations allocated according to a ring topology. Each sub-population is characterized by its own scale factor value. With a probabilistic criterion, that individual displaying the best performance is migrated to the neighbor population and replaces a pseudo-randomly selected individual of the target sub-population. The target sub-population inherits not only this individual but also the scale factor if it seems promising at the current stage of evolution. In addition, a perturbation mechanism enhances the exploration feature of the algorithm. The proposed algorithm has been run on a set of various test problems and then compared to two sequential differential evolution algorithms and three distributed differential evolution algorithms recently proposed in literature and representing state-of-the-art in the field. Numerical results show that the proposed approach seems very efficient for most of the analyzed problems, and outperforms all other algorithms considered in this study.
---
paper_title: Super-fit control adaptation in memetic differential evolution frameworks

paper_content:

This paper proposes the super-fit memetic differential evolution (SFMDE). This algorithm employs a differential evolution (DE) framework hybridized with three meta-heuristics, each having different roles and features. Particle Swarm Optimization assists the DE in the beginning of the optimization process by helping to generate a super-fit individual. The two other meta-heuristics are local searchers adaptively coordinated by means of an index measuring quality of the super-fit individual with respect to the rest of the population. The choice of the local searcher and its application is then executed by means of a probabilistic scheme which makes use of the generalized beta distribution. These two local searchers are the Nelder mead algorithm and the Rosenbrock Algorithm. The SFMDE has been tested on two engineering problems; the first application is the optimal control drive design for a direct current (DC) motor, the second is the design of a digital filter for image processing purposes. Numerical results show that the SFMDE is a flexible and promising approach which has a high performance standard in terms of both final solutions detected and convergence speed.
---
paper_title: DEPSO: hybrid particle swarm with differential evolution operator

paper_content:

A hybrid particle swarm with differential evolution operator, termed DEPSO, which provide the bell-shaped mutations with consensus on the population diversity along with the evolution, while keeping the self-organized particle swarm dynamics, is proposed. Then it is applied to a set of benchmark functions, and the experimental results illustrate its efficiency.
---
paper_title: Differential Evolution Based Particle Swarm Optimization

paper_content:

A new, almost parameter-free optimization algorithm is developed in this paper as a hybrid of the barebones particle swarm optimizer (PSO) and differential evolution (DE). The DE is used to mutate, for each particle, the attractor associated with that particle, defined as a weighted average of its personal and neighborhood best positions. Results of this algorithm are compared to that of the barebones PSO, Von Neumann PSO, a DE PSO, and DE/rand/1/bin. These results show that the new algorithm provides excellent results with the added advantage that no parameter tuning is needed
---
paper_title: A novel particle swarm niching technique based on extensive vector operations

paper_content:

Several techniques have been proposed to extend the particle swarm optimization (PSO) paradigm so that multiple optima can be located and maintained within a convoluted search space. A significant number of these implementations are subswarm-based, that is, portions of the swarm are optimized separately. Niches are formed to contain these subswarms, a process that often requires user-specified parameters. The proposed technique, known as the vector-based PSO, uses a novel approach to locate and maintain niches by using additional vector operations to determine niche boundaries. As the standard PSO uses weighted vector combinations to update particle positions and velocities, the niching technique builds upon existing knowledge of the particle swarm. Once niche boundaries have been calculated, the swarm can be organized into subswarms without prior knowledge of the number of niches and their corresponding niche radii. This paper presents the vector-based PSO with emphasis on its underlying principles. Results for a number of functions with different characteristics are reported and discussed. The performance of the vector-based PSO is also compared to two other niching techniques for particle swarm optimization.
---
paper_title: Locating multiple optima using particle swarm optimization

paper_content:

Many scientific and engineering applications require optimization methods to find more than one solution to multimodal optimization problems. This paper presents a new particle swarm optimization (PSO) technique to locate and refine multiple solutions to such problems. The technique, NichePSO, extends the inherent unimodal nature of the standard PSO approach by growing multiple swarms from an initial particle population. Each subswarm represents a different solution or niche; optimized individually. The outcome of the NichePSO algorithm is a set of particle swarms, each representing a unique solution. Experimental results are provided to show that NichePSO can successfully locate all optima on a small set of test functions. These results are compared with another PSO niching algorithm, lbest PSO, and two genetic algorithm niching approaches. The influence of control parameters is investigated, including the relationship between the swarm size
---
paper_title: Application of particle swarm optimization technique and its variants to generation expansion planning problem

paper_content:

Abstract This paper presents the application of particle swarm optimization (PSO) technique and its variants to least-cost generation expansion planning (GEP) problem. The GEP problem is a highly constrained, combinatorial optimization problem that can be solved by complete enumeration. PSO is one of the swarm intelligence (SI) techniques, which use the group intelligence behavior along with individual intelligence to solve the combinatorial optimization problem. A novel ‘virtual mapping procedure’ (VMP) is introduced to enhance the effectiveness of the PSO approaches. Penalty function approach (PFA) is used to reduce the number of infeasible solutions in the subsequent iterations. In addition to simple PSO, many variants such as constriction factor approach (CFA), Lbest model, hybrid PSO (HPSO), stretched PSO (SPSO) and composite PSO (C-PSO) are also applied to test systems. The differential evolution (DE) technique is used for parameter setting of C-PSO. The PSO and its variants are applied to a synthetic test system of five types of candidate units with 6- and 14-year planning horizon. The results obtained are compared with dynamic programming (DP) in terms of speed and efficiency.
---
paper_title: Hybrid constrained simulated annealing and genetic algorithms for nonlinear constrained optimization

paper_content:

The paper presents a framework that unifies various search mechanisms for solving constrained nonlinear programming (NLP) problems. These problems are characterized by functions that are not necessarily differentiable and continuous. Our proposed framework is based on the first-order necessary and sufficient condition for constrained local minimization in discrete space that shows the equivalence between discrete-neighborhood saddle points and constrained local minima. To look for discrete-neighborhood saddle points, we formulate a discrete constrained NLP in an augmented Lagrangian function and study various mechanisms for performing ascents of the augmented function in the original-variable subspace and descents in the Lagrange-multiplier subspace. Our results show that CSAGA, a combined constrained simulated annealing (SA) and genetic algorithm (GA), performs well. Finally, we apply iterative deepening to determine the optimal number of generations in CSAGA and show that performance is robust with respect to changes in population size.
---
paper_title: Efficient and Accurate Parallel Genetic Algorithms

paper_content:

Preface. Acknowledgments. 1. Introduction. 2. The Gambler's Ruin and Population Sizing. 3. Master-Slave Parallel GAs. 4. Bounding Cases of GAs With Multiple Demes. 5. Markov Chain Models of Multiple Demes. 6. Migration Rates and Optimal Topologies. 7. Migration and Selection Pressure. 8. Fine-Grained and Hierarchical Parallel GAs. 9. Summary, Extensions, and Conclusions. References. Index.
---
paper_title: Environmental/economic power dispatch using a hybrid multi-objective optimization algorithm

paper_content:

Abstract To solve the highly constrained environmental/economic dispatch problem involving conflicting objectives, this paper presents a hybrid multi-objective optimization algorithm based on particle swarm optimization (PSO) and differential evolution (DE). In this algorithm, a PSO with time variant acceleration coefficients is designed to explore the entire search space, while a local version of DE is proposed to exploit the sub-space with sparse solutions. A crowing distance-based approach is introduced to assign the particles’ leaders and to update the external archive. Moreover, a new technique for equality constraints is proposed to hurdle the unfeasible solutions directly. Finally, several optimization trials of the proposed algorithm are carried out on the IEEE 30-bus test system. Results demonstrate superiority of the proposed approach and confirm its potential to solve the multi-objective EED problem.
---
paper_title: Efficient differential evolution using speciation for multimodal function optimization

paper_content:

In this paper differential evolution is extended by using the notion of speciation for solving multimodal optimization problems. The proposed species-based DE (SDE) is able to locate multiple global optima simultaneously through adaptive formation of multiple species (or subpopulations) in an DE population at each iteration step. Each species functions as an DE by itself. Successive local improvements through species formation can eventually transform into global improvements in identifying multiple global optima. In this study the performance of SDE is compared with another recently proposed DE variant CrowdingDE. The computational complexity of SDE, the effect of population size and species radius on SDE are investigated. SDE is found to be more computationally efficient than CrowdingDE over a number of benchmark multimodal test functions.
---
paper_title: A Taxonomy of Hybrid Metaheuristics

paper_content:

Hybrid metaheuristics have received considerable interest these recent years in the field of combinatorial optimization. A wide variety of hybrid approaches have been proposed in the literature. In this paper, a taxonomy of hybrid metaheuristics is presented in an attempt to provide a common terminology and classification mechanisms. The taxonomy, while presented in terms of metaheuristics, is also applicable to most types of heuristics and exact optimization algorithms. ::: ::: As an illustration of the usefulness of the taxonomy an annoted bibliography is given which classifies a large number of hybrid approaches according to the taxonomy.
---
paper_title: A Cooperative approach to particle swarm optimization

paper_content:

The particle swarm optimizer (PSO) is a stochastic, population-based optimization technique that can be applied to a wide range of problems, including neural network training. This paper presents a variation on the traditional PSO algorithm, called the cooperative particle swarm optimizer, or CPSO, employing cooperative behavior to significantly improve the performance of the original algorithm. This is achieved by using multiple swarms to optimize different components of the solution vector cooperatively. Application of the new PSO algorithm on several benchmark optimization problems shows a marked improvement in performance over the traditional PSO.
---
paper_title: Hybridizing PSO and DE for improved vector evaluated multi-objective optimization

paper_content:

This paper introduces a new vector evaluated multi-objective optimization algorithm. The vector evaluated differential evolution particle swarm optimization (VEDEPSO) algorithm is a hybridization of the classical vector evaluated particle swarm optimization (VEPSO) and vector evaluated differential evolution (VEDE) algorithms of Parsopoulos et. al. [9], [10]. Comparisons of VEDEPSO with respect to VEPSO and VEDE on a well known multi-objective benchmark problem set indicated that significant performance improvements can be attributed to the VEDEPSO algorithm.
---
paper_title: Metaheuristics in combinatorial optimization: Overview and conceptual comparison

paper_content:

The field of metaheuristics for the application to combinatorial optimization problems is a rapidly growing field of research. This is due to the importance of combinatorial optimization problems for the scientific as well as the industrial world. We give a survey of the nowadays most important metaheuristics from a conceptual point of view. We outline the different components and concepts that are used in the different metaheuristics in order to analyze their similarities and differences. Two very important concepts in metaheuristics are intensification and diversification. These are the two forces that largely determine the behavior of a metaheuristic. They are in some way contrary but also complementary to each other. We introduce a framework, that we call the I&D frame, in order to put different intensification and diversification components into relation with each other. Outlining the advantages and disadvantages of different metaheuristic approaches we conclude by pointing out the importance of hybridization of metaheuristics as well as the integration of metaheuristics and other methods for optimization.
---
paper_title: An adaptive hybrid optimizer based on particle swarm and differential evolution for global optimization

paper_content:

This paper presents extensive experiments on a hybrid optimization algorithm (DEPSO) we recently developed by combining the advantages of two powerful population-based metaheuristics—differential evolution (DE) and particle swarm optimization (PSO). The hybrid optimizer achieves on-the-fly adaptation of evolution methods for individuals in a statistical learning way. Two primary parameters for the novel algorithm including its learning period and population size are empirically analyzed. The dynamics of the hybrid optimizer is revealed by tracking and analyzing the relative success ratio of PSO versus DE in the optimization of several typical problems. The comparison between the proposed DEPSO and its competitors involved in our previous research is enriched by using multiple rotated functions. Benchmark tests involving scalability test validate that the DEPSO is competent for the global optimization of numerical functions due to its high optimization quality and wide applicability.
---
paper_title: A hybrid of genetic algorithm and particle swarm optimization for recurrent network design

paper_content:

An evolutionary recurrent network which automates the design of recurrent neural/fuzzy networks using a new evolutionary learning algorithm is proposed in this paper. This new evolutionary learning algorithm is based on a hybrid of genetic algorithm (GA) and particle swarm optimization (PSO), and is thus called HGAPSO. In HGAPSO, individuals in a new generation are created, not only by crossover and mutation operation as in GA, but also by PSO. The concept of elite strategy is adopted in HGAPSO, where the upper-half of the best-performing individuals in a population are regarded as elites. However, instead of being reproduced directly to the next generation, these elites are first enhanced. The group constituted by the elites is regarded as a swarm, and each elite corresponds to a particle within it. In this regard, the elites are enhanced by PSO, an operation which mimics the maturing phenomenon in nature. These enhanced elites constitute half of the population in the new generation, whereas the other half is generated by performing crossover and mutation operation on these enhanced elites. HGAPSO is applied to recurrent neural/fuzzy network design as follows. For recurrent neural network, a fully connected recurrent neural network is designed and applied to a temporal sequence production problem. For recurrent fuzzy network design, a Takagi-Sugeno-Kang-type recurrent fuzzy network is designed and applied to dynamic plant control. The performance of HGAPSO is compared to both GA and PSO in these recurrent networks design problems, demonstrating its superiority.
---
paper_title: Hybrid methods using genetic algorithms for global optimization

paper_content:

This paper discusses the trade-off between accuracy, reliability and computing time in global optimization. Particular compromises provided by traditional methods (Quasi-Newton and Nelder-Mead's simplex methods) and genetic algorithms are addressed and illustrated by a particular application in the field of nonlinear system identification. Subsequently, new hybrid methods are designed, combining principles from genetic algorithms and "hill-climbing" methods in order to find a better compromise to the trade-off. Inspired by biology and especially by the manner in which living beings adapt themselves to their environment, these hybrid methods involve two interwoven levels of optimization, namely evolution (genetic algorithms) and individual learning (Quasi-Newton), which cooperate in a global process of optimization. One of these hybrid methods appears to join the group of state-of-the-art global optimization methods: it combines the reliability properties of the genetic algorithms with the accuracy of Quasi-Newton method, while requiring a computation time only slightly higher than the latter.
---
paper_title: A multi-objective genetic local search algorithm and its application to flowshop scheduling

paper_content:

We propose a hybrid algorithm for finding a set of nondominated solutions of a multi objective optimization problem. In the proposed algorithm, a local search procedure is applied to each solution (i.e., each individual) generated by genetic operations. Our algorithm uses a weighted sum of multiple objectives as a fitness function. The fitness function is utilized when a pair of parent solutions are selected for generating a new solution by crossover and mutation operations. A local search procedure is applied to the new solution to maximize its fitness value. One characteristic feature of our algorithm is to randomly specify weight values whenever a pair of parent solutions are selected. That is, each selection (i.e., the selection of two parent solutions) is performed by a different weight vector. Another characteristic feature of our algorithm is not to examine all neighborhood solutions of a current solution in the local search procedure. Only a small number of neighborhood solutions are examined to prevent the local search procedure from spending almost all available computation time in our algorithm. High performance of our algorithm is demonstrated by applying it to multi objective flowshop scheduling problems.
---
paper_title: A Unified View on Hybrid Metaheuristics ?

paper_content:

Manifold possibilities of hybridizing individual metaheuristics with each other and/or with algorithms from other fields exist. A large number of publications documents the benefits and great success of such hybrids. This article overviews several popular hybridization approaches and classifies them based on various characteristics. In particular with respect to low-level hybrids of different metaheuristics, a unified view based on a common pool template is described. It helps in making similarities and different key components of existing metaheuristics explicit. We then consider these key components as a toolbox for building new, effective hybrid metaheuristics. This approach of thinking seems to be superior to sticking too strongly to the philosophies and historical backgrounds behind the different metaheuristic paradigms. Finally, particularly promising possibilities of combining metaheuristics with constraint programming and integer programming techniques are highlighted.
---
paper_title: Hybrid methods using genetic algorithms for global optimization

paper_content:

This paper discusses the trade-off between accuracy, reliability and computing time in global optimization. Particular compromises provided by traditional methods (Quasi-Newton and Nelder-Mead's simplex methods) and genetic algorithms are addressed and illustrated by a particular application in the field of nonlinear system identification. Subsequently, new hybrid methods are designed, combining principles from genetic algorithms and "hill-climbing" methods in order to find a better compromise to the trade-off. Inspired by biology and especially by the manner in which living beings adapt themselves to their environment, these hybrid methods involve two interwoven levels of optimization, namely evolution (genetic algorithms) and individual learning (Quasi-Newton), which cooperate in a global process of optimization. One of these hybrid methods appears to join the group of state-of-the-art global optimization methods: it combines the reliability properties of the genetic algorithms with the accuracy of Quasi-Newton method, while requiring a computation time only slightly higher than the latter.
---
paper_title: A multi-objective genetic local search algorithm and its application to flowshop scheduling

paper_content:

We propose a hybrid algorithm for finding a set of nondominated solutions of a multi objective optimization problem. In the proposed algorithm, a local search procedure is applied to each solution (i.e., each individual) generated by genetic operations. Our algorithm uses a weighted sum of multiple objectives as a fitness function. The fitness function is utilized when a pair of parent solutions are selected for generating a new solution by crossover and mutation operations. A local search procedure is applied to the new solution to maximize its fitness value. One characteristic feature of our algorithm is to randomly specify weight values whenever a pair of parent solutions are selected. That is, each selection (i.e., the selection of two parent solutions) is performed by a different weight vector. Another characteristic feature of our algorithm is not to examine all neighborhood solutions of a current solution in the local search procedure. Only a small number of neighborhood solutions are examined to prevent the local search procedure from spending almost all available computation time in our algorithm. High performance of our algorithm is demonstrated by applying it to multi objective flowshop scheduling problems.
---
paper_title: Hybridizing exact methods and metaheuristics: A taxonomy

paper_content:

The interest about hybrid optimization methods has grown for the last few years. Indeed, more and more papers about cooperation between heuristics and exact techniques are published. In this paper, we propose to extend an existing taxonomy for hybrid methods involving heuristic approaches in order to consider cooperative schemes between exact methods and metaheuristics. First, we propose some natural approaches for the different schemes of cooperation encountered, and we analyse, for each model, some examples taken from the literature. Then we recall and complement the proposed grammar and provide an annotated bibliography.
---
paper_title: Fitness landscape analysis and memetic algorithms for the quadratic assignment problem

paper_content:

In this paper, a fitness landscape analysis for several instances of the quadratic assignment problem (QAP) is performed, and the results are used to classify problem instances according to their hardness for local search heuristics and meta-heuristics based on local search. The local properties of the fitness landscape are studied by performing an autocorrelation analysis, while the global structure is investigated by employing a fitness distance correlation analysis. It is shown that epistasis, as expressed by the dominance of the flow and distance matrices of a QAP instance, the landscape ruggedness in terms of the correlation length of a landscape, and the correlation between fitness and distance of local optima in the landscape together are useful for predicting the performance of memetic algorithms-evolutionary algorithms incorporating local search (to a certain extent). Thus, based on these properties, a favorable choice of recombination and/or mutation operators can be found. Experiments comparing three different evolutionary operators for a memetic algorithm are presented.
---
paper_title: Meta-Lamarckian learning in memetic algorithms

paper_content:

Over the last decade, memetic algorithms (MAs) have relied on the use of a variety of different methods as the local improvement procedure. Some recent studies on the choice of local search method employed have shown that this choice significantly affects the efficiency of problem searches. Given the restricted theoretical knowledge available in this area and the limited progress made on mitigating the effects of incorrect local search method choice, we present strategies for MA control that decide, at runtime, which local method is chosen to locally improve the next chromosome. The use of multiple local methods during a MA search in the spirit of Lamarckian learning is here termed Meta-Lamarckian learning. Two adaptive strategies for Meta-Lamarckian learning are proposed in this paper. Experimental studies with Meta-Lamarckian learning strategies on continuous parametric benchmark problems are also presented. Further, the best strategy proposed is applied to a real-world aerodynamic wing design problem and encouraging results are obtained. It is shown that the proposed approaches aid designers working on complex engineering problems by reducing the probability of employing inappropriate local search methods in a MA, while at the same time, yielding robust and improved design search performance.
---
paper_title: A Taxonomy of Hybrid Metaheuristics

paper_content:

Hybrid metaheuristics have received considerable interest these recent years in the field of combinatorial optimization. A wide variety of hybrid approaches have been proposed in the literature. In this paper, a taxonomy of hybrid metaheuristics is presented in an attempt to provide a common terminology and classification mechanisms. The taxonomy, while presented in terms of metaheuristics, is also applicable to most types of heuristics and exact optimization algorithms. ::: ::: As an illustration of the usefulness of the taxonomy an annoted bibliography is given which classifies a large number of hybrid approaches according to the taxonomy.
---
paper_title: A tutorial for competent memetic algorithms: model, taxonomy, and design issues

paper_content:

The combination of evolutionary algorithms with local search was named "memetic algorithms" (MAs) (Moscato, 1989). These methods are inspired by models of natural systems that combine the evolutionary adaptation of a population with individual learning within the lifetimes of its members. Additionally, MAs are inspired by Richard Dawkin's concept of a meme, which represents a unit of cultural evolution that can exhibit local refinement (Dawkins, 1976). In the case of MA's, "memes" refer to the strategies (e.g., local refinement, perturbation, or constructive methods, etc.) that are employed to improve individuals. In this paper, we review some works on the application of MAs to well-known combinatorial optimization problems, and place them in a framework defined by a general syntactic model. This model provides us with a classification scheme based on a computable index D, which facilitates algorithmic comparisons and suggests areas for future research. Also, by having an abstract model for this class of metaheuristics, it is possible to explore their design space and better understand their behavior from a theoretical standpoint. We illustrate the theoretical and practical relevance of this model and taxonomy for MAs in the context of a discussion of important design issues that must be addressed to produce effective and efficient MAs.
---
paper_title: A Taxonomy of Evolutionary Algorithms in Combinatorial Optimization

paper_content:

This paper shows how evolutionary algorithms can be described in a concise, yet comprehensive and accurate way. A classification scheme is introduced and presented in a tabular form called TEA (Table of Evolutionary Algorithms). It distinguishes between different classes of evolutionary algorithms (e.g., genetic algorithms, ant systems) by enumerating the fundamental ingredients of each of these algorithms. At the end, possible uses of the TEA are illustrated on classical evolutionary algorithms.
---
paper_title: Improved particle swarm optimization combined with chaos

paper_content:

Abstract As a novel optimization technique, chaos has gained much attention and some applications during the past decade. For a given energy or cost function, by following chaotic ergodic orbits, a chaotic dynamic system may eventually reach the global optimum or its good approximation with high probability. To enhance the performance of particle swarm optimization (PSO), which is an evolutionary computation technique through individual improvement plus population cooperation and competition, hybrid particle swarm optimization algorithm is proposed by incorporating chaos. Firstly, adaptive inertia weight factor (AIWF) is introduced in PSO to efficiently balance the exploration and exploitation abilities. Secondly, PSO with AIWF and chaos are hybridized to form a chaotic PSO (CPSO), which reasonably combines the population-based evolutionary searching ability of PSO and chaotic searching behavior. Simulation results and comparisons with the standard PSO and several meta-heuristics show that the CPSO can effectively enhance the searching efficiency and greatly improve the searching quality.
---
paper_title: Hybrid methods using genetic algorithms for global optimization

paper_content:

This paper discusses the trade-off between accuracy, reliability and computing time in global optimization. Particular compromises provided by traditional methods (Quasi-Newton and Nelder-Mead's simplex methods) and genetic algorithms are addressed and illustrated by a particular application in the field of nonlinear system identification. Subsequently, new hybrid methods are designed, combining principles from genetic algorithms and "hill-climbing" methods in order to find a better compromise to the trade-off. Inspired by biology and especially by the manner in which living beings adapt themselves to their environment, these hybrid methods involve two interwoven levels of optimization, namely evolution (genetic algorithms) and individual learning (Quasi-Newton), which cooperate in a global process of optimization. One of these hybrid methods appears to join the group of state-of-the-art global optimization methods: it combines the reliability properties of the genetic algorithms with the accuracy of Quasi-Newton method, while requiring a computation time only slightly higher than the latter.
---
paper_title: An Effective PSO-Based Hybrid Algorithm for Multiobjective Permutation Flow Shop Scheduling

paper_content:

This paper proposes a hybrid algorithm based on particle swarm optimization (PSO) for a multiobjective permutation flow shop scheduling problem, which is a typical NP-hard combinatorial optimization problem with strong engineering backgrounds. Not only does the proposed multiobjective algorithm (named MOPSO) apply the parallel evolution mechanism of PSO characterized by individual improvement, population cooperation, and competition to effectively perform exploration but it also utilizes several adaptive local search methods to perform exploitation. First, to make PSO suitable for solving scheduling problems, a ranked-order value (ROV) rule based on a random key technique to convert the continuous position values of particles to job permutations is presented. Second, a multiobjective local search based on the Nawaz-Enscore-Ham heuristic is applied to good solutions with a specified probability to enhance the exploitation ability. Third, to enrich the searching behavior and to avoid premature convergence, a multiobjective local search based on simulated annealing with multiple different neighborhoods is designed, and an adaptive meta-Lamarckian learning strategy is employed to decide which neighborhood will be used. Due to the fusion of multiple different searching operations, good solutions approximating the real Pareto front can be obtained. In addition, MOPSO adopts a random weighted linear sum function to aggregate multiple objectives to a single one for solution evaluation and for guiding the evolution process in the multiobjective sense. Due to the randomness of weights, searching direction can be enriched, and solutions with good diversity can be obtained. Simulation results and comparisons based on a variety of instances demonstrate the effectiveness, efficiency, and robustness of the proposed hybrid algorithm.
---
paper_title: A multi-objective genetic local search algorithm and its application to flowshop scheduling

paper_content:

We propose a hybrid algorithm for finding a set of nondominated solutions of a multi objective optimization problem. In the proposed algorithm, a local search procedure is applied to each solution (i.e., each individual) generated by genetic operations. Our algorithm uses a weighted sum of multiple objectives as a fitness function. The fitness function is utilized when a pair of parent solutions are selected for generating a new solution by crossover and mutation operations. A local search procedure is applied to the new solution to maximize its fitness value. One characteristic feature of our algorithm is to randomly specify weight values whenever a pair of parent solutions are selected. That is, each selection (i.e., the selection of two parent solutions) is performed by a different weight vector. Another characteristic feature of our algorithm is not to examine all neighborhood solutions of a current solution in the local search procedure. Only a small number of neighborhood solutions are examined to prevent the local search procedure from spending almost all available computation time in our algorithm. High performance of our algorithm is demonstrated by applying it to multi objective flowshop scheduling problems.
---
paper_title: An effective hybrid DE-based algorithm for multi-objective flow shop scheduling with limited buffers

paper_content:

This paper proposes an effective hybrid algorithm based on differential evolution (DE), namely HDE, to solve multi-objective permutation flow shop scheduling problem (MPFSSP) with limited buffers between consecutive machines, which is a typical NP-hard combinatorial optimization problem with strong engineering background. Firstly, to make DE suitable for solving scheduling problems, a largest-order-value (LOV) rule is presented to convert the continuous values of individuals in DE to job permutations. Secondly, after the DE-based exploration, an efficient local search, which is designed based on the landscape of MPFSSP with limited buffers, is applied to emphasize exploitation. Thus, not only does the HDE apply the parallel evolution mechanism of DE to perform effective exploration (global search) in the whole solution space, but it also adopts problem-dependent local search to perform thorough exploitation (local search) in the promising sub-regions. In addition, the concept of Pareto dominance is used to handle the updating of solutions in sense of multi-objective optimization. Moreover, the convergence property of HDE is analyzed by using the theory of finite Markov chain. Finally, simulations and comparisons based on benchmarks demonstrate the effectiveness and efficiency of the proposed HDE.
---
paper_title: ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems

paper_content:

This paper concerns multiobjective optimization in scenarios where each solution evaluation is financially and/or temporally expensive. We make use of nine relatively low-dimensional, nonpathological, real-valued functions, such as arise in many applications, and assess the performance of two algorithms after just 100 and 250 (or 260) function evaluations. The results show that NSGA-II, a popular multiobjective evolutionary algorithm, performs well compared with random search, even within the restricted number of evaluations used. A significantly better performance (particularly, in the worst case) is, however, achieved on our test set by an algorithm proposed herein-ParEGO-which is an extension of the single-objective efficient global optimization (EGO) algorithm of Jones et al. ParEGO uses a design-of-experiments inspired initialization procedure and learns a Gaussian processes model of the search landscape, which is updated after every function evaluation. Overall, ParEGO exhibits a promising performance for multiobjective optimization problems where evaluations are expensive or otherwise restricted in number.
---
paper_title: Accelerating Differential Evolution Using an Adaptive Local Search

paper_content:

We propose a crossover-based adaptive local search (LS) operation for enhancing the performance of standard differential evolution (DE) algorithm. Incorporating LS heuristics is often very useful in designing an effective evolutionary algorithm for global optimization. However, determining a single LS length that can serve for a wide range of problems is a critical issue. We present a LS technique to solve this problem by adaptively adjusting the length of the search, using a hill-climbing heuristic. The emphasis of this paper is to demonstrate how this LS scheme can improve the performance of DE. Experimenting with a wide range of benchmark functions, we show that the proposed new version of DE, with the adaptive LS, performs better, or at least comparably, to classic DE algorithm. Performance comparisons with other LS heuristics and with some other well-known evolutionary algorithms from literature are also presented.
---
paper_title: Hybrid metaheuristics with evolutionary algorithms specializing in intensification and diversification: Overview and progress report

paper_content:

Nowadays, a promising way to obtain hybrid metaheuristics concerns the combination of several search algorithms with strong specialization in intensification and/or diversification. The flexible architecture of evolutionary algorithms allows specialized models to be obtained with the aim of providing intensification and/or diversification. The outstanding role that is played by evolutionary algorithms at present justifies the choice of their specialist approaches as suitable ingredients to build hybrid metaheuristics. This paper focuses on hybrid metaheuristics with evolutionary algorithms specializing in intensification and diversification. We first give an overview of the existing research on this topic, describing several instances grouped into three categories that were identified after reviewing specialized literature. Then, with the aim of complementing the overview and providing additional results and insights on this line of research, we present an instance that consists of an iterated local search algorithm with an evolutionary perturbation technique. The benefits of the proposal in comparison to other iterated local search algorithms proposed in the literature to deal with binary optimization problems are experimentally shown. The good performance of the reviewed approaches and the suitable results shown by our instance allow an important conclusion to be achieved: the use of evolutionary algorithms specializing in intensification and diversification for building hybrid metaheuristics becomes a prospective line of research for obtaining effective search algorithms.
---
paper_title: Adding Local Search to Particle Swarm Optimization

paper_content:

Particle swarm optimization is a stochastic algorithm for optimizing continuous functions. It uses a population of particles that follow trajectories through the search space towards good optima. This paper proposes adding a local search component to PSO to improve its convergence speed. Two possible methods are discussed. The first adds a term containing estimated gradient information to the velocity of each particle. The second explicitly incorporates the Nelder-Mead algorithm, a known local search technique, within PSO. The suggested methods have been applied to the problem of estimating parameters of a gene network model. Results indicate the effectiveness of the proposed strategies.
---
paper_title: A hybrid of genetic algorithm and particle swarm optimization for recurrent network design

paper_content:

An evolutionary recurrent network which automates the design of recurrent neural/fuzzy networks using a new evolutionary learning algorithm is proposed in this paper. This new evolutionary learning algorithm is based on a hybrid of genetic algorithm (GA) and particle swarm optimization (PSO), and is thus called HGAPSO. In HGAPSO, individuals in a new generation are created, not only by crossover and mutation operation as in GA, but also by PSO. The concept of elite strategy is adopted in HGAPSO, where the upper-half of the best-performing individuals in a population are regarded as elites. However, instead of being reproduced directly to the next generation, these elites are first enhanced. The group constituted by the elites is regarded as a swarm, and each elite corresponds to a particle within it. In this regard, the elites are enhanced by PSO, an operation which mimics the maturing phenomenon in nature. These enhanced elites constitute half of the population in the new generation, whereas the other half is generated by performing crossover and mutation operation on these enhanced elites. HGAPSO is applied to recurrent neural/fuzzy network design as follows. For recurrent neural network, a fully connected recurrent neural network is designed and applied to a temporal sequence production problem. For recurrent fuzzy network design, a Takagi-Sugeno-Kang-type recurrent fuzzy network is designed and applied to dynamic plant control. The performance of HGAPSO is compared to both GA and PSO in these recurrent networks design problems, demonstrating its superiority.
---
paper_title: Optimal Contraction Theorem for Exploration–Exploitation Tradeoff in Search and Optimization

paper_content:

Global optimization process can often be divided into two subprocesses: exploration and exploitation. The tradeoff between exploration and exploitation (T:Er&Ei) is crucial in search and optimization, having a great effect on global optimization performance, e.g., accuracy and convergence speed of optimization algorithms. In this paper, definitions of exploration and exploitation are first given based on information correlation among samplings. Then, some general indicators of optimization hardness are presented to characterize problem difficulties. By analyzing a typical contraction-based three-stage optimization process, optimal contraction theorem is presented to show that T:Er&Ei depends on the optimization hardness of problems to be optimized. T:Er&Ei will gradually lean toward exploration as optimization hardness increases. In the case of great optimization hardness, exploration-dominated optimizers outperform exploitation-dominated optimizers. In particular, random sampling will become an outstanding optimizer when optimization hardness reaches a certain degree. Besides, the optimal number of contraction stages increases with optimization hardness. In an optimal contraction way, the whole sampling cost is evenly distributed in all contraction stages, and each contraction takes the same contracting ratio. Furthermore, the characterization of optimization hardness is discussed in detail. The experiments with several typical global optimization algorithms used to optimize three groups of test problems validate the correctness of the conclusions made by T:Er&Ei analysis.
---
paper_title: An effective hybrid optimization strategy for job-shop scheduling problems

paper_content:

Abstract Simulated annealing is a naturally serial algorithm, but its behavior can be controlled by the cooling schedule. Genetic algorithm exhibits implicit parallelism and can retain useful redundant information about what is learned from previous searches by its representation in individuals in the population, but GA may lose solutions and substructures due to the disruptive effects of genetic operators and is not easy to regulate GA's convergence. By reasonably combining these two global probabilistic search algorithms, we develop a general, parallel and easily implemented hybrid optimization framework, and apply it to job-shop scheduling problems. Based on effective encoding scheme and some specific optimization operators, some benchmark job-shop scheduling problems are well solved by the hybrid optimization strategy, and the results are competitive with the best literature results. Besides the effectiveness and robustness of the hybrid strategy, the combination of different search mechanisms and structures can relax the parameter-dependence of GA and SA. Scope and purpose Job-shop scheduling problem (JSP) is one of the most well-known machine scheduling problems and one of the strongly NP-hard combinatorial optimization problems. Developing effective search methods is always an important and valuable work. The scope and purpose of this paper is to present a parallel and easily implemented hybrid optimization framework, which reasonably combines genetic algorithm with simulated annealing. Based on effective encoding scheme and some specific optimization operators, the job-shop scheduling problems are well solved by the hybrid optimization strategy.
---
paper_title: Meta-Lamarckian learning in memetic algorithms

paper_content:

Over the last decade, memetic algorithms (MAs) have relied on the use of a variety of different methods as the local improvement procedure. Some recent studies on the choice of local search method employed have shown that this choice significantly affects the efficiency of problem searches. Given the restricted theoretical knowledge available in this area and the limited progress made on mitigating the effects of incorrect local search method choice, we present strategies for MA control that decide, at runtime, which local method is chosen to locally improve the next chromosome. The use of multiple local methods during a MA search in the spirit of Lamarckian learning is here termed Meta-Lamarckian learning. Two adaptive strategies for Meta-Lamarckian learning are proposed in this paper. Experimental studies with Meta-Lamarckian learning strategies on continuous parametric benchmark problems are also presented. Further, the best strategy proposed is applied to a real-world aerodynamic wing design problem and encouraging results are obtained. It is shown that the proposed approaches aid designers working on complex engineering problems by reducing the probability of employing inappropriate local search methods in a MA, while at the same time, yielding robust and improved design search performance.
---
paper_title: An Effective PSO-Based Memetic Algorithm for Flow Shop Scheduling

paper_content:

This paper proposes an effective particle swarm optimization (PSO)-based memetic algorithm (MA) for the permutation flow shop scheduling problem (PFSSP) with the objective to minimize the maximum completion time, which is a typical non-deterministic polynomial-time (NP) hard combinatorial optimization problem. In the proposed PSO-based MA (PSOMA), both PSO-based searching operators and some special local searching operators are designed to balance the exploration and exploitation abilities. In particular, the PSOMA applies the evolutionary searching mechanism of PSO, which is characterized by individual improvement, population cooperation, and competition to effectively perform exploration. On the other hand, the PSOMA utilizes several adaptive local searches to perform exploitation. First, to make PSO suitable for solving PFSSP, a ranked-order value rule based on random key representation is presented to convert the continuous position values of particles to job permutations. Second, to generate an initial swarm with certain quality and diversity, the famous Nawaz-Enscore-Ham (NEH) heuristic is incorporated into the initialization of population. Third, to balance the exploration and exploitation abilities, after the standard PSO-based searching operation, a new local search technique named NEH_1 insertion is probabilistically applied to some good particles selected by using a roulette wheel mechanism with a specified probability. Fourth, to enrich the searching behaviors and to avoid premature convergence, a simulated annealing (SA)-based local search with multiple different neighborhoods is designed and incorporated into the PSOMA. Meanwhile, an effective adaptive meta-Lamarckian learning strategy is employed to decide which neighborhood to be used in SA-based local search. Finally, to further enhance the exploitation ability, a pairwise-based local search is applied after the SA-based search. Simulation results based on benchmarks demonstrate the effectiveness of the PSOMA. Additionally, the effects of some parameters on optimization performances are also discussed
---
paper_title: A Multiobjective Evolutionary-Simplex Hybrid Approach for the Optimization of Differential Equation Models of Gene Networks

paper_content:

This paper describes genetic and hybrid approaches for multiobjective optimization using a numerical measure called fuzzy dominance. Fuzzy dominance is used when implementing tournament selection within the genetic algorithm (GA). In the hybrid version, it is also used to carry out a Nelder-Mead simplex-based local search. The proposed GA is shown to perform better than NSGA-II and SPEA-2 on standard benchmarks, as well as for the optimization of a genetic model for flowering time control in rice. Adding the local search achieves faster convergence, an important feature in computationally intensive optimization of gene networks. The hybrid version also compares well with ParEGO on a few other benchmarks. The proposed hybrid algorithm is then applied to estimate the parameters of an elaborate gene network model of flowering time control in Arabidopsis. Overall solution quality is quite good by biological standards. Tradeoffs are discussed between accuracy in gene activity levels versus in the plant traits that they influence. These tradeoffs suggest that data mining the Pareto front may be useful in bioinformatics.
---
paper_title: A tutorial for competent memetic algorithms: model, taxonomy, and design issues

paper_content:

The combination of evolutionary algorithms with local search was named "memetic algorithms" (MAs) (Moscato, 1989). These methods are inspired by models of natural systems that combine the evolutionary adaptation of a population with individual learning within the lifetimes of its members. Additionally, MAs are inspired by Richard Dawkin's concept of a meme, which represents a unit of cultural evolution that can exhibit local refinement (Dawkins, 1976). In the case of MA's, "memes" refer to the strategies (e.g., local refinement, perturbation, or constructive methods, etc.) that are employed to improve individuals. In this paper, we review some works on the application of MAs to well-known combinatorial optimization problems, and place them in a framework defined by a general syntactic model. This model provides us with a classification scheme based on a computable index D, which facilitates algorithmic comparisons and suggests areas for future research. Also, by having an abstract model for this class of metaheuristics, it is possible to explore their design space and better understand their behavior from a theoretical standpoint. We illustrate the theoretical and practical relevance of this model and taxonomy for MAs in the context of a discussion of important design issues that must be addressed to produce effective and efficient MAs.
---
paper_title: DE/EDA: A new evolutionary algorithm for global optimization

paper_content:

Differential evolution (DE) was very successful in solving the global continuous optimization problem. It mainly uses the distance and direction information from the current population to guide its further search. Estimation of distribution algorithm (EDA) samples new solutions from a probability model which characterizes the distribution of promising solutions. This paper proposes a combination of DE and EDA (DE/EDA) for the global continuous optimization problem. DE/EDA combines global information extracted by EDA with differential information obtained by DE to create promising solutions. DE/EDA has been compared with the best version of the DE algorithm and an EDA on several commonly utilized test problems. Experimental results demonstrate that DE/EDA outperforms the DE algorithm and the EDA. The effect of the parameters of DE/EDA to its performance is investigated experimentally.
---
paper_title: Multi-objective hybrid PSO using µ-fuzzy dominance

paper_content:

This paper describes a PSO-Nelder Mead Simplex hybrid multi-objective optimization algorithm based on a numerical metric called µ -fuzzy dominance. Within each iteration of this approach, in addition to the position and velocity update of each particle using PSO, the k-means algorithm is applied to divide the population into smaller sized clusters. The Nelder-Mead simplex algorithm is used separately within each cluster for added local search. The proposed algorithm is shown to perform better than MOPSO on several test problems as well as for the optimization of a genetic model for flowering time control in Arabidopsis. Adding the local search achieves faster convergence, an important feature in computationally intensive optimization of gene networks.
---
paper_title: Hill Climbers and Mutational Heuristics in Hyperheuristics

paper_content:

Hyperheuristics are single candidate solution based and simple to maintain mechanisms used in optimization. At each iteration, as a higher level of abstraction, a hyperheuristic chooses and applies one of the heuristics to a candidate solution. In this study, the performance contribution of hill climbing operators along with the mutational heuristics are analyzed in depth in four different hyperheuristic frameworks. Four different hill climbing operators and three mutational operators are used during the experiments. Various subsets of the heuristics are evaluated on fourteen well-known benchmark functions.
---
paper_title: Classification of adaptive memetic algorithms: a comparative study

paper_content:

Adaptation of parameters and operators represents one of the recent most important and promising areas of research in evolutionary computations; it is a form of designing self-configuring algorithms that acclimatize to suit the problem in hand. Here, our interests are on a recent breed of hybrid evolutionary algorithms typically known as adaptive memetic algorithms (MAs). One unique feature of adaptive MAs is the choice of local search methods or memes and recent studies have shown that this choice significantly affects the performances of problem searches. In this paper, we present a classification of memes adaptation in adaptive MAs on the basis of the mechanism used and the level of historical knowledge on the memes employed. Then the asymptotic convergence properties of the adaptive MAs considered are analyzed according to the classification. Subsequently, empirical studies on representatives of adaptive MAs for different type-level meme adaptations using continuous benchmark problems indicate that global-level adaptive MAs exhibit better search performances. Finally we conclude with some promising research directions in the area.
---
paper_title: Large scale evolutionary optimization using cooperative coevolution q

paper_content:

Evolutionary algorithms (EAs) have been applied with success to many numerical and combinatorial optimization problems in recent years. However, they often lose their effectiveness and advantages when applied to large and complex problems, e.g., those with high dimensions. Although cooperative coevolution has been proposed as a promising framework for tackling high-dimensional optimization problems, only limited studies were reported by decomposing a high-dimensional problem into single variables (dimensions). Such methods of decomposition often failed to solve nonseparable problems, for which tight interactions exist among different decision variables. In this paper, we propose a new cooperative coevolution framework that is capable of optimizing large scale nonseparable problems. A random grouping scheme and adaptive weighting are introduced in problem decomposition and coevolution. Instead of conventional evolutionary algorithms, a novel differential evolution algorithm is adopted. Theoretical analysis is presented in this paper to show why and how the new framework can be effective for optimizing large nonseparable problems. Extensive computational studies are also carried out to evaluate the performance of newly proposed algorithm on a large number of benchmark functions with up to 1000 dimensions. The results show clearly that our framework and algorithm are effective as well as efficient for large scale evolutionary optimisation problems. We are unaware of any other evolutionary algorithms that can optimize 1000-dimension nonseparable problems as effectively and efficiently as we have done.
---
paper_title: Combining Global and Local Surrogate Models to Accelerate Evolutionary Optimization

paper_content:

In this paper, we present a novel surrogate-assisted evolutionary optimization framework for solving computationally expensive problems. The proposed framework uses computationally cheap hierarchical surrogate models constructed through online learning to replace the exact computationally expensive objective functions during evolutionary search. At the first level, the framework employs a data-parallel Gaussian process based global surrogate model to filter the evolutionary algorithm (EA) population of promising individuals. Subsequently, these potential individuals undergo a memetic search in the form of Lamarckian learning at the second level. The Lamarckian evolution involves a trust-region enabled gradient-based search strategy that employs radial basis function local surrogate models to accelerate convergence. Numerical results are presented on a series of benchmark test functions and on an aerodynamic shape design problem. The results obtained suggest that the proposed optimization framework converges to good designs on a limited computational budget. Furthermore, it is shown that the new algorithm gives significant savings in computational cost when compared to the traditional evolutionary algorithm and other surrogate assisted optimization frameworks
---
paper_title: A Probabilistic Memetic Framework

paper_content:

Memetic algorithms (MAs) represent one of the recent growing areas in evolutionary algorithm (EA) research. The term MAs is now widely used as a synergy of evolutionary or any population-based approach with separate individual learning or local improvement procedures for problem search. Quite often, MAs are also referred to in the literature as Baldwinian EAs, Lamarckian EAs, cultural algorithms, or genetic local searches. In the last decade, MAs have been demonstrated to converge to high-quality solutions more efficiently than their conventional counterparts on a wide range of real-world problems. Despite the success and surge in interests on MAs, many of the successful MAs reported have been crafted to suit problems in very specific domains. Given the restricted theoretical knowledge available in the field of MAs and the limited progress made on formal MA frameworks, we present a novel probabilistic memetic framework that models MAs as a process involving the decision of embracing the separate actions of evolution or individual learning and analyzing the probability of each process in locating the global optimum. Further, the framework balances evolution and individual learning by governing the learning intensity of each individual according to the theoretical upper bound derived while the search progresses. Theoretical and empirical studies on representative benchmark problems commonly used in the literature are presented to demonstrate the characteristics and efficacies of the probabilistic memetic framework. Further, comparisons to recent state-of-the-art evolutionary algorithms, memetic algorithms, and hybrid evolutionary-local search demonstrate that the proposed framework yields robust and improved search performance.
---
paper_title: Hyper-Heuristics: An Emerging Direction In Modern Search Technology

paper_content:

This chapter introduces and overviews an emerging methodology in search and optimisation. One of the key aims of these new approaches, which have been termed hyperheuristics, is to raise the level of generality at which optimisation systems can operate. An objective is that hyper-heuristics will lead to more general systems that are able to handle a wide range of problem domains rather than current meta-heuristic technology which tends to be customised to a particular problem or a narrow class of problems. Hyper-heuristics are broadly concerned with intelligently choosing the right heuristic or algorithm in a given situation. Of course, a hyper-heuristic can be (often is) a (meta-)heuristic and it can operate on (meta-)heuristics. In a certain sense, a hyper-heuristic works at a higher level when compared with the typical application of meta-heuristics to optimisation problems, i.e., a hyper-heuristic could be thought of as a (meta)-heuristic which operates on lower level (meta-)heuristics. In this chapter we will introduce the idea and give a brief history of this emerging area. In addition, we will review some of the latest work to be published in the field.
---
paper_title: Meta-Lamarckian learning in memetic algorithms

paper_content:

Over the last decade, memetic algorithms (MAs) have relied on the use of a variety of different methods as the local improvement procedure. Some recent studies on the choice of local search method employed have shown that this choice significantly affects the efficiency of problem searches. Given the restricted theoretical knowledge available in this area and the limited progress made on mitigating the effects of incorrect local search method choice, we present strategies for MA control that decide, at runtime, which local method is chosen to locally improve the next chromosome. The use of multiple local methods during a MA search in the spirit of Lamarckian learning is here termed Meta-Lamarckian learning. Two adaptive strategies for Meta-Lamarckian learning are proposed in this paper. Experimental studies with Meta-Lamarckian learning strategies on continuous parametric benchmark problems are also presented. Further, the best strategy proposed is applied to a real-world aerodynamic wing design problem and encouraging results are obtained. It is shown that the proposed approaches aid designers working on complex engineering problems by reducing the probability of employing inappropriate local search methods in a MA, while at the same time, yielding robust and improved design search performance.
---
paper_title: CONTROL OF UNSTEADY SOLIDIFICATION VIA OPTIMIZED MAGNETIC FIELDS

paper_content:

ABSTRACT This article presents a numerical procedure for automatically controlling desired features of a melt undergoing solidification by applying an external magnetic field whose time-varying intensity and spatial distribution are obtained by the use of a hybrid optimization algorithm. The intensities of the magnets along the boundaries of the container were discretized by using B-splines. The inverse problem is then formulated to find the magnetic boundary conditions (the coefficients of the B-splines) in such a way that the gradients of temperature along the gravity direction are minimized at each instant as the solidification front advances through a moving melt. For this task, a hybrid optimization code was used that automatically switches among the following six optimization modules; the Davidon-Fletcher-Powell (DFP) gradient method, a genetic algorithm (GA), the Nelder-Mead (NM) simplex method, quasi-Newton algorithm of Pshenichny-Danilin (LM), differential evolution (DE), and sequential quadratic...
---
paper_title: Tackling high dimensional nonseparable optimization problems by cooperatively coevolving particle swarms

paper_content:

This paper attempts to address the question of scaling up Particle Swarm Optimization (PSO) algorithms to high dimensional optimization problems. We present a cooperative coevolving PSO (CCPSO) algorithm incorporating random grouping and adaptive weighting, two techniques that have been shown to be effective for handling high dimensional nonseparable problems. The proposed CCPSO algorithms out-performed a previously developed coevolving PSO algorithm on nonseparable functions of 30 dimensions. Furthermore, the scalability of the proposed algorithm to high dimensional nonseparable problems (of up to 1000 dimensions) is examined and compared with two existing coevolving Differential Evolution (DE) algorithms, and new insights are obtained. Our experimental results show the proposed CCPSO algorithms can perform reasonably well with only a small number of evaluations. The results also suggest that both the random grouping and adaptive weighting schemes are viable approaches that can be generalized to other evolutionary optimization methods.
---
paper_title: A tutorial for competent memetic algorithms: model, taxonomy, and design issues

paper_content:

The combination of evolutionary algorithms with local search was named "memetic algorithms" (MAs) (Moscato, 1989). These methods are inspired by models of natural systems that combine the evolutionary adaptation of a population with individual learning within the lifetimes of its members. Additionally, MAs are inspired by Richard Dawkin's concept of a meme, which represents a unit of cultural evolution that can exhibit local refinement (Dawkins, 1976). In the case of MA's, "memes" refer to the strategies (e.g., local refinement, perturbation, or constructive methods, etc.) that are employed to improve individuals. In this paper, we review some works on the application of MAs to well-known combinatorial optimization problems, and place them in a framework defined by a general syntactic model. This model provides us with a classification scheme based on a computable index D, which facilitates algorithmic comparisons and suggests areas for future research. Also, by having an abstract model for this class of metaheuristics, it is possible to explore their design space and better understand their behavior from a theoretical standpoint. We illustrate the theoretical and practical relevance of this model and taxonomy for MAs in the context of a discussion of important design issues that must be addressed to produce effective and efficient MAs.
---
paper_title: A Cooperative approach to particle swarm optimization

paper_content:

The particle swarm optimizer (PSO) is a stochastic, population-based optimization technique that can be applied to a wide range of problems, including neural network training. This paper presents a variation on the traditional PSO algorithm, called the cooperative particle swarm optimizer, or CPSO, employing cooperative behavior to significantly improve the performance of the original algorithm. This is achieved by using multiple swarms to optimize different components of the solution vector cooperatively. Application of the new PSO algorithm on several benchmark optimization problems shows a marked improvement in performance over the traditional PSO.
---
paper_title: No free lunch theorems for optimization

paper_content:

A framework is developed to explore the connection between effective optimization algorithms and the problems they are solving. A number of "no free lunch" (NFL) theorems are presented which establish that for any algorithm, any elevated performance over one class of problems is offset by performance over another class. These theorems result in a geometric interpretation of what it means for an algorithm to be well suited to an optimization problem. Applications of the NFL theorems to information-theoretic aspects of optimization and benchmark measures of performance are also presented. Other issues addressed include time-varying optimization problems and a priori "head-to-head" minimax distinctions between optimization algorithms, distinctions that result despite the NFL theorems' enforcing of a type of uniformity over all algorithms.
---
paper_title: A Particle Swarm Optimization Algorithm with Differential Evolution

paper_content:

Differential evolution (DE) is a simple evolutionary algorithm that has shown superior performance in the global continuous optimization. It mainly utilizes the differential information to guide its further search. But the differential information also results in instability of performance. Particle swarm optimization (PSO) has been developing rapidly and has been applied widely since it is introduced, as it can converge quickly. But PSO easily got stuck in local optima because it easily loses the diversity of swarm. This paper proposes a combination of DE and PSO (termed DEPSO) that makes up their disadvantages. DEPSO combines the differential information obtained by DE with the memory information extracted by PSO to create the promising solutions. Finally, DEPSO is tested to solve several benchmark optimization problems. The experimental results show the effectiveness of DEPSO algorithm for the multimodal function, and also verify that DEPSO can perform better than other algorithms (DE, CPSO) in solving the benchmark problems.
---

Title: Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy

Section 1: Introduction
Description 1: Introduce the background of hybrid optimization, key objectives of hybridizing DE and PSO, and the motivation behind this paper.

Section 2: Classical Differential Evolution
Description 2: Provide an overview of the traditional DE algorithm, its mechanisms, and commonly used variants.

Section 3: Classical Particle Swarm Optimization
Description 3: Present the traditional PSO algorithm, its mechanisms, and commonly used variants.

Section 4: Unified Views on Particle Swarm Optimization and Differential Evolution
Description 4: Discuss the similarities and differences between DE and PSO and the unified understanding of these algorithms.

Section 5: Previous DEPSOs
Description 5: Review previous works on DEPSO, classify them into categories based on their characteristics, and discuss specific examples.

Section 6: Taxonomy on Hybridization Strategies
Description 6: Propose a new taxonomy of hybridization strategies, including hybridization factors such as Hybridization Level (HL), Operating Order (OO), Type of Information Transfer (TIT), and Type of Transferred Information (TTI).

Section 7: Differences and Connections With Previous Taxonomies
Description 7: Compare the proposed taxonomy with previous taxonomies and discuss their advantages and connections.

Section 8: Discussion and Future Research on Hybrid Optimization
Description 8: Discuss key issues in hybrid optimization design, especially the tradeoff between exploration and exploitation, and identify promising directions for future research.

Section 9: Conclusion
Description 9: Summarize the key findings of the paper, the importance of the proposed taxonomy, and the potential for future research in hybrid optimization.

Section 10: Acknowledgement
Description 10: Acknowledge any contributions or support received in the preparation of this paper.