Feature selection is a critical and prominent task in machine learning. To reduce the dimension of the feature set while maintaining the accuracy of the performance is the main aim of the feature selection problem. Various methods have been developed to classify the datasets. However, metaheuristic algorithms have achieved great attention in solving numerous optimization problem. Therefore, this paper presents an extensive literature review on solving feature selection problem using metaheuristic algorithms which are developed in the ten years (2009-2019). Further, metaheuristic algorithms have been classified into four categories based on their behaviour. Moreover, a categorical list of more than a hundred metaheuristic algorithms is presented. To solve the feature selection problem, only binary variants of metaheuristic algorithms have been reviewed and corresponding to their categories, a detailed description of them explained. The metaheuristic algorithms in solving feature selection problem are given with their binary classification, name of the classifier used, datasets and the evaluation metrics. After reviewing the papers, challenges and issues are also identified in obtaining the best feature subset using different metaheuristic algorithms. Finally, some research gaps are also highlighted for the researchers who want to pursue their research in developing or modifying metaheuristic algorithms for classification. For an application, a case study is presented in which datasets are adopted from the UCI repository and numerous metaheuristic algorithms are employed to obtain the optimal feature subset.

/pdf/metaheuristic-algorithms-on-feature-selection-a-survey-of-ilv6sofidu.pdf

Metaheuristic Algorithms on Feature Selection: A Survey of One Decade of Research (2009-2019)

The performance of most metaheuristic algorithms depends on parameters whose settings essentially serve as a key function in determining the quality of the solution and the efficiency of the search. A trend that has emerged recently is to make the algorithm parameters automatically adapt to different problems during optimization, thereby liberating the user from the tedious and time-consuming task of manual setting. These fine-tuning techniques continue to be the object of ongoing research. Differential evolution (DE) is a simple yet powerful population-based metaheuristic. It has demonstrated good convergence, and its principles are easy to understand. DE is very sensitive to its parameter settings and mutation strategy; thus, this study aims to investigate these settings with the diverse versions of adaptive DE algorithms. This study has two main objectives: (1) to present an extension for the original taxonomy of evolutionary algorithms (EAs) parameter settings that has been overlooked by prior research and therefore minimize any confusion that might arise from the former taxonomy and (2) to investigate the various algorithmic design schemes that have been used in the different variants of adaptive DE and convey them in a new classification style. In other words, this study describes in depth the structural analysis and working principle that underlie the promising and recent work in this field, to analyze their advantages and disadvantages and to gain future insights that can further improve these algorithms. Finally, the interpretation of the literature and the comparative analysis of the algorithmic schemes offer several guidelines for designing and implementing adaptive DE algorithms. The proposed design framework provides readers with the main steps required to integrate any proposed meta-algorithm into parameter and/or strategy adaptation schemes.

Algorithmic design issues in adaptive differential evolution schemes: Review and taxonomy

Swarm intelligence (SI) is the collective behavior of decentralized, self-organized natural or artificial systems. Monarch butterfly optimization (MBO) algorithm is a class of swarm intelligence metaheuristic algorithm inspired by the migration behavior of monarch butterflies. Through the migration operation and butterfly adjusting operation, individuals in MBO are updated. MBO can outperform many state-of-the-art optimization techniques when solving global numerical optimization and engineering problems. This paper presents a comprehensive review of the MBO algorithm including its modifications, hybridizations, variants, and applications. Additionally, further research directions for MBO are discussed. This review study serves as a solid reference for future studies in the arena of SI and in particular the MBO algorithm.

Monarch butterfly optimization: A comprehensive review

In recent years, several multi-method and multi-operator-based algorithms have been proposed for solving optimization problems. Generally, their performance is better than other algorithms that based on a single operator and/or algorithm. However, they do not perform consistently well over all the problems tested in the literature. In this paper, we propose an improved optimization algorithm that uses the benefits of multiple differential evolution operators, with more emphasis placed on the best-performing operator. The performance of the proposed algorithm is tested by solving 10 problems with 5, 10, 15 and 20 dimensions taken from CEC2020 competition on single objective bound constrained optimization, with its results outperforming both single operator-based and different state-of-the-art algorithms.

Improved Multi-operator Differential Evolution Algorithm for Solving Unconstrained Problems

In this article, an effective metaheuristic algorithm named multi-trial vector-based differential evolution (MTDE) is proposed. The MTDE is distinguished by introducing an adaptive movement step designed based on a new multi-trial vector approach named MTV, which combines different search strategies in the form of trial vector producers (TVPs). In the developed MTV approach, the TVPs are applied on their dedicated subpopulation, which are distributed by a winner-based distribution policy, and share their experiences efficiently by using a life-time archive. The MTV can be deployed by different types of TVPs, particularly, we use the MTV approach in the MTDE algorithm by three TVPs: representative based trial vector producer, local random based trial vector producer, and global best history based trial vector producer. Therefore, this study introduces the MTV approach to boost the performance of the MTDE and demonstrates its advantages in dealing with problems of different levels of complexity. The performance of the proposed MTDE algorithm is evaluated on CEC 2018 benchmark suite which include unimodal, multimodal, hybrid, and composition functions and four complex engineering design problems. The experimental and statistical results are compared with state-of-the-art metaheuristic algorithms: GWO, WOA, SSA, HHO, CoDE, EPSDE, QUATRE, and MKE. The results demonstrate that the MTDE algorithm shows improved performance and benefits from high accuracy of optimal solutions obtained.

MTDE: An effective multi-trial vector-based differential evolution algorithm and its applications for engineering design problems

This paper proposes a novel nature-inspired algorithm called Gaining Sharing Knowledge based Algorithm (GSK) for solving optimization problems over continuous space. The GSK algorithm mimics the process of gaining and sharing knowledge during the human life span. It is based on two vital stages, junior gaining and sharing phase and senior gaining and sharing phase. The present work mathematically models these two phases to achieve the process of optimization. In order to verify and analyze the performance of GSK, numerical experiments on a set of 30 test problems from the CEC2017 benchmark for 10, 30, 50 and 100 dimensions. Besides, the GSK algorithm has been applied to solve the set of real world optimization problems proposed for the IEEE-CEC2011 evolutionary algorithm competition. A comparison with 10 state-of-the-art and recent metaheuristic algorithms are executed. Experimental results indicate that in terms of robustness, convergence and quality of the solution obtained, GSK is significantly better than, or at least comparable to state-of-the-art approaches with outstanding performance in solving optimization problems especially with high dimensions.

Gaining-sharing knowledge based algorithm for solving optimization problems: a novel nature-inspired algorithm

This paper presents adaptive guided differential evolution algorithm (AGDE) for solving global numerical optimization problems over continuous space. In order to utilize the information of good and bad vectors in the DE population, the proposed algorithm introduces a new mutation rule. It uses two random chosen vectors of the top and the bottom 100p% individuals in the current population of size NP while the third vector is selected randomly from the middle [NP-2(100p %)] individuals. This new mutation scheme helps maintain effectively the balance between the global exploration and local exploitation abilities for searching process of the DE. Besides, a novel and effective adaptation scheme is used to update the values of the crossover rate to appropriate values without either extra parameters or prior knowledge of the characteristics of the optimization problem. In order to verify and analyze the performance of AGDE, Numerical experiments on a set of 28 test problems from the CEC2013 benchmark for 10, 30, and 50 dimensions, including a comparison with classical DE schemes and some recent evolutionary algorithms are executed. Experimental results indicate that in terms of robustness, stability and quality of the solution obtained, AGDE is significantly better than, or at least comparable to state-of-the-art approaches.

Adaptive guided differential evolution algorithm with novel mutation for numerical optimization

This paper introduces an enhancement of the recent developed Gaining Sharing Knowledge based algorithm, dubbed as GSK. This algorithm is an excellent example of a contemporary nature-based algorithm which is inspired from the human life behavior of gaining and sharing knowledge to solve the optimization task. GSK algorithm simulates the natural phenomena of human gaining and sharing knowledge using two main phases: junior and senior. A set of initial solutions are generated at the beginning of the search which are consideredjuniors. Later, the individuals are moving to senior stage by interacting with the environment and cooperating with other solutions during the search.The key idea in this work is to extend and improve the original GSK algorithm by proposing adaptive settings to the two important control parameters: knowledge factor and knowledge ratio. These two parameters are responsible to control junior and senior gaining and sharing phases between the solutions during the optimization loop. The algorithm is named AGSK and tested on the recent benchmark suite on bound constrained numerical optimization which consists of different challenging optimization problems with different dimensions. This benchmark is presented in IEEE-CEC2020 competition. When compared with other state-of-the-art algorithms including original GSK, AGSK shows superior performance.

Evaluating the Performance of Adaptive GainingSharing Knowledge Based Algorithm on CEC 2020 Benchmark Problems

During last two decades, Differential Evolution (DE) proved to be one of the most popular and successful evolutionary algorithms for solving global optimization problems over continuous space. Proposing new mutation strategies to improve the optimization performance of (DE) is considered a significant research study. In DE, mutation operation plays a vital role in the performance of the algorithm. Therefore, in this paper, comprehensive analysis of the contributions on basic and novel mutation strategies that were proposed between 1995 and 2020 is presented. A new taxonomy based on the structure of the novel mutations is proposed. Numerical experiments on a set of 30 test problems from the CEC2017 benchmark for 10, 30, 50 and 100 dimensions, including a comparison with classical DE schemes and recent mutations schemes are executed. Furthermore, theoretical, and empirical convergence behavior analysis of all mutations is discussed. The paper also presents many recommendations, guidelines, insights, and suggestions for experienced practitioners and interested researchers in designing and developing effective and efficient DE algorithms to address various optimization problems in different fields.

/pdf/differential-evolution-mutations-taxonomy-comparison-and-3izlrzws7f.pdf

Differential Evolution Mutations: Taxonomy, Comparison and Convergence Analysis

Adaptive guided differential evolution algorithm (AGDE) is a differential evolution (DE) algorithm that utilizes the information of good and bad vectors in the population, it introduced a novel mutation rule in order to balance effectively the exploration and exploitation tradeoffs. It divided the population into three clusters (best, better and worst) with sizes 100p%, NP − 2 * 100% and 100% respectively. where p is the proportion of the partition with respect to the total number of individuals in the population (NP). AGDE selects three random individuals, one of each partition to implement the mutation process. Besides, a novel adaptation scheme was proposed in order to update the value of crossover rate without previous knowledge about the characteristics of the problems. This paper introduces enhanced AGDE (EAGDE) with non-linear population size reduction, which gradually decreases the population size according to a non-linear function. Moreover, a newly developed rule developed to determine the initial population size, that is related to the dimensionality of the problems. The performance of the proposed algorithm is evaluated using CEC2013 benchmarks and the results are compared with the state-of-art DE and non-DE algorithms, the results showed a great competitiveness for the proposed algorithm over the other algorithms, and the original AGDE.

Ali Khater Mohamed

Papers

Gaining-sharing knowledge based algorithm for solving optimization problems: a novel nature-inspired algorithm

Adaptive guided differential evolution algorithm with novel mutation for numerical optimization

Evaluating the Performance of Adaptive GainingSharing Knowledge Based Algorithm on CEC 2020 Benchmark Problems

Differential Evolution Mutations: Taxonomy, Comparison and Convergence Analysis

Real-Parameter Unconstrained Optimization Based on Enhanced AGDE Algorithm