Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Differential Evolution (DE) is arguably one of the most powerful and versatile evolutionary optimizers for the continuous parameter spaces in recent times. Almost 5 years have passed since the first comprehensive survey article was published on DE by Das and Suganthan in 2011. Several developments have been reported on various aspects of the algorithm in these 5 years and the research on and with DE have now reached an impressive state. Considering the huge progress of research with DE and its applications in diverse domains of science and technology, we find that it is a high time to provide a critical review of the latest literatures published and also to point out some important future avenues of research. The purpose of this paper is to summarize and organize the information on these current developments on DE. Beginning with a comprehensive foundation of the basic DE family of algorithms, we proceed through the recent proposals on parameter adaptation of DE, DE-based single-objective global optimizers, DE adopted for various optimization scenarios including constrained, large-scale, multi-objective, multi-modal and dynamic optimization, hybridization of DE with other optimizers, and also the multi-faceted literature on applications of DE. The paper also presents a dozen of interesting open problems and future research issues on DE.

Recent advances in differential evolution – An updated survey

This paper presents a novel nature-inspired optimization paradigm, named as squirrel search algorithm (SSA). This optimizer imitates the dynamic foraging behaviour of southern flying squirrels and their efficient way of locomotion known as gliding. Gliding is an effective mechanism used by small mammals for travelling long distances. The present work mathematically models this behaviour to realize the process of optimization. The efficiency of the proposed SSA is evaluated using statistical analysis, convergence rate analysis, Wilcoxon's test and ANOVA on classical as well as modern CEC 2014 benchmark functions. An extensive comparative study is carried out to exhibit the effectiveness of SSA over other well-known optimizers in terms of optimization accuracy and convergence rate. The proposed algorithm is implemented on a real-time Heat Flow Experiment to check its applicability and robustness. The results demonstrate that SSA provides more accurate solutions with high convergence rate as compared to other existing optimizers.

A novel nature-inspired algorithm for optimization: Squirrel search algorithm

In recent years, the research community has witnessed an explosion of literature dealing with the mimicking of behavioral patterns and social phenomena observed in nature towards efficiently solving complex computational tasks. This trend has been especially dramatic in what relates to optimization problems, mainly due to the unprecedented complexity of problem instances, arising from a diverse spectrum of domains such as transportation, logistics, energy, climate, social networks, health and industry 4.0, among many others. Notwithstanding this upsurge of activity, research in this vibrant topic should be steered towards certain areas that, despite their eventual value and impact on the field of bio-inspired computation, still remain insufficiently explored to date. The main purpose of this paper is to outline the state of the art and to identify open challenges concerning the most relevant areas within bio-inspired optimization. An analysis and discussion are also carried out over the general trajectory followed in recent years by the community working in this field, thereby highlighting the need for reaching a consensus and joining forces towards achieving valuable insights into the understanding of this family of optimization techniques.

Bio-inspired computation: Where we stand and what's next

Whale Optimization Algorithm (WOA) is an optimization algorithm developed by Mirjalili and Lewis in 2016. An overview of WOA is described in this paper, rooted from the bubble-net hunting strategy, besides an overview of WOA applications that are used to solve optimization problems in various categories. The best solution has been determined to make something as functional and effective as possible through the optimization process by minimizing or maximizing the parameters involved in the problems. Research and engineering attention have been paid to Meta-heuristics for purposes of decision-making given the growing complexity of models and the needs for quick decision making in the engineering. An updated review of research of WOA is provided in this paper for hybridization, improved, and variants. The categories included in the reviews are Engineering, Clustering, Classification, Robot Path, Image Processing, Networks, Task Scheduling, and other engineering applications. According to the reviewed literature, WOA is mostly used in the engineering area to solve optimization problems. Providing an overview and summarizing the review of WOA applications are the aims of this paper.

A comprehensive survey: Whale Optimization Algorithm and its applications

Class imbalance is a long-standing problem relevant to a number of real-world applications of deep learning. Oversampling techniques, which are effective for handling class imbalance in classical learning systems, can not be directly applied to end-to-end deep learning systems. We propose a three-player adversarial game between a convex generator, a multi-class classifier network, and a real/fake discriminator to perform oversampling in deep learning systems. The convex generator generates new samples from the minority classes as convex combinations of existing instances, aiming to fool both the discriminator as well as the classifier into misclassifying the generated samples. Consequently, the artificial samples are generated at critical locations near the peripheries of the classes. This, in turn, adjusts the classifier induced boundaries in a way which is more likely to reduce misclassification from the minority classes. Extensive experiments on multiple class imbalanced image datasets establish the efficacy of our proposal.

/pdf/generative-adversarial-minority-oversampling-3juyud5r7m.pdf

Generative Adversarial Minority Oversampling

The classification accuracy of a $k$ -nearest neighbor ( $k$ NN) classifier is largely dependent on the choice of the number of nearest neighbors denoted by $k$ . However, given a data set, it is a tedious task to optimize the performance of $k$ NN by tuning $k$ . Moreover, the performance of $k$ NN degrades in the presence of class imbalance, a situation characterized by disparate representation from different classes. We aim to address both the issues in this paper and propose a variant of $k$ NN called the Adaptive $k$ NN (Ada- $k$ NN). The Ada- $k$ NN classifier uses the density and distribution of the neighborhood of a test point and learns a suitable point-specific $k$ for it with the help of artificial neural networks. We further improve our proposal by replacing the neural network with a heuristic learning method guided by an indicator of the local density of a test point and using information about its neighboring training points. The proposed heuristic learning algorithm preserves the simplicity of $k$ NN without incurring serious computational burden. We call this method Ada- $k$ NN2. Ada- $k$ NN and Ada- $k$ NN2 perform very competitive when compared with $k$ NN, five of $k$ NN’s state-of-the-art variants, and other popular classifiers. Furthermore, we propose a class-based global weighting scheme (Global Imbalance Handling Scheme or GIHS) to compensate for the effect of class imbalance. We perform extensive experiments on a wide variety of data sets to establish the improvement shown by Ada- $k$ NN and Ada- $k$ NN2 using the proposed GIHS, when compared with $k$ NN, and its 12 variants specifically tailored for imbalanced classification.

Sankha Subhra Mullick

Papers

Recent advances in differential evolution – An updated survey

Generative Adversarial Minority Oversampling

Adaptive Learning-Based $k$ -Nearest Neighbor Classifiers With Resilience to Class Imbalance

Appropriateness of performance indices for imbalanced data classification: An analysis

A switched parameter differential evolution with optional blending crossover for scalable numerical optimization