This paper presents a new form of Genetic Programming called Cartesian Genetic Programming in which a program is represented as an indexed graph. The graph is encoded in the form of a linear string of integers. The inputs or terminal set and node outputs are numbered sequentially. The node functions are also separately numbered. The genotype is just a list of node connections and functions. The genotype is then mapped to an indexed graph that can be executed as a program. Evolutionary algorithms are used to evolve the genotype in a symbolic regression problem (sixth order polynomial) and the Santa Fe Ant Trail. The computational effort is calculated for both cases. It is suggested that hit effort is a more reliable measure of computational efficiency. A neutral search strategy that allows the fittest genotype to be replaced by another equally fit genotype (a neutral genotype) is examined and compared with non-neutral search for the Santa Fe ant problem. The neutral search proves to be much more effective.

Cartesian Genetic Programming

We present grammatical evolution, an evolutionary algorithm that can evolve complete programs in an arbitrary language using a variable-length binary string. The binary genome determines which production rules in a Backus-Naur form grammar definition are used in a genotype-to-phenotype mapping process to a program. We demonstrate how expressions and programs of arbitrary complexity may be evolved and compare its performance to genetic programming.

Grammatical evolution

Two major goals in machine learning are the discovery and improvement of solutions to complex problems. In this paper, we argue that complexification, i.e. the incremental elaboration of solutions through adding new structure, achieves both these goals. We demonstrate the power of complexification through the NeuroEvolution of Augmenting Topologies (NEAT) method, which evolves increasingly complex neural network architectures. NEAT is applied to an open-ended coevolutionary robot duel domain where robot controllers compete head to head. Because the robot duel domain supports a wide range of strategies, and because coevolution benefits from an escalating arms race, it serves as a suitable testbed for studying complexification. When compared to the evolution of networks with fixed structure, complexifying evolution discovers significantly more sophisticated strategies. The results suggest that in order to discover and improve complex solutions, evolution, and search in general, should be allowed to complexify as well as optimize.

/pdf/competitive-coevolution-through-evolutionary-3ymms8k7o8.pdf

Competitive coevolution through evolutionary complexification

In a previous work it was argued that by studying evolved designs of gradually increasing scale, one might be able to discern new, efficient, and generalisable principles of design. These ideas are tested in the context of designing digital circuits, particularly arithmetic circuits. This process of discovery is seen as a principle extraction loop in which the evolved data is analysed both phenotypically and genotypically by processes of data mining and landscape analysis. The information extracted is then fed back into the evolutionary algorithm to enhance its search capabilities and hence increase the likelihood of identifying new principles which explain how to build systems which are too large to evolve.

Principles in the Evolutionary Design of Digital Circuits—Part II

In CGP, programs are represented in the form of directed acyclic graphs. These graphs are represented as a two-dimensional grid of computational nodes. The genes that make up the genotype in CGP are integers that represent where a node gets its data, what operations the node performs on the data and where the output data required by the user is to be obtained. When the genotype is decoded, some nodes may be ignored. This happens when node outputs are not used in the calculation of output data. When this happens, we refer to the nodes and their genes as ‘non-coding’. We call the program that results from the decoding of a genotype a phenotype. The genotype in CGP has a fixed length. However, the size of the phenotype (in terms of the number of computational nodes) can be anything from zero nodes to the number of nodes defined in the genotype. A phenotype would have zero nodes if all the program outputs were directly connected to program inputs. A phenotype would have the same number of nodes as defined in the genotype when every node in the graph was required. The genotype–phenotype mapping used in CGP is one of its defining characteristics. The types of computational node functions used in CGP are decided by the user and are listed in a function look-up table. In CGP, each node in the directed graph

Cartesian Genetic Programming.

Various techniques for statistical analysis of the structure of fitness landscapes have been proposed. An important feature of these techniques is that they study the ruggedness of landscapes by measuring their correlation characteristics. This paper proposes a new information analysis of fitness landscapes. The underlying idea is to consider a fitness landscape as an ensemble of objects that are related to the fitness of neighboring points. Three information characteristics of the ensemble are defined and studied. They are termed: information content, partial information content, and information stability. The information characteristics of a range of landscapes with known correlation features are analyzed in an attempt to reveal the advantages of the information analysis. We show that the proposed analysis is an appropriate tool for investigating the structure of fitness landscapes.

Information Characteristics and the Structure of Landscapes

The paper studies the role of neutrality in the fitness landscapes associated with the evolutionary design of digital circuits and particularly the three-bit binary multiplier. For the purpose of the study, digital circuits are evolved extrinsically on an array of logic cells. To evolve on an array of cells, a genotype-phenotype mapping has been devised by which neutrality can be embedded in the resulting fitness landscape. It is argued that landscape neutrality is beneficial for digital circuit evolution.

The Advantages of Landscape Neutrality in Digital Circuit Evolution

This paper introduces a new methodology of evolving electronic circuits by which the process of evolutionary design is guaranteed to produce a functionally correct solution. The method employs a mapping to represent an electronic circuit on an array of logic cells that is further encoded within a genotype. The mapping is many-to-one and thus there are many genotypes that have equal fitness values. Genotypes with equal fitness values define subgraphs in the resulting fitness landscapes referred to as neutral networks. This is further used in the design of a neutral network that connects the conventional with other more efficient designs. To explore such a network a navigation strategy is defined by which the space of all functionally correct circuits can be explored. The paper shows that very efficient digital circuits can be obtained by evolving from the conventional designs. Results for several binary multiplier circuits such as the three and four-bit multipliers are reported. The evolved solution for the three-bit multiplier consists of 23 two-input logic gates that in terms of number of two-input gates used is 23.3% more efficient than the most efficient known conventional design. The logic operators required to implement this circuit are 14 ANDs, 9 XORs, and 2 inversions (NOT). The evolved four-bit multiplier consists of 57 two-input logic gates that is 10.9% more efficient (in terms of number of two-input gates used) than the most efficient known conventional design. The optimal size of the target circuits is also studied by measuring the length of the neutral walks from the obtained designs.

Towards the automatic design of more efficient digital circuits

The theory of fitness landscapes has been developed to provide a suitable mathematical framework for studying the evolvability of a variety of complex systems In evolutionary computation the notion of evolvability refers to the efficiency of evolutionary search It has been shown that the structure of a fitness landscape affects the ability of evolutionary algorithms to search Three characteristics specify the structure of landscapes These are the landscape smoothness, ruggedness and neutrality The interplay of these characteristics plays a vital role in evolutionary search This has motivated the appearance of a variety of techniques for studying the structure of fitness landscapes An important feature of these techniques is that they characterize the landscapes by their smoothness and ruggedness, ignoring the existence of neutrality Perhaps, the reason for this is that the role of neutrality in evolutionary search is still poorly understoodIn this chapter some recent results on the spectral properties of the algebraic structures of fitness landscapes are summarized to provide a basis for studying the landscape structure This approach is further employed to introduce an information analysis that characterizes the structure of fitness landscapes in terms of their smoothness, ruggedness and neutrality The findings are finally applied in a study of the fitness landscapes generated by evolving digital circuits using an idealized model of a field-programmable gate array The landscapes of this engineering problem are quite different from many recently studied landscapes that tend to be defined over simplified combinatorial and optimization problems The difference originates from the genotype representation that is a configuration defined over two completely different alphabets This makes the study of the corresponding landscapes much more involved It is shown that the circuit evolution landscapes are products of subspaces with different characteristics They are landscapes with vast neutrality and sharply differentiated plateau

Vesselin K. Vassilev

Papers

Principles in the Evolutionary Design of Digital Circuits—Part II

Information Characteristics and the Structure of Landscapes

The Advantages of Landscape Neutrality in Digital Circuit Evolution

Towards the automatic design of more efficient digital circuits

Smoothness, ruggedness and neutrality of fitness landscapes: from theory to application