Data Mining - Concepts and Techniques.

Reference IMAC-CONF-2008-007Afficher la publication dans Web of Science Notice creee le 2008-03-06, modifiee le 2016-12-23

A Bounded Index for Cluster Validity

A cluster validity index is used to select which clustering algorithm to apply for a given problem. It works by evaluating the quality of a partition, as output by a candidate clustering algorithm, getting around the common case of the lack of an expert in the given domain of discourse. Most existing validity indexes make assumptions, such as each cluster of the partition having an underlying structure, for example, a hypersphere, yielding incorrect evaluations when they do not hold. Here, we propose a new cluster validity index, which attempts to avoid this bias using an ensemble of distinct supervised classifiers; this way the bias is not attributable to a specific classifier, but to a collection thereof, hence alleviating the problem. The rationale behind our index is that a good partition should induce the construction of also a good classifier; the better the classification performance, the better the quality of the partition under evaluation. Notice how we use the partition to be assessed as a sort of labeled dataset, where each object is labeled with the cluster label it belongs to. We have tested our index on 50 numerical datasets, grouped using six different clustering algorithms. In our experiments, our index outperforms five validity indexes, including the most popular ones.

Cluster validation using an ensemble of supervised classifiers

https://iopscience.iop.org/article/10.1088/1742-6596/1235/1/012015/pdf

Enhancement Clustering Evaluation Result of Davies-Bouldin Index with Determining Initial Centroid of K-Means Algorithm

External clustering validity index based on chi-squared statistical test

Purpose – The most commonly used approaches for cluster validation are based on indices but the majority of the existing cluster validity indices do not work well on data sets of different complexities. The purpose of this paper is to propose a new cluster validity index (ARSD index) that works well on all types of data sets. Design/methodology/approach – The authors introduce a new compactness measure that depicts the typical behaviour of a cluster where more points are located around the centre and lesser points towards the outer edge of the cluster. A novel penalty function is proposed for determining the distinctness measure of clusters. Random linear search-algorithm is employed to evaluate and compare the performance of the five commonly known validity indices and the proposed validity index. The values of the six indices are computed for all nc ranging from (nc min, nc max) to obtain the optimal number of clusters present in a data set. The data sets used in the experiments include shaped, Gaussian...

A new cluster validity index using maximum cluster spread based compactness measure

Elucidating the patterns hidden in gene expression data offers an opportunity for identifying co-expressed genes and biologically relevant grouping of genes. However, the large number of genes and the complexity of biological networks greatly increase the challenges of comprehending and interpreting the microarray data. A first step toward addressing this challenge is the use of clustering techniques. Validation of results obtained from a clustering algorithm is an important part of the clustering process. In this paper, we propose a new cluster validity index (ARPoints index) for the purpose of cluster validation. A new approach to determine the compactness measure and distinctness measure of clusters is presented. We revisit commonly known indices and conduct a thorough comparison of these indices with the proposed index and provide a summary of performance evaluation of different indices. Experimental results show that the proposed index performs better than the commonly known cluster validity indices.

A novel point density based validity index for clustering gene expression datasets

Most of the clustering algorithms are sensitive to the input parameters and produce different clustering results for different input parameters for same datasets. A number of methods and indices have been proposed for validating results of a clustering process. The most commonly used approaches for cluster validation are based on internal indices. In this paper, we propose a new cluster validity index (ARSpread index) for the purpose of cluster validation and determining number of clusters present in a dataset. Local and global data spread based approach is proposed to measure the compactness of a cluster. A distinctness measure that is based on a penalty function is incorporated in the proposed index. We conduct a thorough comparison of five commonly known indices with the proposed index and provide a summary of experimental performance of different indices. Experimental results show that the proposed new index performs better than the commonly known indices.

Local and Global Data Spread Based Index for Determining Number of Clusters in a Dataset

One of the challenges with the wind power renewable energy source lies in integrating this intermittent power source into the electricity grid. Prediction of wind speeds is essential for forecasting the periods when intermittent wind power is generated. Several meteorological parameters like wind direction, air temperature, atmospheric pressure, relative humidity can be used to predict wind speeds. We propose a cluster based approach for extracting patterns in meteorological data that aid in predicting wind speed. Clustering is done by using compactness measure and distinctness measure as discussed in the paper. This approach automatically determines number of clusters present in meteorological data. Patterns are mined from the extracted clusters with the aid of ID3 algorithm. The mined patterns are used to predict the wind speed patterns in a usable and comprehensive manner. Extensive experimentation using 3100 data points indicate that the proposed approach produces good results.

Cluster based approach for mining patterns to predict wind speed

This paper has been developed to draw a comparison between various available approaches for dialogue management. Currently there is much interest in building interactive human-computer interfaces which involve spoken input and output. Spoken dialogue system usually combines speech recognition with natural language understanding, language generation and dialogue management. Dialog systems are created for domain specific applications, so that a high demand for a flexible dialog system framework arises. There have been several approaches to dialog management. In this paper we present three different approaches to the dialog management.

Romana Riyaz

Papers

A new cluster validity index using maximum cluster spread based compactness measure

A novel point density based validity index for clustering gene expression datasets

Local and Global Data Spread Based Index for Determining Number of Clusters in a Dataset

Cluster based approach for mining patterns to predict wind speed

A comparative study of various approaches for dialogue management