A mixture of experts committee machine to design compensators for intensity modulated radiation therapy

This paper presents a new algorithm to produce a near optimal mixture of experts model (MEM) architecture for a continuous mapping. The MEM is applied to a new method incorporating photon scatter for designing compensators for intensity modulated radiation therapy. The algorithm utilizes the fuzzy C-means clustering algorithm to partition data before training commences. A reduction in the size of training sets also allows the Levenberg-Marquardt algorithm to be implemented. As a result, both training time and validation error are reduced. A 71% reduction in prediction error compared with that of a single neural network is achieved.     

A cluster validity index for fuzzy clustering

A new cluster validity index is proposed for the validation of partitions of object data produced by the fuzzy c-means algorithm. The proposed validity index uses a variation measure and a separation measure between two fuzzy clusters. A good fuzzy partition is expected to have a low degree of variation and a large separation distance. Testing of the proposed index and nine previously formulated indices on well-known data sets shows the superior effectiveness and reliability of the proposed index in comparison to other indices and the robustness of the proposed index in noisy environments.     

Evaluation of the performance of clustering algorithms in kernel-induced feature space

By using a kernel function, data that are not easily separable in the original space can be clustered into homogeneous groups in the implicitly transformed high-dimensional feature space. Kernel k-means algorithms have recently been shown to perform better than conventional k-means algorithms in unsupervised classification. However, few reports have examined the benefits of using a kernel function and the relative merits of the various kernel clustering algorithms with regard to the data distribution. In this study, we reformulated four representative clustering algorithms based on a kernel function and evaluated their performances for various data sets. The results indicate that each kernel clustering algorithm gives markedly better performance than its conventional counterpart for almost all data sets. Of the kernel clustering algorithms studied in the present work, the kernel average linkage algorithm gives the most accurate clustering results.     

Mountain c-regressions method

Since Quandt [The estimation of the parameters of a linear regression system obeying two separate regimes, Journal of the American Statistical Association 53 (1958) 873-880] initiated the research on 2-regressions analysis, switching regression had been widely studied and applied in psychology, economics, social science and music perception. In fuzzy clustering, the fuzzy c-means (FCM) is the most commonly used algorithm. Hathaway and Bezdek [Switching regression models and fuzzy clustering, IEEE Transactions on Fuzzy Systems 1 (1993) 195-204] embedded FCM into switching regression where it was called fuzzy c-regressions (FCR). However, the FCR always depends heavily on initial values. In this paper, we propose a mountain c-regressions (MCR) method for solving the initial-value problem. First, we perform data transformation for the switching regression data set, and then implement the modified mountain clustering on the transformed data to extract c cluster centers. These extracted c cluster centers in the transformed space will correspond to c regression models in the original data set. The proposed MCR method can form well-estimated c regression models for switching regression data sets. According to the properties of transformation, the proposed MCR is also robust to noise and outliers. Several examples show the effectiveness and superiority of our proposed method.     

On cluster validity index for estimation of the optimal number of fuzzy clusters

A new cluster validity index is proposed that determines the optimal partition and optimal number of clusters for fuzzy partitions obtained from the fuzzy c-means algorithm. The proposed validity index exploits an overlap measure and a separation measure between clusters. The overlap measure, which indicates the degree of overlap between fuzzy clusters, is obtained by computing an inter-cluster overlap. The separation measure, which indicates the isolation distance between fuzzy clusters, is obtained by computing a distance between fuzzy clusters. A good fuzzy partition is expected to have a low degree of overlap and a larger separation distance. Testing of the proposed index and nine previously formulated indexes on well-known data sets showed the superior effectiveness and reliability of the proposed index in comparison to other indexes.     

Fast global k-means clustering using cluster membership and inequality

In this paper, we present a fast global k-means clustering algorithm by making use of the cluster membership and geometrical information of a data point. This algorithm is referred to as MFGKM. The algorithm uses a set of inequalities developed in this paper to determine a starting point for the jth cluster center of global k-means clustering. Adopting multiple cluster center selection (MCS) for MFGKM, we also develop another clustering algorithm called MFGKM+MCS. MCS determines more than one starting point for each step of cluster split; while the available fast and modified global k-means clustering algorithms select one starting point for each cluster split. Our proposed method MFGKM can obtain the least distortion; while MFGKM+MCS may give the least computing time. Compared to the modified global k-means clustering algorithm, our method MFGKM can reduce the computing time and number of distance calculations by a factor of 3.78-5.55 and 21.13-31.41, respectively, with the average distortion reduction of 5,487 for the Statlog data set. Compared to the fast global k-means clustering algorithm, our method MFGKM+MCS can reduce the computing time by a factor of 5.78-8.70 with the average reduction of distortion of 30,564 using the same data set. The performances of our proposed methods are more remarkable when a data set with higher dimension is divided into more clusters.     

Enhanced neural gas network for prototype-based clustering

In practical cluster analysis tasks, an efficient clustering algorithm should be less sensitive to parameter configurations and tolerate the existence of outliers. Based on the neural gas (NG) network framework, we propose an efficient prototype-based clustering (PBC) algorithm called enhanced neural gas (ENG) network. Several problems associated with the traditional PBC algorithms and original NG algorithm such as sensitivity to initialization, sensitivity to input sequence ordering and the adverse influence from outliers can be effectively tackled in our new scheme. In addition, our new algorithm can establish the topology relationships among the prototypes and all topology-wise badly located prototypes can be relocated to represent more meaningful regions. Experimental results¹on synthetic and UCI datasets show that our algorithm possesses superior performance in comparison to several PBC algorithms and their improved variants, such as hard c-means, fuzzy c-means, NG, fuzzy possibilistic c-means, credibilistic fuzzy c-means, hard/fuzzy robust clustering and alternative hard/fuzzy c-means, in static data clustering tasks with a fixed number of prototypes.     

Comparing the performance of traditional cluster analysis, self-organizing maps and fuzzy C-means method for strategic grouping

Strategic group analysis comprises of clustering of firms within an industry according to their similarities with respect to a set of strategic dimensions and investigating the performance implications of strategic group membership. One of the challenges of strategic group analysis is the selection of the clustering method. In this study, the results of the strategic group analysis of Turkish contractors are presented to compare the performances of traditional cluster analysis techniques, self-organizing maps (SOM) and fuzzy C-means method (FCM) for strategic grouping. Findings reveal that traditional cluster analysis methods cannot disclose the overlapping strategic group structure and position of companies within the same strategic group. It is concluded that SOM and FCM can reveal the typology of the strategic groups better than traditional cluster analysis and they are more likely to provide useful information about the real strategic group structure.     

Sample-weighted clustering methods

Although there have been many researches on cluster analysis considering feature (or variable) weights, little effort has been made regarding sample weights in clustering. In practice, not every sample in a data set has the same importance in cluster analysis. Therefore, it is interesting to obtain the proper sample weights for clustering a data set. In this paper, we consider a probability distribution over a data set to represent its sample weights. We then apply the maximum entropy principle to automatically compute these sample weights for clustering. Such method can generate the sample-weighted versions of most clustering algorithms, such as k-means, fuzzy c-means (FCM) and expectation & maximization (EM), etc. The proposed sample-weighted clustering algorithms will be robust for data sets with noise and outliers. Furthermore, we also analyze the convergence properties of the proposed algorithms. This study also uses some numerical data and real data sets for demonstration and comparison. Experimental results and comparisons actually demonstrate that the proposed sample-weighted clustering algorithms are effective and robust clustering methods.     

Fast modified global k-means algorithm for incremental cluster construction

The k-means algorithm and its variations are known to be fast clustering algorithms. However, they are sensitive to the choice of starting points and are inefficient for solving clustering problems in large datasets. Recently, incremental approaches have been developed to resolve difficulties with the choice of starting points. The global k-means and the modified global k-means algorithms are based on such an approach. They iteratively add one cluster center at a time. Numerical experiments show that these algorithms considerably improve the k-means algorithm. However, they require storing the whole affinity matrix or computing this matrix at each iteration. This makes both algorithms time consuming and memory demanding for clustering even moderately large datasets. In this paper, a new version of the modified global k-means algorithm is proposed. We introduce an auxiliary cluster function to generate a set of starting points lying in different parts of the dataset. We exploit information gathered in previous iterations of the incremental algorithm to eliminate the need of computing or storing the whole affinity matrix and thereby to reduce computational effort and memory usage. Results of numerical experiments on six standard datasets demonstrate that the new algorithm is more efficient than the global and the modified global k-means algorithms.     

Unsupervised possibilistic clustering

In fuzzy clustering, the fuzzy c-means (FCM) clustering algorithm is the best known and used method. Since the FCM memberships do not always explain the degrees of belonging for the data well, Krishnapuram and Keller proposed a possibilistic approach to clustering to correct this weakness of FCM. However, the performance of Krishnapuram and Keller's approach depends heavily on the parameters. In this paper, we propose another possibilistic clustering algorithm (PCA) which is based on the FCM objective function, the partition coefficient (PC) and partition entropy (PE) validity indexes. The resulting membership becomes the exponential function, so that it is robust to noise and outliers. The parameters in PCA can be easily handled. Also, the PCA objective function can be considered as a potential function, or a mountain function, so that the prototypes of PCA can be correspondent to the peaks of the estimated function. To validate the clustering results obtained through a PCA, we generalized the validity indexes of FCM. This generalization makes each validity index workable in both fuzzy and possibilistic clustering models. By combining these generalized validity indexes, an unsupervised possibilistic clustering is proposed. Some numerical examples and real data implementation on the basis of the proposed PCA and generalized validity indexes show their effectiveness and accuracy.     

Further reduced form of wavelet feature for text independent speaker recognition

A text independent speaker recognition system based on wavelet transform derived from fuzzy c-means clustering is proposed. The fuzzy c-means clustering is applied to the speaker data compression in spectrum domain. A set of experiments are conducted, which gives a 95% recognition rate for 100 Mandarin speakers.     

A fuzzy clustering algorithm based on evolutionary programming

In this paper, a fuzzy clustering method based on evolutionary programming (EPFCM) is proposed. The algorithm benefits from the global search strategy of evolutionary programming, to improve fuzzy c-means algorithm (FCM). The cluster validity can be measured by some cluster validity indices. To increase the convergence speed of the algorithm, we exploit the modified algorithm to change the number of cluster centers dynamically. Experiments demonstrate EPFCM can find the proper number of clusters, and the result of clustering does not depend critically on the choice of the initial cluster centers. The probability of trapping into the local optima will be very lower than FCM.     

Picture fuzzy clustering: a new computational intelligence method

Fuzzy clustering especially fuzzy \(C\)-means (FCM) is considered as a useful tool in the processes of pattern recognition and knowledge discovery from a database; thus being applied to various crucial, socioeconomic applications. Nevertheless, the clustering quality of FCM is not high since this algorithm is deployed on the basis of the traditional fuzzy sets, which have some limitations in the membership representation, the determination of hesitancy and the vagueness of prototype parameters. Various improvement versions of FCM on some extensions of the traditional fuzzy sets have been proposed to tackle with those limitations. In this paper, we consider another improvement of FCM on the picture fuzzy sets, which is a generalization of the traditional fuzzy sets and the intuitionistic fuzzy sets, and present a novel picture fuzzy clustering algorithm, the so-called FC-PFS. A numerical example on the IRIS dataset is conducted to illustrate the activities of the proposed algorithm. The experimental results on various benchmark datasets of UCI Machine Learning Repository under different scenarios of parameters of the algorithm reveal that FC-PFS has better clustering quality than some relevant clustering algorithms such as FCM, IFCM, KFCM and KIFCM.     

Fast and robust fuzzy c-means clustering algorithms incorporating local information for image segmentation

Fuzzy c-means (FCM) algorithms with spatial constraints (FCM_S) have been proven effective for image segmentation. However, they still have the following disadvantages: (1) although the introduction of local spatial information to the corresponding objective functions enhances their insensitiveness to noise to some extent, they still lack enough robustness to noise and outliers, especially in absence of prior knowledge of the noise; (2) in their objective functions, there exists a crucial parameter α used to balance between robustness to noise and effectiveness of preserving the details of the image, it is selected generally through experience; and (3) the time of segmenting an image is dependent on the image size, and hence the larger the size of the image, the more the segmentation time. In this paper, by incorporating local spatial and gray information together, a novel fast and robust FCM framework for image segmentation, i.e., fast generalized fuzzy c-means (FGFCM) clustering algorithms, is proposed. FGFCM can mitigate the disadvantages of FCM_S and at the same time enhances the clustering performance. Furthermore, FGFCM not only includes many existing algorithms, such as fast FCM and enhanced FCM as its special cases, but also can derive other new algorithms such as FGFCM_S1 and FGFCM_S2 proposed in the rest of this paper. The major characteristics of FGFCM are: (1) to use a new factor S_ij as a local (both spatial and gray) similarity measure aiming to guarantee both noise-immunity and detail-preserving for image, and meanwhile remove the empirically-adjusted parameter α; (2) fast clustering or segmenting image, the segmenting time is only dependent on the number of the gray-levels q rather than the size N(?q) of the image, and consequently its computational complexity is reduced from O(NcI₁) to O(qcI₂), where c is the number of the clusters, I₁ and are the numbers of iterations, respectively, in the standard FCM and our proposed fast segmentation method. The experiments on the synthetic and real-world images show that FGFCM algorithm is effective and efficient.     

Some connectivity based cluster validity indices

Identification of the correct number of clusters and the appropriate partitioning technique are some important considerations in clustering where several cluster validity indices, primarily utilizing the Euclidean distance, have been used in the literature. In this paper a new measure of connectivity is incorporated in the definitions of seven cluster validity indices namely, DB-index, Dunn-index, Generalized Dunn-index, PS-index, I-index, XB-index and SV-index, thereby yielding seven new cluster validity indices which are able to automatically detect clusters of any shape, size or convexity as long as they are well-separated. Here connectivity is measured using a novel approach following the concept of relative neighborhood graph. It is empirically established that incorporation of the property of connectivity significantly improves the capabilities of these indices in identifying the appropriate number of clusters. The well-known clustering techniques, single linkage clustering technique and K-means clustering technique are used as the underlying partitioning algorithms. Results on eight artificially generated and three real-life data sets show that connectivity based Dunn-index performs the best as compared to all the other six indices. Comparisons are made with the original versions of these seven cluster validity indices.     

Fuzzy clustering algorithms for unsupervised change detection in remote sensing images

In this paper, we propose a context-sensitive technique for unsupervised change detection in multitemporal remote sensing images. The technique is based on fuzzy clustering approach and takes care of spatial correlation between neighboring pixels of the difference image produced by comparing two images acquired on the same geographical area at different times. Since the ranges of pixel values of the difference image belonging to the two clusters (changed and unchanged) generally have overlap, fuzzy clustering techniques seem to be an appropriate and realistic choice to identify them (as we already know from pattern recognition literatures that fuzzy set can handle this type of situation very well). Two fuzzy clustering algorithms, namely fuzzy c-means (FCM) and Gustafson-Kessel clustering (GKC) algorithms have been used for this task in the proposed work. For clustering purpose various image features are extracted using the neighborhood information of pixels. Hybridization of FCM and GKC with two other optimization techniques, genetic algorithm (GA) and simulated annealing (SA), is made to further enhance the performance. To show the effectiveness of the proposed technique, experiments are conducted on two multispectral and multitemporal remote sensing images. A fuzzy cluster validity index (Xie-Beni) is used to quantitatively evaluate the performance. Results are compared with those of existing Markov random field (MRF) and neural network based algorithms and found to be superior. The proposed technique is less time consuming and unlike MRF does not require any a priori knowledge of distributions of changed and unchanged pixels.     

An interval number distance- and ranking-based method for remotely sensed image fuzzy clustering

ABSTRACT

Fuzzy c-means clustering is an important non-supervised classification method for remote-sensing images and is based on type-1 fuzzy set theory. Type-1 fuzzy sets use singleton values to express the membership grade; therefore, such sets cannot describe the uncertainty of the membership grade. Interval type-2 fuzzy c-means (IT2FCM) clustering and relevant methods are based on interval type-2 fuzzy sets. Real vectors are used to describe the clustering centres, and the average values of the upper and lower membership grades are used to determine the classification of each pixel. Thus, the width information for interval clustering centres and interval membership grades are ignored. The main contribution of this article is to propose an improved IT2FCM* algorithm by adopting interval number distance (IND) and ranking methods, which use the width information of interval clustering centres and interval membership grades, thus distinguishing this method from existing fuzzy clustering methods. Three different IND definitions are tested, and the distance definition proposed by Li shows the best performance. The second contribution of this work is that two fuzzy cluster validity indices, FS- and XB-, are improved using the IND. Three types of multi/hyperspectral remote-sensing data sets are used to test this algorithm, and the experimental results show that the IT2FCM* algorithm based on the IND proposed by Li performs better than the IT2FCM algorithm using four cluster validity indices, the confusion matrix, and the kappa coefficient (κ). Additionally, the improved FS- index has more indicative ability than the original FS- index.     

Adapting k-means for supervised clustering

k-means is traditionally viewed as an algorithm for the unsupervised clustering of a heterogeneous population into a number of more homogeneous groups of objects. However, it is not necessarily guaranteed to group the same types (classes) of objects together. In such cases, some supervision is needed to partition objects which have the same label into one cluster. This paper demonstrates how the popular k-means clustering algorithm can be profitably modified to be used as a classifier algorithm. The output field itself cannot be used in the clustering but it is used in developing a suitable metric defined on other fields. The proposed algorithm combines Simulated Annealing with the modified k-means algorithm. We apply the proposed algorithm to real data sets, and compare the output of the resultant classifier to that of C4.5.