基于MapReduce框架下K-means的改进算法

针对海量数据背景下K-means聚类结果不稳定和收敛速度较慢的问题,提出了基于MapReduce框架下的K-means改进算法。首先,为了能获得K-means聚类的初始簇数,利用凝聚层次聚类法对数据集进行聚类,并用轮廓系数对聚类结果进行初步评价,将获得数据集的簇数作为K-means算法的初始簇中心进行聚类；其次,为了能适应于海量数据的聚类挖掘,将改进的K-means算法部署在MapReduce框架上进行运算。实验结果表明,在单机性能上,该方法具有较高的准确率和召回率,同时也具有较强的聚类稳定性；在集群性能上,也具有较好的加速比和运行速度。

An Improved K-means Algorithm based on MapReduce Framework

Focusing on the unstable result and slow convergence of K-means clustering algorithm for huge amount of data, an improved K-means algorithm based on MapReduce framework was proposed. Firstly, in order to obtain the initial cluster number of K-means clustering, we use hierarchical clustering method to cluster the dataset, and evaluate the clustering result by silhouette coefficient. The cluster number of the acquired data set is clustered as the initial cluster center of the K-means algorithm. Secondly, in order to adapt to the clustering mining of massive data, we use the modified K-means algorithm to deploy in the MapReduce framework. The experimental results show that the proposed method has high precision and recall rate and strong clustering stability in single machine performance, and also has better speedup ratio and running speed in clustering performance.