FDBSCAN:一种快速 DBSCAN算法

聚类分析是一门重要的技术 ,在数据挖掘、统计数据分析、模式匹配和图象处理等领域具有广泛的应用前景 .目前 ,人们已经提出了许多聚类算法 .其中 ,DBSCAN是一种性能优越的基于密度的空间聚类算法 .利用基于密度的聚类概念 ,用户只需输入一个参数 ,DBSCAN算法就能够发现任意形状的类 ,并可以有效地处理噪声 .文章提出了一种加快 DBSCAN算法的方法 .新算法以核心对象邻域中所有对象的代表对象为种子对象来扩展类 ,从而减少区域查询次数 ,降低 I/ O开销 .实验结果表明 ,FDBSCAN能够有效地     

一种改进的基于密度的抽样聚类算法

基于密度的聚类算法DBSCAN是一种有效的空间聚类算法,它能够发现任意形状的聚类并且有效地处理噪声。然而,DBSCAN算法也有一些缺点,例如,①在聚类时只考虑空间属性没有考虑非空间属性;②在对大规模空间数据库进行聚类分析时需要较大的内存支持和I/O消耗。为此,在分析DBSCAN算法不足的基础上,提出了一种改进的基于密度的抽样聚类(improved density-based spatial clustering algorithm with sampling,IDBSCAS)算法,使之能够有效地处理大规模空间数据库,并且它不仅考虑了空间属性也考虑了非空间属性。2维空间数据的测试结果表明,该算法是可行、有效的。     

基于数据分区的DBSCAN算法

数据聚类在数据挖掘、模式识别、图像处理和数据压缩等领域有着广泛的应用。ＤＢＳＣＡＮ是一种基于密度的空间聚类算法,在处理空间数据时具有快速、有效处理噪声点和发现任意形状的聚类等优点,但由于直接对数据库进行操作,在数据量大的时间就需要较多的内存和Ｉ／Ｏ开销;此外,当数据密度和聚类间的距离不均匀时聚类质量较差,为此,在分析ＤＢＳＣＡＮ算法不足的基础上,提出了一个基于数据分区的ＤＢＳＣＡＮ算法,测试结果表     

基于Web-Log Mining的Web文档聚类

速度和效果是聚类算法面临的两大问题.DBSCAN(density based spatial clustering of applications with noise)是典型的基于密度的一种聚类方法,对于大型数据库的聚类实验显示了它在速度上的优越性.提出了一种基于密度的递归聚类算法(recursive density based clustering algorithm,简称RDBC),此算法可以智能地、动态地修改其密度参数.RDBC是基于DBSCAN的一种改进算法,其运算复杂度和DBSCAN相同.通过在Web文档上的聚类实验,结果表明,RDBC不但保留了DBSCAN高速度的优点,而且聚类效果大大优于DBSCAN.     

一种基于密度的快速聚类算法

聚类是数据挖掘领域中的一个重要研究方向,聚类技术在统计数据分析、模式识别、图像处理等领域有广泛应用,迄今为止人们提出了许多用于大规模数据库的聚类算法。基于密度的聚类算法DBSCAN就是一个典型代表。以DBSCAN为基础,提出了一种基于密度的快速聚类算法。新算法以核心对象领域中所有对象的代表对象为种子对象来扩展类,从而减少区域查询次数,降低I/O开销,实现快速聚类,对二维空间数据测试表明：快速算法能够有效地对大规模数据库进行聚类,速度上数倍于已有DBSCAN算法。     

A new hybrid method based on partitioning-based DBSCAN and ant clustering

Clustering problem is an unsupervised learning problem. It is a procedure that partition data objects into matching clusters. The data objects in the same cluster are quite similar to each other and dissimilar in the other clusters. Density-based clustering algorithms find clusters based on density of data points in a region. DBSCAN algorithm is one of the density-based clustering algorithms. It can discover clusters with arbitrary shapes and only requires two input parameters. DBSCAN has been proved to be very effective for analyzing large and complex spatial databases. However, DBSCAN needs large volume of memory support and often has difficulties with high-dimensional data and clusters of very different densities. So, partitioning-based DBSCAN algorithm (PDBSCAN) was proposed to solve these problems. But PDBSCAN will get poor result when the density of data is non-uniform. Meanwhile, to some extent, DBSCAN and PDBSCAN are both sensitive to the initial parameters. In this paper, we propose a new hybrid algorithm based on PDBSCAN. We use modified ant clustering algorithm (ACA) and design a new partitioning algorithm based on ‘point density’ (PD) in data preprocessing phase. We name the new hybrid algorithm PACA-DBSCAN. The performance of PACA-DBSCAN is compared with DBSCAN and PDBSCAN on five data sets. Experimental results indicate the superiority of PACA-DBSCAN algorithm.     

基于数据交叠分区的并行DBSCAN算法*

DBSCAN是基于密度的聚类算法的一个典型代表,它对空间数据库聚类有很好的性能。然而,在对大规模数据库聚类时,DBSCAN需要大量内存支持并伴随着I/O开销。随着高性能计算机的发展,特别是集群式计算机的出现,提供了一种解决DBSCAN算法缺陷的方法。测试表明,它极大地降低了DBSCAN对时间和空间的需要。     

基于数据分区的并行DBSCAN算法

DBSCAN是基于密度的聚类算法的一个典型代表算法，它对空间数据库聚类有很好的性能．然而，在对大规模数据库聚类时，DBSCAN需要大量内存支持并伴随着I／O开销．随着高性能计算机的发展，特别是集群式计算机出现，给我们提供了一种解决DBSCAN算法缺陷的方法，本文提出一种建立在集群式高性能计算机上基于数据分区并行DBSCAN算法．测试结果表明，它极大地降低了DBSCAN对时间和空间的需要．     

Clustering in very large databases based on distance and density

Clustering in vergy large databases or data warehouses,with many applications in areas such as spatial computation,web information coollection,pattern recognition and econmic analysis,is a huge task that challenges data mining researches.Current clustering methods always have the problems:1)scanning the whole databased leads to high I/O cost and expensive maintenance(e.g.,R^*-tree);2)pre-specifying the uncertain parameter k,with which clustering can only be refined by trial and test many times;3) lacking high efficiency in treating arbitrary shape under very large data set environment.In this paper,we first present a new hybrid-clustering algorithm to solve these problesm,This new algorithm,which combines both distance and density strategies,can handle any arbitrary shape clusters effectively.It makes full use of statistics information in mining to reduce the time complexity greatly while keeping good clustering quality.Furthermore,this algorithm can easily eliminate noises and inentify outliers.An experimental evaluation is performed on a spatial database with this method and other popular clustering algorithms(CURE and DBSCAN).The results show that our algorithm outperforms them in terms of efficiency and cost,and even gets much more speedup as the data size scales up much larger.     

PHD: an efficient data clustering scheme using partition space technique for knowledge discovery in large databases

Rapid technological advances imply that the amount of data stored in databases is rising very fast. However, data mining can discover helpful implicit information in large databases. How to detect the implicit and useful information with lower time cost, high correctness, high noise filtering rate and fit for large databases is of priority concern in data mining, specifying why considerable clustering schemes have been proposed in recent decades. This investigation presents a new data clustering approach called PHD, which is an enhanced version of KIDBSCAN. PHD is a hybrid density-based algorithm, which partitions the data set by K-means, and then clusters the resulting partitions with IDBSCAN. Finally, the closest pairs of clusters are merged until the natural number of clusters of data set is reached. Experimental results reveal that the proposed algorithm can perform the entire clustering, and efficiently reduce the run-time cost. They also indicate that the proposed new clustering algorithm conducts better than several existing well-known schemes such as the K-means, DBSCAN, IDBSCAN and KIDBSCAN algorithms. Consequently, the proposed PHD algorithm is efficient and effective for data clustering in large databases.     

MR-DBSCAN: a scalable MapReduce-based DBSCAN algorithm for heavily skewed data

DBSCAN (density-based spatial clustering of applications with noise) is an important spatial clustering technique that is widely adopted in numerous applications. As the size of datasets is extremely large nowadays, parallel processing of complex data analysis such as DBSCAN becomes indispensable. However, there are three major drawbacks in the existing parallel DBSCAN algorithms. First, they fail to properly balance the load among parallel tasks, especially when data are heavily skewed. Second, the scalability of these algorithms is limited because not all the critical sub-procedures are parallelized. Third, most of them are not primarily designed for shared-nothing environments, which makes them less portable to emerging parallel processing paradigms. In this paper, we present MR-DBSCAN, a scalable DBSCAN algorithm using MapReduce. In our algorithm, all the critical sub-procedures are fully parallelized. As such, there is no performance bottleneck caused by sequential processing. Most importantly, we propose a novel data partitioning method based on computation cost estimation. The objective is to achieve desirable load balancing even in the context of heavily skewed data. Besides, We conduct our evaluation using real large datasets with up to 1.2 billion points. The experiment results well confirm the efficiency and scalability of MR-DBSCAN.     

基于数据场的改进DBSCAN聚类算法

DBSCAN(density based spatial clustering of applications with noise)算法是一种典型的基于密度的聚类算法。该算法可以识别任意形状的类簇,但聚类结果依赖于参数Eps和MinPts的选择,而且对于一些密度差别较大的数据集,可能得不到具有正确类簇个数的聚类结果,也可能将部分数据错分为噪声。为此,利用数据场能较好描述数据分布,反映数据关系的优势,提出了一种基于数据场的改进DBSCAN聚类算法。该算法引入平均势差的概念,在聚类过程中动态地确定每个类的Eps和平均势差,从而能够在一些密度相差较大的数据集上得到较好的聚类结果。实验表明,所提算法的性能优于DBSCAN算法。     

停留点空间聚类在景区热点分析中的应用

各种集成位置服务（LBS）的社交和旅游类APP的广泛应用,产生了大量轨迹空间数据,利用这些轨迹数据挖掘游客聚集密度高的热门景点区域,对景区的智慧服务和应急管理具有重要意义。为此,提出了一种基于轨迹停留点空间聚类的景区热点分析方法。重点研究了聚类速度快、能处理噪声、可以发现空间任意形状聚簇的DBSCAN算法,针对其参数需人工选择的不足,提出了一种根据数据统计分布特性来自适应确定参数的改进方法。分别采用人工合成二维数据集、四维Iris真实数据集和景区轨迹停留点三种不同的数据进行了DBSCAN聚类分析及对比实验,结果表明该方法可以自动产生合理的聚簇划分,优于传统DBSCAN和k-means等算法。最后,依据轨迹停留点的空间聚类结果,在ArcGIS软件中实现Getis-Ord Gi*热点分析与制图,并依据分析结果对不同旅游景点进行热度分级,形成的热门景点分布与景区掌握的实际热度信息基本一致,证实了提出方法的有效性。     

一种基于划分的不同参数值的DBSCAN算法

聚类是数据挖掘领域中一个重要的研究方向,DBSCAN是一种基于密度的聚类算法.该算法将具有足够高密度的区域划分成簇,并可以在带有“噪声”的空间数据库中发现任意形状的簇.分析DBSCAN算法发现存在如下问题：当数据分布不均匀时,由于使用统一的全局变量,使得聚类的效果差.针对这一缺陷,提出了一种基于数据划分的思想,并对各个局部数据集采取不同的参数值分别进行聚类,最后合并各局部聚类结果.实验结果表明,改进后的算法有效并可行.     

空间数据库的聚类方法

1 引言近年来,数据库的数量和单个数据库的容量都大大增长了。比如,空间物体数据库包括几十亿个望远镜图像,NASA地球观测系统每小时都会产生50GB的数据。这么大的数据量已经远远超出了人为分析解释的能力范围。数据库中的知识发现(KDD)是识别数据中有价值的、新的、潜在有用的、可理解的模式的一     

智能用电数据的采集与预处理

解决好智能用电网络数据采集和传输过程中的数据缺失和噪声问题,提高其用电数据的数据质量,才能在智能用电云平台中有效的运用各种用电大数据分析与预测算法。本文在总结智能用电网络的数据采集与数据传输特点,及分析智能用电云平台对用电数据的数据质量要求的基础上,提出了智能用电网络的用电数据预处理方法。对智能用电终端采集的用电数据归一化处理后,利用聚类算法从噪声、模糊、随机数据中提取出正常数据,本文对比验证了K-均值聚类和基于密度的空间聚类两种算法的聚类效果。相比K-均值聚类算法,密度的空间聚类两种算法在检测数据噪声点的同时,可自动获取复杂形状数据集的聚类数量,更适合智能用电网络的用电数据预处理。     

增量式K-Medoids聚类算法

聚类是一种非常有用的数据挖掘方法,可用于发现隐藏在数据背后的分组和数据分布信息。目前已经提出了许多聚类算法及其变种,但在增量式聚类算法研究方面所做的工作较少。当数据集因更新而发生变化时,数据挖掘的结果也应该进行相应的更新。由于数据量大,在更新后的数据集上重新执行聚类算法以更新挖掘结果显然比较低效,因此亟待研究增量式聚类算法。该文通过对K-Medoids聚类算法的改进,提出一种增量式K-Medoids聚类算法。它能够很好地解决传统聚类算法在伸缩性、数据定期更新时所面临的问题。     

常规挖掘算法在离群数据检测中的应用

数据挖掘以发现常规模式为主体,但离群数据在欺诈分析及安全领域具有重要分析价值,离群数据检测已成为数据挖掘的重要内容。对聚类与分类以及关联规则分析中典型的常规数据挖掘算法如何处理离群数据进行全面分析与总结,讨论了BIRCH、CURE、Chameleon、DBSCAN以及基于共享最近邻的聚类算法以及基于不平衡分类和基于非频繁模式的离群检测技术,给出了一种利用K-最近邻算法的离群数据检测方法,并报告了测试结果。     

Clustering in Dynamic Spatial Databases

Efficient clustering in dynamic spatial databases is currently an open problem with many potential applications. Most traditional spatial clustering algorithms are inadequate because they do not have an efficient support for incremental clustering.In this paper, we propose DClust, a novel clustering technique for dynamic spatial databases. DClust is able to provide multi-resolution view of the clusters, generate arbitrary shapes clusters in the presence of noise, generate clusters that are insensitive to ordering of input data and support incremental clustering efficiently. DClust utilizes the density criterion that captures arbitrary cluster shapes and sizes to select a number of representative points, and builds the Minimum Spanning Tree (MST) of these representative points, called R-MST. After the initial clustering, a summary of the cluster structure is built. This summary enables quick localization of the effect of data updates on the current set of clusters. Our experimental results show that DClust outperforms existing spatial clustering methods such as DBSCAN, C2P, DENCLUE, Incremental DBSCAN and BIRCH in terms of clustering time and accuracy of clusters found.