双重高斯混合模型的EM算法的聚类问题研究

EM算法是参数估计的重要方法,其算法核心是根据已有的数据来迭代计算似然函数,使之收敛于某个最优值.半监督聚类是利用少部分标签的数据辅助大量未标签的数据进行的聚类分析.文章介绍了一种基于双重高斯混合模型的EM算法,在无监督学习中增加一些已标记的样本,利用已标记的样本得到初始参数,研究了半监督条件下的双重高斯混合模型的EM聚类算法.实验表明,该算法较无监督学习而言,提升了样本的识别率,有良好的聚类性能.这种算法模型也可以作为一种基础模型,具有一定的应用领域.     

有约束的半监督聚类方法

在数据挖掘领域的很多实际应用中,获取大量的无标签样本非常容易,而获取有标签的样本通常需要付出较大的代价,并且有时不可能得到所有的数据的标签,半监督聚类就是使用一小部分的标签数据对无标签数据的聚类过程进行指导。提出了一种新的半监督聚类算法,它利用标签数据提供的信息来初步确定数据的相似性和不相似性标准,并在聚类过程中对其进行自动调整,利用它们对聚类过程进行约束和指导。通过在标准数据集高斯数据集上的测试,该算法相对于无指导聚类来说有更高的精度和更快的速度。     

Facing the reality of data stream classification: coping with scarcity of labeled data

Recent approaches for classifying data streams are mostly based on supervised learning algorithms, which can only be trained with labeled data. Manual labeling of data is both costly and time consuming. Therefore, in a real streaming environment where large volumes of data appear at a high speed, only a small fraction of the data can be labeled. Thus, only a limited number of instances will be available for training and updating the classification models, leading to poorly trained classifiers. We apply a novel technique to overcome this problem by utilizing both unlabeled and labeled instances to train and update the classification model. Each classification model is built as a collection of micro-clusters using semi-supervised clustering, and an ensemble of these models is used to classify unlabeled data. Empirical evaluation of both synthetic and real data reveals that our approach outperforms state-of-the-art stream classification algorithms that use ten times more labeled data than our approach.     

一种基于谱聚类的半监督聚类方法

半监督聚类利用少部分标签的数据辅助大量未标签的数据进行非监督的学习,从而提高聚类的性能。提出一种基于谱聚类的半监督聚类算法,其利用标签数据的信息,调整点与点之间的距离所形成的距离矩阵,而后基于被调整的距离矩阵进行谱聚类。实验表明,该算法较之于已提出的半监督聚类算法,获得了更好的聚类性能。     

一种改进的半监督K-Means聚类算法

半监督聚类利用部分标签的数据辅助未标签的数据进行学习,从而提高聚类的性能。针对基于K-means的聚类算法发现非球状簇能力差的问题,本文提出新的处理思想,即把已标签数据对未标签数据的引力影响加入到类别分配决策中,给出了类与点的引力影响度定义,设计了带引力参数的半监督K-means聚类算法。实验表明,该算法在处理非球状簇分布的聚类时比现有的半监督K-means方法效果更好。     

基于量子粒子群优化的网络入侵检测算法

提出了一种将量子粒子群优化算法和半监督模糊核聚类算法相结合的混合算法,用以解决入侵检测算法中模糊聚类算法对初始值敏感,容易陷入局部最优的问题。该算法对少量标记数据进行监督聚类得到正确模型,运用这个模型指导大量未标记数据进行聚类,扩充标记数据集合,对仍没有确定标记的数据利用量子粒子群优化的模糊核聚类算法进行聚类,确定其标记类型。通过KDD CUP99实验数据的仿真,实验结果表明,该算法在入侵检测中能获得理想的检测率和误检率。     

A novel approach for distance-based semi-supervised clustering using functional link neural network

Semi-supervised clustering is gaining importance these days since neither supervised nor unsupervised learning methods in a stand-alone manner provide satisfactory results. Existing semi-supervised clustering techniques are mostly based on pair-wise constraints, which could be misleading. These semi-supervised clustering algorithms also fail to address the problem of dealing with attributes having different weights. In most of the real-life applications, all attributes do not have equal importance and hence same weights cannot be assigned for each attribute. In this paper, a novel distance-based semi-supervised clustering algorithm has been proposed, which uses functional link neural network (FLNN) for finding weights for attributes with small amount of labeled data for further use in parametric Minkowski’s model for clustering. In FLNN, the nonlinearity is captured by enhancing the input using orthonormal basis functions. The effectiveness of the approach has been illustrated over a number of datasets taken from UCI machine learning repository. Comparative performance evaluation demonstrates that the proposed approach outperforms the existing semi-supervised clustering algorithms. The proposed approach has also been successfully used to cluster the crime locations and to find crime hot spots in India on the data provided by National Crime Records Bureau (NCRB).     

基于密度自适应邻域相似图的半监督谱聚类

谱聚类是基于谱图划分理论的一种聚类算法,传统的谱聚类算法属于无监督学习算法,只能利用单一数据来进行聚类。针对这种情况,提出一种基于密度自适应邻域相似图的半监督谱聚类（DAN-SSC）算法。DAN-SSC算法在传统谱聚类算法的基础上结合了半监督学习的思想,很好地解决了传统谱聚类算法无法充分利用所有数据,不得不对一些有标签数据进行舍弃的问题;将少量的成对约束先验信息扩散至整个空间,使其能更好地对聚类过程进行指导。实验结果表明,DAN-SSC算法具有可行性和有效性。     

Combating with extremely noisy samples in weakly supervised slot filling for automatic diagnosis

Slot filling, to extract entities for specific types of information (slot), is a vitally important modular of dialogue systems for automatic diagnosis. Doctor responses can be regarded as the weak supervision of patient queries. In this way, a large amount of weakly labeled data can be obtained from unlabeled diagnosis dialogue, alleviating the problem of costly and time-consuming data annotation. However, weakly labeled data suffers from extremely noisy samples. To alleviate the problem, we propose a simple and effective Co-Weak-Teaching method. The method trains two slot filling models simultaneously. These two models learn from two different weakly labeled data, ensuring learning from two aspects. Then, one model utilizes selected weakly labeled data generated by the other, iteratively. The model, obtained by the Co-Weak-Teaching on weakly labeled data, can be directly tested on testing data or sequentially fine-tuned on a small amount of human-annotated data. Experimental results on these two settings illustrate the effectiveness of the method with an increase of 8.03% and 14.74% in micro and macro f1 scores, respectively.     

Semi-supervised image clustering with multi-modal information

How to organize and retrieve images is now a great challenge in various domains. Image clustering is a key tool in some practical applications including image retrieval and understanding. Traditional image clustering algorithms consider a single set of features and use ad hoc distance functions, such as Euclidean distance, to measure the similarity between samples. However, multi-modal features can be extracted from images. The dimension of multi-modal data is very high. In addition, we usually have several, but not many labeled images, which lead to semi-supervised learning. In this paper, we propose a framework of image clustering based on semi-supervised distance learning and multi-modal information. First we fuse multiple features and utilize a small amount of labeled images for semi-supervised metric learning. Then we compute similarity with the Gaussian similarity function and the learned metric. Finally, we construct a semi-supervised Laplace matrix for spectral clustering and propose an effective clustering method. Extensive experiments on some image data sets show the competent performance of the proposed algorithm.     

基于图收缩的半监督聚类算法

为了在只有少量已知标记的数据集中获得较好的聚类效果,提出了一种基于图收缩的半监督聚类算法。首先将整个样本空间中的数据表达为一个带权图,再根据给出的must-link约束,对图进行边收缩的修改,进而增强must-link约束。在此基础上引入图拉普拉斯算子,结合cannot-link约束将样本空间投影到一个特征子空间。最后在子空间上进行聚类分析。实验结果表明,该方法不仅提高了对复杂数据的聚类结果,而且在约束对数量较少时也能获得较好的结果。     

一种数据挖掘中的W-PAM限制聚类算法

在数据挖掘中由于每个数据对象对于知识发现的作用是不同的,为了区分这些相异之处,给每个对象赋予一定量的值,因此在PAM聚类算法的基础上提出一种W-PAM(Weight Partitioning Around Medoids)聚类算法,它为簇中数据对象加入权重来提高算法的准确率,此外利用数据对象间的关联限制能够提高聚类算法的效果。探讨了一种W-PAM算法与关联限制相结合的限制聚类算法,该算法同时拥有W-PAM算法和关联限制的优点。实验结果证明,W-PAM的限制聚类算法可以更有效地利用所给的关联限制来改善聚类效果,提高算法的准确率。     

自适应半监督模糊谱聚类算法

半监督聚类利用少部分标签的数据辅助大量未标签的数据进行非监督的学习,从而提高聚类的性能。大部分的谱聚类算法都需事先确定聚类数目,利用半监督机器学习技术和自适应聚类算法,解决算法中存在的聚类数目需要事先确定、易陷入局部最优、收敛速度缓慢、对孤立点敏感等缺陷。实验证明该算法有很好的聚类效果。     

高效的半监督多层次入侵检测算法

针对基于监督学习的入侵检测算法需要的大量有标签数据难以收集，无监督学习算法准确率不高，且对R2L及U2R两类攻击检测率低等问题，提出一种高效的半监督多层次入侵检测算法。首先，利用Kd-tree的索引结构，利用加权密度在高密度样本区选择K-means算法的初始聚类中心；然后，将聚类之后的数据分为三个类簇，将无标签类簇和混合类簇借助Tri-training采用加权投票规则扩充有标签数据集；最后，利用二叉树形结构设计层次化分类模型，在NSL-KDD数据集上进行了实验验证。结果表明半监督多层次入侵检测模型能够在利用少量有标签数据的情况下，对R2L及U2R的检测率分别达到49.38%、81.14%，有效提高R2L及U2R两类攻击的检测率，从而降低系统的漏报率。     

Active learning enabled activity recognition

Activity recognition in smart environment has been investigated rigorously in recent years. Researchers are enhancing the underlying activity discovery and recognition process by adding various dimensions and functionalities. But one significant barrier still persists which is collecting the ground truth information. Ground truth is very important to initialize a supervised learning of activities. Due to a large variety in number of Activities of Daily Living (ADLs), acknowledging them in a supervised way is a non-trivial research problem. Most of the previous researches have referenced a subset of ADLs and to initialize their model, they acquire a vast amount of informative labeled training data. On the other hand to collect ground truth and differentiate ADLs, human intervention is indispensable. As a result it takes an immense effort and raises privacy concerns to collect a reasonable amount of labeled data. In this paper, we propose to use active learning to alleviate the labeling effort and ground truth data collection in activity recognition pipeline. We investigate and analyze different active learning strategies to scale activity recognition and propose a dynamic k-means clustering based active learning approach. Experimental results on real data traces from a retirement community-(IRB #HP-00064387) help validate the early promise of our approach.     

一种进化半监督式模糊聚类的入侵检测算法

在入侵检测系统中,未知标签数据容易获得,标签数据较难获得,对此提出了一种基于进化半监督式模糊聚类入侵检测算法。算法利用标签数据信息担任染色体的角色,引导非标签数据每个模糊分类的进化过程,能够使用少量的标签数据和大量未知标签数据生成入侵检测系统分类器,可处理模糊类标签,不易陷入局部最优,适合并行结构的实现。实验结果表明,算法有较高的检测率。     

数据驱动的蒸发器在线建模方法

针对蒸发器离线建模方法对变量运行工况范围要求较大的问题，利用K-means算法对辨识模型的观测数据进行聚类筛选处理，提出一种基于数据的蒸发器在线建模方法。首先利用DB准则和PSO算法提出K-means算法中最优分类数K*和最优初始聚类中心的确定方法，提高算法的收敛速度，并使用改进的K-means算法获得各簇聚类中心来代替辨识模型的观测数据，减少模型辨识的数据量。然后利用已有的蒸发器模型结构以及模型辨识方法，对模型进行辨识。实验结果表明：利用聚类筛选前、后的观测数据所辨识的模型精度基本相当，分别在±3%和±3.5%以内。最后利用在线观测数据到各聚类中心欧氏距离的分析判断，提出蒸发器的在线建模方法。该方法可以先采用小工况范围的少量离线数据辨识模型，再利用在线数据修正模型参数，扩大模型的适用范围。     

基于困惑度数据挑选的半监督声学建模*

针对资源稀少情况下小语种的声学建模问题,提出根据解码后文本的困惑度挑选无监督数据并重新训练声学模型的策略.使用少量精标数据训练得到一个初始种子模型后,解码大量无监督数据,计算解码后的文本与精标数据文本的困惑度,从中挑选与精标数据相近的数据,再将这些数据与原有精标数据共同用于声学模型训练.为了提高解码的无监督数据的正确性,在基于深层神经网络的模型参数训练中,当最后一次模型参数更新时只使用精标数据修正网络参数.在NIST 2015年关键词识别比赛中Swahili语的VLLP识别任务上,相比其它方法,文中方法的识别率有一定提升.     

基于改进Leaders算子的审计潜在疑点发现

数据库查询方法审计疑点发现依赖于审计人员先验知识,当经验不足且审计数据量巨大时,难以发挥大数据优势并从海量数据中发现疑点。为解决这一问题,提出基于改进Leaders算子迭代聚类的审计大数据潜在疑点发现方法。该方法在无先验知识的情形下,通过Leaders算法自动完成审计大数据的初始聚类,在此基础上通过随机抽样融合方法对初始聚类结果优化,最后通过多次迭代聚类的方法,对实例数较少或可疑程度易被掩盖的小簇进一步聚类,实现审计大 数据的精确聚类,并将实例较少且行为明显异常的数据聚类识别为潜在疑点,配合审计人员审计经验快速精确定位审计疑点。实验结果验证了算法的有效性,表明算法有助于从海量数据中自主发现审计疑点,缩小疑点筛查范围,提高审计效率。     

基于差分隐私保护的DP-DBScan聚类算法研究

差分隐私保护是一种基于数据失真的隐私保护方法,通过添加随机噪声使敏感数据失真的同时也保证数据的统计特性。针对DBScan聚类算法在聚类分析过程中会泄露隐私的问题,提出一种新的基于差分隐私保护的DP-DBScan聚类算法。在满足ε-差分隐私保护的前提下,DP-DBScan聚类算法在基于密度的DBScan聚类算法上引入并实现了差分隐私保护。算法能够有效地保护个人隐私,适用于不同规模和不同维度的数据集。实验结果表明,与DBScan聚类算法相比,DP-DBScan聚类算法在添加少量随机噪声的情况下能保持聚类的有效性并获得差分隐私保护。