位置加权文本聚类算法

文本聚类是自然语言处理研究中一项重要研究课题,文本聚类技术广泛地应用于信息检索、Web挖掘和数字图书馆等领域。本文针对特征词在文档中的不同位置对文档的贡献大小不同,提出了基于特征词的位置加权文本聚类改进算法——TCABPW。通过选取反映文档主题的前L个高权值的特征项构造新的文本特征向量,采用层次聚类和K-means文本聚类相结合的改进算法实现文本聚类。实验结果表明,提出的改进算法在不影响聚类质量的情况下大大地降低了文本聚类的维度,在稳定性和纯度上都有显著提高,获得了较好的聚类效果。

A Text Clustering Algorithm Based on Position Weighting

Document clustering is an important research topic of natural language processing and is widely applicable in the areas such as information retrieval,web mining and digital libraries.Because the feature terms of different positions in the document are different for the article's contribution,TCABPW(a text clustering algorithm based on position weighting) is proposed in this paper.We construct a new text vector by selecting L-top-weight text that reflects the topical subject of the document and it is used to realize text clustering by hierarchical clustering and the K-means method.The results show that without affecting the quality of text clustering,the algorithm can not only greatly reduce the high dimension of text clustering,but also can significantly increase the stability and purity of text clutering,and can also produce the clusering effect of good quality.