知识增益: 文本分类中一种新的特征选择方法

特征选择在文本分类中起重要的作用。文档频率(DF)、信息增益(IG)和互信息(MI)等特征选择方法在文本分类中广泛应用。已有的实验结果表明,IG是最有效的特征选择算法之一,该方法基于申农提出的信息论。本文基于粗糙集理论,提出了一种新的特征选择方法(KG算法),该方法依据粗糙集理论关于知识的观点,即知识是分类事物的能力,将知识进行量化,提出知识增益的概念,得到基于知识增益的特征选择方法。在两个通用的语料集OHSUMED和NewsGroup上进行分类实验发现KG算法均超过IG的性能,特别是在特征空间的维数降到低维时尤其明显,可见KG算法有较好的性能;

Knowledge Gain: An New Feature Selection Method in Text Categorization

Feature selection(FS) plays an important role in text categorization(TC).Automatic feature selection methods such as document frequency thresholding(DF),information gain(IG),mutual information(MI),and so on are commonly applied in text categorization [J].Existing experiments show IG is one of the most effective methods.In this paper,a feature selection method is proposed based on Rough Set theory.According to Rough set theory,knowledge about a universe of objects may be defined as classifications based on certain properties of the objects,i.e.rough set theory assume that knowledge is an ability to partition objects.We quantify the ability of classify objects,and call the amount of this ability as knowledge quantity and then following this quantification,we put forward a notion "knowledge Gain" and put forward a knowledge gain-based feature selection method(KG method).Experiments on NewsGroup collection and OHSUMED collection show that KG performs better than the IG method,specially,on extremely aggressive reduction.