Combining rough decisions for intelligent text mining using Dempster’s rule

An important issue in text mining is how to make use of multiple pieces knowledge discovered to improve future decisions. In this paper, we propose a new approach to combining multiple sets of rules for text categorization using Dempster’s rule of combination. We develop a boosting-like technique for generating multiple sets of rules based on rough set theory and model classification decisions from multiple sets of rules as pieces of evidence which can be combined by Dempster’s rule of combination. We apply these methods to 10 of the 20-newsgroups—a benchmark data collection (Baker and McCallum 1998), individually and in combination. Our experimental results show that the performance of the best combination of the multiple sets of rules on the 10 groups of the benchmark data is statistically significant and better than that of the best single set of rules. The comparative analysis between the Dempster–Shafer and the majority voting (MV) methods along with an overfitting study confirm the advantage and the robustness of our approach.     

基于置信度的手写体数字识别多分类器动态组合

多分类器组合利用不同分类器、不同特征之间的互补性，提高了组合分类器的识别率。传统的组合方法里，各分类器在组合中所承担的角色是固定的，而实际应用中，对于不同的测试样本，每个分类器识别结果的可信度是不同的。该文根据分类器置信度理论，提出了各类别的置信度。用测试样本自身的置信度信息实现分类器的动态组合，并把这种动态组合方法具体应用到手写体数字的识别。这种方法还可以在不影响已有数据的情况下添加新的分类器进行组合。     

多分类器系统的泛组合规则研究与应用

现有的多分类器系统采用固定的组合算子,适用性较差。将泛逻辑的柔性化思想引入多分类器系统中,应用泛组合运算模型建立了泛组合规则。泛组合规则采用遗传算法进行参数估计,对并行结构的多分类器系统具有良好的适用性。在时间序列数据集上的分类实验结果表明,泛组合规则的分类性能优于乘积规则、均值规则、中值规则、最大规则、最小规则、投票规则等固定组合规则。     

多分类器组合研究

文章提出了一种多分类器的组合方法,它利用了参与组合的分类器提供的度量层次上的两类信息:对训练样本的决策信息;对待识样本的决策信息。首先对这两类信息进行集成,进而给出了组合分类器的判定规则。用该方法对手写体汉字作分类识别,实验结果显示,较之其它几种方法,它有更高的正确识别率。     

改进的贝叶斯多分类器组合规则

多分类器组合是提高识别效果的一条有效途径。文中提出一种用于多分类器组合的改进贝叶斯规则,即首先通过对大量样本的统计获得有关每个分类器识别性能的先验知识,将其作为多分类器组合的依据。组合时对每个类设置不同的阈值,使组合效果得以改善,这些阈值可以通过训练获得。在数字识别中的应用结果表明,改进的贝叶斯规则可以使多分类器的组合结果识别率和置信度得到明显提高。     

Neural network ensembles: combining multiple models for enhanced performance using a multistage approach

Abstract: Neural network ensembles (sometimes referred to as committees or classifier ensembles) are effective techniques to improve the generalization of a neural network system. Combining a set of neural network classifiers whose error distributions are diverse can generate better results than any single classifier. In this paper, some methods for creating ensembles are reviewed, including the following approaches: methods of selecting diverse training data from the original source data set, constructing different neural network models, selecting ensemble nets from ensemble candidates and combining ensemble members' results. In addition, new results on ensemble combination methods are reported.     

Building ensemble classifiers using belief functions and OWA operators

A pervasive task in many forms of human activity is classification. Recent interest in the classification process has focused on ensemble classifier systems. These types of systems are based on a paradigm of combining the outputs of a number of individual classifiers. In this paper we propose a new approach for obtaining the final output of ensemble classifiers. The method presented here uses the Dempster–Shafer concept of belief functions to represent the confidence in the outputs of the individual classifiers. The combing of the outputs of the individual classifiers is based on an aggregation process which can be seen as a fusion of the Dempster rule of combination with a generalized form of OWA operator. The use of the OWA operator provides an added degree of flexibility in expressing the way the aggregation of the individual classifiers is performed.     

Combining Classifiers with Meta Decision Trees

The paper introduces meta decision trees (MDTs), a novel method for combining multiple classifiers. Instead of giving a prediction, MDT leaves specify which classifier should be used to obtain a prediction. We present an algorithm for learning MDTs based on the C4.5 algorithm for learning ordinary decision trees (ODTs). An extensive experimental evaluation of the new algorithm is performed on twenty-one data sets, combining classifiers generated by five learning algorithms: two algorithms for learning decision trees, a rule learning algorithm, a nearest neighbor algorithm and a naive Bayes algorithm. In terms of performance, stacking with MDTs combines classifiers better than voting and stacking with ODTs. In addition, the MDTs are much more concise than the ODTs and are thus a step towards comprehensible combination of multiple classifiers. MDTs also perform better than several other approaches to stacking.     

Plurality voting-based multiple classifier systems: statistically independent with respect to dependent classifier sets

The simultaneous use of multiple classifiers has been shown to provide performance improvement in classification problems. The selection of an optimal set of classifiers is an important part of multiple classifier systems and the independence of classifier outputs is generally considered to be an advantage for obtaining better multiple classifier systems. In this paper, the need for the classifier independence is interrogated from classification performance point of view. The performance achieved with the use of classifiers having independent joint distributions is compared to some other classifiers which are defined to have best and worst joint distributions. These distributions are obtained by formulating the combination operation as an optimization problem. The analysis revealed several important observations about classifier selection which are then used to analyze the problem of selecting an additional classifier to be used with the available multiple classifier system.     

Troika - An improved stacking schema for classification tasks

Stacking is a general ensemble method in which a number of base classifiers are combined using one meta-classifier which learns their outputs. Such an approach provides certain advantages: simplicity; performance that is similar to the best classifier; and the capability of combining classifiers induced by different inducers. The disadvantage of stacking is that on multiclass problems, stacking seems to perform worse than other meta-learning approaches. In this paper we present Troika, a new stacking method for improving ensemble classifiers. The new scheme is built from three layers of combining classifiers. The new method was tested on various datasets and the results indicate the superiority of the proposed method to other legacy ensemble schemes, Stacking and StackingC, especially when the classification task consists of more than two classes.     

基于多分类器集成的手写体金融汉字识别

用基于遗传算法的全局优化技术动态地选择一组分类器,并根据应用的背景,采用合适的集成规则进行集成,从而综合了不同分类器的优势和互补性,提高了分类性能。实验结果表明,通过将遗传算法引入到多分类器集成系统的设计过程,其分类性能明显优于传统的单分类器的分类方法。     

多种分类器融合的遥感影像分类

在遥感影像分类应用中,不同分类器的分类精度是不同的,而同一分类器对不同类别的分类精度也是不相同的。多分类器结合的思想就是利用现有分类器之间的互补性,通过适当的方法将不同的分类器之间进行优势互补,往往可以得到比单个分类器更好的分类结果。本文研究了如何在Matlab下采用最短距离分类器、贝叶斯分类器、BP神经网络分类器对影像进行分类,并采用投票法进行多种分类器结合的遥感影像分类,最后进行分类后处理。实验结果表明多分类器结合的遥感影像分类比单一分类器分类的精度高。     

A Feature-Based Serial Approach to Classifier Combination

: A new approach to the serial multi-stage combination of classifiers is proposed. Each classifier in the sequence uses a smaller subset of features than the subsequent classifier. The classification provided by a classifier is rejected only if its decision is below a predefined confidence level. The approach is tested on a two-stage combination of k-Nearest Neighbour classifiers. The features to be used by the first classifier in the combination are selected by two stand-alone algorithms (Relief and Info-Fuzzy Network, or IFN) and a hybrid method, called ‘IFN + Relief’. The feature-based approach is shown empirically to provide a substantial decrease in the computational complexity, while maintaining the accuracy level of a single-stage classifier or even improving it. Received: 24 November 2000, Received in revised form: 30 November 2001, Accepted: 05 June 2002 ID="A1" Correspondence and offprint requests to: M. Last, Department of Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva 84105, Israel. Email: mlast@bgumail.bgu.ac.il     

基于相关性和有效互补性分析的多分类器组合方法

定义了分类器组合中的相关向量和有效互补性的概念,并提出了一种新的组合准 则,即最大有效互补准则.对人脸图象作正交小波变换,得到它在不同频带上的四个子图象, 然后分别提取奇异值特征.实验表明,这四组特征之间以及相应的分类结果之间的相关性都 较小,组合结果明显优于原始图象的奇异值特征的分类效果,并优于常用的组合方法--计 分法的效果.     

基于Multi-Agent的分类器融合

针对决策层输出的分类器融合问题，该文提出了一种基于Multi-Agent思想的融合算法，该算法将分类器融合问题建模为人类发源地问题，通过引入决策共现矩阵，并在智能体之间进行信息交互，从而利用了分类器之间的决策相关信息，算法根据在融合训练集上得到的统计参量，指导各个智能体向不同类别溯源，并通过智能体之间的信息交换改变溯源概率，最终达到群体决策，得到决策类别，本文在标准数据集上对该算法进行了实验研究，通过与其它一些融合方法的比较，得出在用于融合的分类器较少时，该算法得到比其它方法更低的分类错误率，其空间复杂度相对BKS方法较小，实验证实，该算法是收敛的。     

A genetic encoding approach for learning methods for combining classifiers

Several studies have reported that the ensemble of classifiers can improve the performance of a stand-alone classifier. In this paper, we propose a learning method for combining the predictions of a set of classifiers.The method described in this paper uses a genetic-based version of the correspondence analysis for combining classifiers. The correspondence analysis is based on the orthonormal representation of the labels assigned to the patterns by a pool of classifiers. In this paper instead of the orthonormal representation we use a pool of representations obtained by a genetic algorithm. Each single representation is used to train a different classifiers, these classifiers are combined by vote rule.The performance improvement with respect to other learning-based fusion methods is validated through experiments with several benchmark datasets.     

基于小波包和组合分类器的脑电信号分类

为了提高脑思维任务分类精度,提出了一种基于小波包分解和多分类器投票组合的运动想象任务分类方法。该方法利用小波包分解对经过预处理的脑电信号进行分解,提取所有频带上的相对小波包能量特征;根据不同脑思维任务下左右半脑各通道间的差异性对C3、C4两通道求取特定频带上的小波包系数的L-2范数作为特征;采用基于投票策略的组合分类器对两种联合特征进行分类,得到了92.85%的识别精度。实验结果表明,联合特征向量较好地反映了左右手运动想象脑电信号的事件相关去同步（ERD）和事件相关同步（ERS）的本质特性;组合分类器识别效果优于单一分类器。     

On optimal pairwise linear classifiers for normal distributions: the d-dimensional case

We consider the well-studied pattern recognition problem of designing linear classifiers. When dealing with normally distributed classes, it is well known that the optimal Bayes classifier is linear only when the covariance matrices are equal. This was the only known condition for classifier linearity. In a previous work, we presented the theoretical framework for optimal pairwise linear classifiers for two-dimensional normally distributed random vectors. We derived the necessary and sufficient conditions that the distributions have to satisfy so as to yield the optimal linear classifier as a pair of straight lines.In this paper we extend the previous work to d-dimensional normally distributed random vectors. We provide the necessary and sufficient conditions needed so that the optimal Bayes classifier is a pair of hyperplanes. Various scenarios have been considered including one which resolves the multi-dimensional Minsky’s paradox for the perceptron. We have also provided some three-dimensional examples for all the cases, and tested the classification accuracy of the corresponding pairwise-linear classifier. In all the cases, these linear classifiers achieve very good performance. To demonstrate that the current pairwise-linear philosophy yields superior discriminants on real-life data, we have shown how linear classifiers determined using a maximum-likelihood estimate (MLE) applicable for this approach, yield better accuracy than the discriminants obtained by the traditional Fisher's classifier on a real-life data set. The multi-dimensional generalization of the MLE for these classifiers is currently being investigated.     

多分类器融合实现机型识别

针对空战目标识别中机型识别这一问题,提出了基于多分类器融合的识别方法。该方法以战术性能参数为输入,便于满足空战的实时性要求。通过广泛收集数据,得到机型识别的分类特征,选取分类特征的子集作为单分类器的特征,用BP网络设计单分类器,然后选用性能优良的和规则进行分类器融合,求得最终的决策。实验结果表明,多分类器融合的识别性能明显优于参与融合的分类器,也优于相同输入的单分类器。该方法的另一特点是能够进行缺省推理,因而有较强的抗干扰能力,适合真实战场环境的需要。     

基于全信息相关度的动态多分类器融合

AdaB00st采用级联方法生成各基分类器,较好地体现了分类器之间的差异性和互补性.其存在的问题是,在迭代的后期,训练分类器越来越集中在某一小区域的样本上,生成的基分类器体现不同区域的分类特征.根据基分类器的全局分类性能得到固定的投票权重,不能体现基分类器在不同区域上的局部性能差别.因此,本文基于Ada-Boost融合方法,利用待测样本与各分类器的全信息相关度描述基分类器的局部分类性能,提出基于全信息相关度的动态多分类器融合方法,根据各分类器对待测样本的局部分类性能动态确定分类器组合和权重.仿真实验结果表明,该算法提高了融合分类性能.