基于双重扰动的选择性支持向量机集成

为了进一步提升支持向量机泛化性能,提出一种基于双重扰动的选择性支持向量机集成算法。利用Boosting方法对训练集进行扰动基础上,采用基于相对核的粗糙集相对约简与重采样技术相结合的动态约简算法进行特征扰动以生成个体成员,然后基于负相关学习理论构造遗传个体适应度函数,利用加速遗传算法选择权重大于阈值的最优个体进行加权集成。实验结果表明,该算法具有较高的泛化性能和较低的时、空复杂性,是一种高效的集成方法。     

基于动态粗糙集约简的选择性支持向量机集成

支持向量机集成是提高支持向量机泛化性能的有效手段,个体支持向量机的泛化能力及其之间的差异性是影响集成性能的关键因素。为了进一步提升支持向量机整体泛化性能,提出利用动态粗糙集的选择性支持向量机集成算法。首先在利用Boosting算法对样本进行扰动基础上,采用遗传算法改进的粗糙集与重采样技术相结合的动态约简算法进行特征扰动,获得稳定、泛化能力较强的属性约简集,继而生成差异性较大的个体学习器;然后利用模糊核聚类根据个体学习器在验证集上的泛化误差来选择最优个体;并用支持向量机算法对最优个体进行非线性集成。通过在UCI数据集进行仿真,结果表明算法能明显提高支持向量机的泛化性能,具有较低的时、空复杂性,是一种高效、稳定的集成方法。     

粗SVM方法及其在发动机故障诊断中的应用

支持向量机是一种具有完备统计学习理论基础和出色学习性能的新型机器学习方法,它能够较好地克服过学习和泛化能力低等缺陷.但是在利用支持向量机的分类算法处理实际问题时,该算法的计算速度较慢、处理问题效率较低.文中介绍了一种新的学习算法粗SVM分类方法,就是将粗糙集和支持向量机相结合,利用粗糙集对支持向量机的训练样本进行预处理,通过属性约简方法以减少属性个数,且在属性约简过程中选出几组合适的属性集组成新的属性集,使模型具有一定的抗信息丢失能力,同时充分利用SCM的良好推广性能,从而缩短样本的训练时间,实现快速故障诊断.对航空发动机故障诊断的实验结果表明了该方法的优越性. 型机器学习方法,它能够较好地克服过学习和泛化能力低等缺陷.但是在利用支持向量机的分类算法处理实际问题时,该算法的计算速度较慢、处理问题效率较低.文中介绍了一种新的学习算法粗SVM分类方法,就是将粗糙集和支持向量机相结合,利用粗糙集对支持向量机的训练样本进行预处理,通过属性约简方法以减少属性个数,且在属性约筒过程中选出几组合适的属性集组成新的属性集,使模型具有一定的抗信息丢失能力,同时充分利用SCM的良好推广性能,从而缩短样本的训练时间,实现快速故障诊 .对航空发动机故障诊断的实验结果表明了该方法的优越性. 型机器学习方法     

基于属性约简及支持向量机的医疗诊断决策研究

把基于粗集的属性约简方法与支持向量机分类器的基本理论相结合,以提高决策分类的综合性能。将所提方法应用于医疗诊断决策,并对属性约简前后的决策性能进行了比较分析。试验结果表明,利用约简后的数据集,计算复杂性降低,内存需求减少,同时仍能保持较高的决策准确率。     

集成学习中基于离散化方法的基分类器构造研究

为构造集成学习中具有差异性的基分类器,提出基于数据离散化的基分类器构造方法,并用于支持向量机集成。该方法采用粗糙集和布尔推理离散化算法处理训练样本集,能有效删除不相关和冗余的属性,提高基分类器的准确性和差异性。实验结果表明,所提方法能取得比传统集成学习算法Bagging和Adaboost更好的性能。     

基于属性约简与半监督协同训练的入侵检测算法

入侵检测数据具有信息冗余量大、标记数据难以获得等特点。传统入侵检测方法难以消除冗余信息并且需要大量已标记样本做训练集,导致检测效率降低,实用性下降。为了解决上述问题,提出一种结合属性约简与半监督协同训练的算法。该算法充分发挥了大量未标记样本的监督作用。首先将入侵数据进行属性约简,利用约简结果建立一个支持向量机(SVM)基分类器,然后将其与另外两个SVM辅助分类器做协同训练。如此,分类器界面得到反复修正,分类器的性能逐步得到改善,最终分类精度得到明显提高。在入侵检测数据集KDDCUP99上的仿真实验结果表明,该算法不仅可以提高检测精度,同时还具有良好的可行性、稳定性。     

OCSVM集成模型解决支持向量机的大规模训练问题

该文针对集成方法实现支持向量机大规模训练的相关问题进行了深入研究,提出了一种称为"DD-Boosting"的成员分类器产生算法,能够在大规模数据集情况下利用类似Boosting技术产生稳定、高泛化性能的成员分类器。在此基础上,推导出基于OCSVM的分类器集成模型,实验仿真表明,该集成模型能够获得比主投票方法更好的泛化性能,且通过调整正则参数避免了训练过拟合问题。     

基于粗糙集的火灾图像特征选择与识别

针对图像型火灾探测方法检测准确度和实时性间的矛盾,提出了基于粗糙集的火灾图像特征选择和识别算法。首先通过对火焰图像特征的深入研究发现,在燃烧能量的驱动下火焰的上边缘极不规则,出现明显的震动现象,而下边缘却恰恰相反; 基于此特点,可利用上下边缘抖动投影个数比作为火焰区别于边缘形状较规则的干扰。然后,选择火焰的6个显著特征构造训练样本,在火灾分类能力不受影响的前提下,使用实验所得的特征量归类表对训练样本进行属性约简,并将约简后的信息系统属性训练支持向量机模型,实现火灾探测。最后与传统支持向量机火灾探测算法做了比较。实验结果表明:将粗糙集作为支持向量机分类器的前置系统,把粗糙集理论的属性约简引入到支持向量机中,可以大大消除样本集冗余属性,降低了火灾图像特征空间的维数,减少了分类器训练和检测数据,在保证识别精度的同时,提高了算法的速度和泛化能力。     

一种基于凸壳算法的SVM集成方法

为提高支持向量机(SVM)集成的训练速度,提出一种基于凸壳算法的SVM集成方法,得到训练集各类数据的壳向量,将其作为基分类器的训练集,并采用Bagging策略集成各个SVM。在训练过程中,通过抛弃性能较差的基分类器,进一步提高集成分类精度。将该方法用于3组数据,实验结果表明,SVM集成的训练和分类速度平均分别提高了266％和25％。     

基于Gabor小波和属性约简的人脸识别

人脸识别方法易受光照、姿态和表情变化的影响,针对这一问题,提出了一种基于Gabor小波和粗糙集属性约简的人脸识别方法。该方法先对人脸图像进行Gabor小波变换,将小波变换的系数作为人脸图像的特征向量;然后结合信息论中信息熵与互信息的概念定义了粗糙集里的一种新的属性重要度,并以此属性重要度为启发式信息进行约简数据集,从而对所得的人脸图像特征进行降维,并采用支持向量机进行分类。实验结果表明,该算法降低了支持向量机分类器的复杂度,有较好的识别性能。     

基于动态加权的粗糙子空间集成

提出一种基于动态加权的粗糙子空间集成方法EROS-DW。利用粗糙集属性约简方法获得多个特征约简子集,并据此训练基分类器。在分类阶段,根据给定待测样本的具体特征动态地为每个基分类器指派相应的权重,采用加权投票组合规则集成各分类器的输出结果。利用UCI标准数据集对该方法的性能进行测试。实验结果表明,相较于经典的集成方法,EROS-DW方法可以获得更高的分类准确率。     

一种基于粗糙集理论的组合分类器构造方法

首先分析了粗糙集理论和神经网络这两种理论的特点及其互补性,然后提出了一种构造组合分类器的新方法C3RST。新方法包括两个步骤,先对训练数据集进行约简,以此确定单个神经网络分类器的结构以及在组合分类器中要包含的分类器数目;然后将这些分类器组合起来,组合过程中各单个分类器的权值由粗糙集理论中的基本概念——属性重要性来决定。最后,在一些标准数据集上做实验验证C3RST的分类性能,结果表明该方法是有效的。     

基于多机器学习竞争策略的短时雷电预报

传统的雷电数据预测方法往往采用单一最优机器学习算法,较少考虑气象数据的时空变化等现象。针对该现象,提出一种基于集成策略的多机器学习短时雷电预报算法。首先,对气象数据进行属性约简,降低数据维度;其次,在数据集上训练多种异构机器学习分类器,并基于预测质量筛选最优基分类器;最后,通过对最优基分类器训练权重,并结合集成策略产生最终分类器。实验表明,该方法优于传统单最优方法,其平均预测准确率提高了9.5%。     

Diversity-based combination of non-parametric classifiers for EMG signal decomposition

Non-parametric classification procedures based on a certainty measure and nearest neighbour rule for motor unit potential classification (MUP) during electromyographic (EMG) signal decomposition were explored. A diversity-based classifier fusion approach is developed and evaluated to achieve improved classification performance. The developed system allows the construction of a set of non-parametric base classifiers and then automatically chooses, from the pool of base classifiers, subsets of classifiers to form candidate classifier ensembles. The system selects the classifier ensemble members by exploiting a diversity measure for selecting classifier teams. The kappa statistic is used as the diversity measure to estimate the level of agreement between base classifier outputs, i.e., to measure the degree of decision similarity between base classifiers. The pool of base classifiers consists of two kinds of classifiers: adaptive certainty-based classifiers (ACCs) and adaptive fuzzy k-NN classifiers (AFNNCs) and both utilize different types of features. Once the patterns are assigned to their classes, by the classifier fusion system, firing pattern consistency statistics for each class are calculated to detect classification errors in an adaptive fashion. Performance of the developed system was evaluated using real and simulated EMG signals and was compared with the performance of the constituent base classifiers and the performance of the fixed ensemble containing the full set of base classifiers. Across the EMG signal data sets used, the diversity-based classifier fusion approach had better average classification performance overall, especially in terms of reducing classification errors.     

Cluster‐based ensemble of classifiers

This paper presents cluster‐based ensemble classifier – an approach toward generating ensemble of classifiers using multiple clusters within classified data. Clustering is incorporated to partition data set into multiple clusters of highly correlated data that are difficult to separate otherwise and different base classifiers are used to learn class boundaries within the clusters. As the different base classifiers engage on different difficult‐to‐classify subsets of the data, the learning of the base classifiers is more focussed and accurate. A selection rather than fusion approach achieves the final verdict on patterns of unknown classes. The impact of clustering on the learning parameters and accuracy of a number of learning algorithms including neural network, support vector machine, decision tree and k‐NN classifier is investigated. A number of benchmark data sets from the UCI machine learning repository were used to evaluate the cluster‐based ensemble classifier and the experimental results demonstrate its superiority over bagging and boosting.     

EROS: Ensemble rough subspaces

Ensemble learning is attracting much attention from pattern recognition and machine learning domains for good generalization. Both theoretical and experimental researches show that combining a set of accurate and diverse classifiers will lead to a powerful classification system. An algorithm, called FS-PP-EROS, for selective ensemble of rough subspaces is proposed in this paper. Rough set-based attribute reduction is introduced to generate a set of reducts, and then each reduct is used to train a base classifier. We introduce an accuracy-guided forward search and post-pruning strategy to select part of the base classifiers for constructing an efficient and effective ensemble system. The experiments show that classification accuracies of ensemble systems with accuracy-guided forward search strategy will increase at first, arrive at a maximal value, then decrease in sequentially adding the base classifiers. We delete the base classifiers added after the maximal accuracy. The experimental results show that the proposed ensemble systems outperform bagging and random subspace methods in terms of accuracy and size of ensemble systems. FS-PP-EROS can keep or improve the classification accuracy with very few base classifiers, which leads to a powerful and compact classification system.     

基于改进动态集成选择算法的乳腺肿块辅助诊断模型

动态集成选择算法中,待测样本的能力区域由固定样本组成,这会影响分类器选择,因此提出一种基于动态能力区域策略的DES-DCR-CIER算法。首先采用异构分类器生成基分类器池,解决同构集成分类器差异性较小和异构集成分类器数目较少的问题;然后采用相互自适应K近邻算法、逼近样本集距离中心和剔除类别边缘样本三个步骤得到待测样本的动态能力区域,基于整体互补性指数选择一组互补性强的分类器;最后通过ER规则对分类器组进行合成。在安徽合肥某三甲医院的八位超声科医生乳腺肿块诊断数据集和美国威斯康辛州乳腺癌诊断公开数据集上的实验表明,基于DES-DCR-CIER算法的诊断模型精度更优。     

A dempster-shafer theoretic framework for boosting based ensemble design

Training set resampling based ensemble design techniques are successfully used to reduce the classification errors of the base classifiers. Boosting is one of the techniques used for this purpose where each training set is obtained by drawing samples with replacement from the available training set according to a weighted distribution which is modified for each new classifier to be included in the ensemble. The weighted resampling results in a classifier set, each being accurate in different parts of the input space mainly specified the sample weights. In this study, a dynamic integration of boosting based ensembles is proposed so as to take into account the heterogeneity of the input sets. An evidence-theoretic framework is developed for this purpose so as to take into account the weights and distances of the neighboring training samples in both training and testing boosting based ensembles. The effectiveness of the proposed technique is compared to the AdaBoost algorithm using three different base classifiers.     

The random boosting ensemble classifier for land-use image classification

This paper presents a random boosting ensemble (RBE) classifier for remote sensing image classification, which introduces the random projection feature selection and bootstrap methods to obtain base classifiers for classifier ensemble. The RBE method is built based on an improved boosting framework, which is quite efficient for the few-shot problem due to the bootstrap in use. In RBE, kernel extreme machine (KELM) is applied to design base classifiers, which actually make RBE quite efficient due to feature reduction. The experimental results on the remote scene image classification demonstrate that RBE can effectively improve the classification performance, and resulting into a better generalization ability on the 21-class land-use dataset and the India pine satellite scene dataset.

     

基于差异性和准确性的加权调和平均度量的 基因表达数据选择性集成算法

基分类器之间的差异性和单个基分类器自身的准确性是影响集成系统泛化性能的两个重要因素,针对差异性和准确性难以平衡的问题,提出了一种基于差异性和准确性的加权调和平均（D-A-WHA）度量基因表达数据的选择性集成算法。以核超限学习机（KELM）作为基分类器,通过D-A-WHA度量调节基分类器之间的差异性和准确性,最后选择一组准确性较高并且与其他基分类器差异性较大的基分类器组合进行集成。通过在UCI基因数据集上进行仿真实验,实验结果表明,与传统的Bagging、Adaboost等集成算法相比,基于D-A-WHA度量的选择性集成算法分类精度和稳定性都有显著的提高,且能有效应用于癌症基因数据的分类中。