几种机器学习方法在人脸识别中的性能比较

BP神经网络、RBF神经网络、支持向量机（SVM）和集成学习是目前应用最为广泛的四种机器学习方法。将这四种常用的机器学习方法分别应用于人脸识别,并利用ORL人脸图像库对各学习方法性能进行了测试和评估。测试结果表明SVM和集成学习在实验中取得了较好的性能,最适合用于人脸识别中特征分类器。     

Forecasting Bank Failure: Base Learners,Ensembles and Hybrid Ensembles

The prediction of bankruptcy for financial companies, especially banks, has been extensively researched area and creditors, auditors, stockholders and senior managers are all interested in bank bankruptcy prediction. In this paper, three common machine learning models namely Logistic, J48 and Voted Perceptron are used as the base learners. In addition, an attribute-base ensemble learning method namely Random Subspaces and two instance-base ensemble learning methods namely Bagging and Multi-Boosting are employed to enhance the prediction accuracy of conventional machine learning models for bank failure prediction. The models are grouped in the following families of approaches: (i) conventional machine learning models, (ii) ensemble learning models and (iii) hybrid ensemble learning models. Experimental results indicate a clear outperformance of hybrid ensemble machine learning models over conventional base and ensemble models. These results indicate that hybrid ensemble learning models can be used as a reliable predicting model for bank failures.     

Bankruptcy prediction using support vector machine with optimal choice of kernel function parameters

Bankruptcy prediction has drawn a lot of research interests in previous literature, and recent studies have shown that machine learning techniques achieved better performance than traditional statistical ones. This paper applies support vector machines (SVMs) to the bankruptcy prediction problem in an attempt to suggest a new model with better explanatory power and stability. To serve this purpose, we use a grid-search technique using 5-fold cross-validation to find out the optimal parameter values of kernel function of SVM. In addition, to evaluate the prediction accuracy of SVM, we compare its performance with those of multiple discriminant analysis (MDA), logistic regression analysis (Logit), and three-layer fully connected back-propagation neural networks (BPNs). The experiment results show that SVM outperforms the other methods.     

改进的粒子群算法及其SVM参数优化应用

支持向量机是一种性能优越的机器学习算法,而其参数的选择对建模精度和泛化性能等有着重要的影响,也是目前机器学习研究的一个重要方向。在简要介绍基本粒子群优化（PSO）算法的基础上,提出了一种量子粒子群优化算法,给出了其实现方式,并通过4个基准测试函数进行性能对比评价。基于这种量子粒子群优化算法,对最小二乘支持向量机（LS-SVM）的参数优化进行了研究。仿真结果表明,量子粒子群优化算法能给出很好的优化结果。     

Fast detection of impact location using kernel extreme learning machine

Damage location detection has direct relationship with the field of aerospace structure as the detection system can inspect any exterior damage that may affect the operations of the equipment. In the literature, several kinds of learning algorithms have been applied in this field to construct the detection system and some of them gave good results. However, most learning algorithms are time-consuming due to their computational complexity so that the real-time requirement in many practical applications cannot be fulfilled. Kernel extreme learning machine (kernel ELM) is a learning algorithm, which has good prediction performance while maintaining extremely fast learning speed. Kernel ELM is originally applied to this research to predict the location of impact event on a clamped aluminum plate that simulates the shell of aerospace structures. The results were compared with several previous work, including support vector machine (SVM), and conventional back-propagation neural networks (BPNN). The comparison result reveals the effectiveness of kernel ELM for impact detection, showing that kernel ELM has comparable accuracy to SVM but much faster speed on current application than SVM and BPNN.     

基于C-AdaBoost模型的乳腺癌预测研究

机器学习和深度学习技术可用于解决医学分类预测中的许多问题,其中一些分类算法的预测精度较高,而另一些算法的精度有限。提出了基于C-AdaBoost模型的集成学习算法,对乳腺癌疾病进行预测,发现了判断乳腺癌是否复发、乳腺癌肿瘤是否为良性的最优特征组合。通过逐步回归方法对现有特征进行二次选取,并结合C-AdaBoost模型使得预测效果更优。大量实验表明,基于C-AdaBoost模型的算法的预测准确率比SVM、Naive Bayes、RandomForest以及传统的集成学习模型等机器学习分类器的准确率最多可提高19.5%,从而可以更好地帮助医生进行临床决策。     

Software effort estimation accuracy prediction of machine learning techniques: A systematic performance evaluation

Software effort estimation accuracy is a key factor in effective planning, controlling, and delivering a successful software project within budget and schedule. The overestimation and underestimation both are the key challenges for future software development, henceforth there is a continuous need for accuracy in software effort estimation. The researchers and practitioners are striving to identify which machine learning estimation technique gives more accurate results based on evaluation measures, datasets and other relevant attributes. The authors of related research are generally not aware of previously published results of machine learning effort estimation techniques. The main aim of this study is to assist the researchers to know which machine learning technique yields the promising effort estimation accuracy prediction in software development. In this article, the performance of the machine learning ensemble and solo techniques are investigated on publicly and non-publicly domain datasets based on the two most commonly used accuracy evaluation metrics. We used the systematic literature review methodology proposed by Kitchenham and Charters. This includes searching for the most relevant papers, applying quality assessment (QA) criteria, extracting data, and drawing results. We have evaluated a state-of-the-art accuracy performance of 35 selected studies (17 ensemble, 18 solo) using mean magnitude of relative error and PRED (25) as a set of reliable accuracy metrics for performance evaluation of accuracy among two techniques to report the research questions stated in this study. We found that machine learning techniques are the most frequently implemented in the construction of ensemble effort estimation (EEE) techniques. The results of this study revealed that the EEE techniques usually yield a promising estimation accuracy than the solo techniques.     

Empirical analysis of support vector machine ensemble classifiers

Ensemble classification – combining the results of a set of base learners – has received much attention in the machine learning community and has demonstrated promising capabilities in improving classification accuracy. Compared with neural network or decision tree ensembles, there is no comprehensive empirical research in support vector machine (SVM) ensembles. To fill this void, this paper analyses and compares SVM ensembles with four different ensemble constructing techniques, namely bagging, AdaBoost, Arc-X4 and a modified AdaBoost. Twenty real-world data sets from the UCI repository are used as benchmarks to evaluate and compare the performance of these SVM ensemble classifiers by their classification accuracy. Different kernel functions and different numbers of base SVM learners are tested in the ensembles. The experimental results show that although SVM ensembles are not always better than a single SVM, the SVM bagged ensemble performs as well or better than other methods with a relatively higher generality, particularly SVMs with a polynomial kernel function. Finally, an industrial case study of gear defect detection is conducted to validate the empirical analysis results.     

基于证据理论的多类分类支持向量机集成

针对多类分类问题,研究支持向量机集成中的分类器组合架构与方法.分析已有的多类级和两类级支持向量机集成架构的不足后,提出两层的集成架构.在此基础上,研究基于证据理论的支持向量机度量层输出信息融合方法,针对一对多与一对一两种多类扩展策略,分别定义基本概率分配函数,并根据证据冲突程度采用不同的证据组合规则.在一对多策略下,采用经典的Dempster规则;在一对一策略下则提出一条新的规则,以组合冲突严重的证据.实验表明,两层架构优于多类级架构,证据理论方法能有效地利用两类支持向量机的度量层输出信息,取得了满意的结果.     

一种回归SVM选择性集成方法

泛化能力是机器学习关心的一个根本问题,采用集成学习技术可以有效地提高泛化能力.本文提出了一种将支持向量机(Support Vector Machine, SVM)进行选择性集成回归的方法.通过引入三个阈值,可以选择合适的子SVM,从而进一步提高了整个集成学习的效率.实验结果表明,本文提出的选择性集成方法可以在一定程度上解决SVM的模型选择问题和大规模数据集的学习问题,与传统的集成方法Bagging相比具有更高的泛化能力.     

类别不均衡学习中的抽样策略研究

类别不均衡学习在信用评估、客户流失预测、医学诊断、短文本情感分析、标记学习、评分预测等众多领域有广泛的应用，是机器学习研究和应用的热点方向之一，近年来逐渐引起学术界和工业界的广泛关注。目前解决类别不均衡问题主要有三种方法：数据级解决方法、算法级解决方法和集成解决方法。侧重于对近年来类别不均衡学习中的抽样策略研究进展进行综述，介绍类别不均衡学习的基本框架，对类别不均衡学习中三种主要的抽样策略（过抽样、欠抽样和混合抽样）相关研究进展进行前沿概括、比较和分析，对类别不均衡学习的抽样策略中有待研究的难点、热点及发展趋势进行展望。     

Ensemble of Efficient Minimal Learning Machines for Classification and Regression

Minimal Learning Machine (MLM) is a recently proposed supervised learning algorithm with performance comparable to most state-of-the-art machine learning methods. In this work, we propose ensemble methods for classification and regression using MLMs. The goal of ensemble strategies is to produce more robust and accurate models when compared to a single classifier or regression model. Despite its successful application, MLM employs a computationally intensive optimization problem as part of its test procedure (out-of-sample data estimation). This becomes even more noticeable in the context of ensemble learning, where multiple models are used. Aiming to provide fast alternatives to the standard MLM, we also propose the Nearest Neighbor Minimal Learning Machine and the Cubic Equation Minimal Learning Machine to cope with classification and single-output regression problems, respectively. The experimental assessment conducted on real-world datasets reports that ensemble of fast MLMs perform comparably or superiorly to reference machine learning algorithms.     

Developing an SVM-based ensemble learning system for customer risk identification collaborating with customer relationship management

In this study, we propose a support vector machine (SVM)-based ensemble learning system for customer relationship management (CRM) to help enterprise managers effectively manage customer risks from the risk aversion perspective. This system differs from the classical CRM for retaining and targeting profitable customers; the main focus of the proposed SVM-based ensemble learning system is to identify high-risk customers in CRM for avoiding possible loss. To build an effective SVM-based ensemble learning system, the effects of ensemble members’ diversity, ensemble member selection and different ensemble strategies on the performance of the proposed SVM-based ensemble learning system are each investigated in a practical CRM case. Through experimental analysis, we find that the Bayesian-based SVM ensemble learning system with diverse components and choose from space selection strategy show the best performance over various testing samples.     

Extreme learning machine for structured output spaces

Recently, extreme learning machine (ELM) has attracted increasing attention due to its successful applications in classification, regression, and ranking. Normally, the desired output of the learning system using these machine learning techniques is a simple scalar output. However, there are many applications in machine learning which require more complex output rather than a simple scalar one. Therefore, structured output is used for such applications where the system is trained to predict structured output instead of simple one. Previously, support vector machine (SVM) has been introduced for structured output learning in various applications. However, from machine learning point of view, ELM is known to offer better generalization performance compared to other learning techniques. In this study, we extend ELM to more generalized framework to handle complex outputs where simple outputs are considered as special cases of it. Besides the good generalization property of ELM, the resulting model will possesses rich internal structure that reflects task-specific relations and constraints. The experimental results show that structured ELM achieves similar (for binary problems) or better (for multi-class problems) generalization performance when compared to ELM. Moreover, as verified by the simulation results, structured ELM has comparable or better precision performance with structured SVM when tested for more complex output such as object localization problem on PASCAL VOC2006. Also, the investigation on parameter selections is presented and discussed for all problems.

     

一种基于概率校正和集成学习的肠癌肝转移预测模型

提出一种基于概率校正和集成学习的机器学习模型,用来预测患者肠癌肝转移的概率。首先将AdaBoost和Class-bal-anced SVM的概率结果进行校正,再将其结果和Logistic回归的预测结果进行集成,获得最终的预测结果。预测模型在复旦大学附属肿瘤医院的肠癌患者数据集上与其他算法如AdaBoost、Class-balanced SVM、Logistic回归算法进行了比较,结果显示该模型具有更好的AUC性能,更适合于医生的临床辅助诊断。模型的AUC性能在UCI数据集上进一步得到了验证。     

Financial distress prediction using support vector machines: Ensemble vs. individual

Financial distress prediction (FDP) is of great importance to both inner and outside parts of companies. Though lots of literatures have given comprehensive analysis on single classifier FDP method, ensemble method for FDP just emerged in recent years and needs to be further studied. Support vector machine (SVM) shows promising performance in FDP when compared with other single classifier methods. The contribution of this paper is to propose a new FDP method based on SVM ensemble, whose candidate single classifiers are trained by SVM algorithms with different kernel functions on different feature subsets of one initial dataset. SVM kernels such as linear, polynomial, RBF and sigmoid, and the filter feature selection/extraction methods of stepwise multi discriminant analysis (MDA), stepwise logistic regression (logit), and principal component analysis (PCA) are applied. The algorithm for selecting SVM ensemble's base classifiers from candidate ones is designed by considering both individual performance and diversity analysis. Weighted majority voting based on base classifiers’ cross validation accuracy on training dataset is used as the combination mechanism. Experimental results indicate that SVM ensemble is significantly superior to individual SVM classifier when the number of base classifiers in SVM ensemble is properly set. Besides, it also shows that RBF SVM based on features selected by stepwise MDA is a good choice for FDP when individual SVM classifier is applied.     

Functional networks as a new data mining predictive paradigm to predict permeability in a carbonate reservoir

Permeability prediction has been a challenge to reservoir engineers due to the lack of tools that measure it directly. The most reliable data of permeability obtained from laboratory measurements on cores do not provide a continuous profile along the depth of the formation. Recently, researchers utilized statistical regression, neural networks, and fuzzy logic to estimate both permeability and porosity from well logs. Unfortunately, due to both uncertainty and imprecision, the developed predictive modelings are less accurate compared to laboratory experimental core data. This paper presents functional networks as a novel approach to forecast permeability using well logs in a carbonate reservoir. The new intelligence paradigm helps to overcome the most common limitations of the existing modeling techniques in statistics, data mining, machine learning, and artificial intelligence communities. To demonstrate the usefulness of the functional networks modeling strategy, we briefly describe its learning algorithm through simple distinct examples. Comparative studies were carried out using real-life industry wireline logs to compare the performance of the new framework with the most popular modeling schemes, such as linear/nonlinear regression, neural networks, and fuzzy logic inference systems. The results show that the performance of functional networks (separable and generalized associativity) architecture with polynomial basis is accurate, reliable, and outperforms most of the existing predictive data mining modeling approaches. Future work can be achieved using different structure of functional networks with different basis, interaction terms, ensemble and hybrid strategies, different clustering, and outlier identification techniques within different oil and gas challenge problems, namely, 3D passive seismic, identification of lithofacies types, history matching, rock mechanics, viscosity, risk assessment, and reservoir characterization.     

Failure mode classification and bearing capacity prediction for reinforced concrete columns based on ensemble machine learning algorithm

Failure mode (FM) and bearing capacity of reinforced concrete (RC) columns are key concerns in structural design and/or performance assessment procedures. The failure types, i.e., flexure, shear, or mix of the above two, will greatly affect the capacity and ductility of the structure. Meanwhile, the design methodologies for structures of different failure types will be totally different. Therefore, developing efficient and reliable methods to identify the FM and predict the corresponding capacity is of special importance for structural design/assessment management. In this paper, an intelligent approach is presented for FM classification and bearing capacity prediction of RC columns based on the ensemble machine learning techniques. The most typical ensemble learning method, adaptive boosting (AdaBoost) algorithm, is adopted for both classification and regression (prediction) problems. Totally 254 cyclic loading tests of RC columns are collected. The geometric dimensions, reinforcing details, material properties are set as the input variables, while the failure types (for classification problem) and peak capacity forces (for regression problem) are set as the output variables. The results indicate that the model generated by the AdaBoost learning algorithm has a very high accuracy for both FM classification (accuracy = 0.96) and capacity prediction (R² = 0.98). Different learning algorithms are also compared and the results show that ensemble learning (especially AdaBoost) has better performance than single learning. In addition, the bearing capacity predicted by the AdaBoost is also compared to that by the empirical formulas provided by the design codes, which shows an obvious superior of the proposed method. In summary, the machine learning technique, especially the ensemble learning, can provide an alternate to the conventional mechanics-driven models in structural design in this big data time.     

利用粗糙集理论提高SVM预测系统的实时性

支持向量机是一种新的机器学习方法,它具有良好的推广性和分类精确性。但是在利用支持向量机的分类算法处理实际问题时,该算法的计算速度较慢、处理问题效率较低。文中介绍了一种新的学习算法,就是将粗糙集和支持向量机相结合,利用粗糙集对支持向量机的训练样本进行预处理,从而缩短样本的训练时间,提高基于SVM预测系统实时性。文中最后利用该方法进行了数据试验,试验结果表明了该方法可以大大缩短样本的训练时间,提高基于支持向量机处理预测系统的效率。从而也证明了该方法的有效性。     

GA_SJ在SVM核参数优化中的应用

支持向量机（SVM）是一种性能良好的机器学习方法,但是对于其参数的选择还缺少系统的理论作为指导。针对经典的SVM参数选择方法--遗传算法的一些不足,提出了改进,并将其与SVM相结合,得到自动选择核参数并进行SVM训练的算法即GA_SJ算法。该算法通过将随机搜索引入到遗传算法当中,并采用最优保存策略和动态的交叉和变异概率,有效地提高了遗传算法的效率。数值实验结果证实了GA_SJ算法在SVM参数优化中的可行性和有效性,而且得到的SVM具有较高的分类性能。