Efficiently processing deterministic approximate aggregation query on massive data

Financial distress prediction is very important to financial institutions who must be able to make critical decisions regarding customer loans. Bankruptcy prediction and credit scoring are the two main aspects considered in financial distress prediction. To assist in this determination, thereby lowering the risk borne by the financial institution, it is necessary to develop effective prediction models for prediction of the likelihood of bankruptcy and estimation of credit risk. A number of financial distress prediction models have been constructed, which utilize various machine learning techniques, such as single classifiers and classifier ensembles, but improving the prediction accuracy is the major research issue. In addition, aside from improving the prediction accuracy, there have been very few studies that specifically consider lowering the Type I error. In practice, Type I errors need to receive careful consideration during model construction because they can affect the cost to the financial institution. In this study, we introduce a classifier ensemble approach designed to reduce the misclassification cost. The outputs produced by multiple classifiers are combined by utilizing the unanimous voting (UV) method to find the final prediction result. Experimental results obtained based on four relevant datasets show that our UV ensemble approach outperforms the baseline single classifiers and classifier ensembles. Specifically, the UV ensemble not only provides relatively good prediction accuracy and minimizes Type I/II errors, but also produces the smallest misclassification cost.     

Aggregating multiple classification results using Choquet integral for financial distress early warning

Financial distress prediction methods based on combination classifier become a rising trend in this field. This paper applies Choquet integral to ensemble single classifiers and proposes a Choquet integral-based combination classifier for financial distress early warning. Also, as the conditions between training and pattern recognition cannot be completely consistent, so this paper proposes an adaptive fuzzy measure by using the dynamic information in the single classifier pattern recognition results which is more reasonable than the static prior fuzzy density. Finally, a comparative analysis based on Chinese listed companies’ real data is conducted to verify prediction accuracy and stability of the combination classifier. The experiment results indicate that financial distress prediction using Choquet integral-based combination classifier has higher average accuracy and stability than single classifiers.     

基于混合多距离度量的多分类器加权集成研究

在机器学习领域,分类器加权在小样本数据集中的分类正确率较低。为此,提出一种基于混合距离度量的多分类器加权集成方法。结合欧氏距离、曼哈顿距离、切比雪夫距离,设计混合的距离度量加权方法,使用加权投票组合规则集成各分类器的输出结果。实验结果表明,该方法鲁棒性较好,分类正确率较高。     

An integrated TOPSIS crow search based classifier ensemble: In application to stock index price movement prediction

Predicting future stock index price movement has always been a fascinating research area both for the investors who wish to yield a profit by trading stocks and for the researchers who attempt to expose the buried information from the complex stock market time series data. This prediction problem can be addressed as a binary classification problem with two class labels, one for the increasing movement and other for the decreasing movement. In literature, a wide range of classifiers has been tested for this application. As the performance of individual classifier varies for a diverse dataset with respect to different performance measures, it is impractical to acknowledge a specific classifier to be the best one. Hence, designing an efficient classifier ensemble instead of an individual classifier is fetching increasing attention from many researchers. Again selection of base classifiers and deciding their preferences in ensemble with respect to a variety of performance criteria can be considered as a Multi Criteria Decision Making (MCDM) problem. In this paper, an integrated TOPSIS Crow Search based weighted voting classifier ensemble is proposed for stock index price movement prediction. Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS), one of the popular MCDM techniques, is suggested for ranking and selecting a set of base classifiers for the ensemble whereas the weights of the classifiers used in the ensemble are tuned by the Crow Search method. The proposed ensemble model is validated for prediction of stock index price over the historical prices of BSE SENSEX, S&P500 and NIFTY 50 stock indices. The model has shown better performance compared to individual classifiers and other ensemble models such as majority voting, weighted voting, differential evolution and particle swarm optimization based classifier ensemble.     

MCELCCh-FDP: Financial distress prediction with classifier ensembles based on firm life cycle and Choquet integral

Financial distress prediction (FDP) has always been an important issue in the business and financial management. This research proposed a novel multiple classifier ensemble model based on firm life cycle and Choquet integral for FDP, named MCELCCh-FDP, as a new approach to tackle with financial distress. Empirical study based on Chinese listed companies’ real data is conducted, and the results show that the proposed MCELCCh-FDP model has higher prediction accuracy than single classifiers. In order to verify the prediction capability of firm life cycle and Choquet integral in FDP model, comparative analysis is conducted. The experiment results indicate that the introduction of firm life cycle and Choquet integral in FDP can greatly enhance prediction accuracy.     

Combining cluster analysis with classifier ensembles to predict financial distress

The ability to accurately predict business failure is a very important issue in financial decision-making. Incorrect decision-making in financial institutions is very likely to cause financial crises and distress. Bankruptcy prediction and credit scoring are two important problems facing financial decision support. As many related studies develop financial distress models by some machine learning techniques, more advanced machine learning techniques, such as classifier ensembles and hybrid classifiers, have not been fully assessed. The aim of this paper is to develop a novel hybrid financial distress model based on combining the clustering technique and classifier ensembles. In addition, single baseline classifiers, hybrid classifiers, and classifier ensembles are developed for comparisons. In particular, two clustering techniques, Self-Organizing Maps (SOMs) and k-means and three classification techniques, logistic regression, multilayer-perceptron (MLP) neural network, and decision trees, are used to develop these four different types of bankruptcy prediction models. As a result, 21 different models are compared in terms of average prediction accuracy and Type I & II errors. By using five related datasets, combining Self-Organizing Maps (SOMs) with MLP classifier ensembles performs the best, which provides higher predication accuracy and lower Type I & II errors.     

Financial distress prediction using support vector machines: Ensemble vs. individual

Financial distress prediction (FDP) is of great importance to both inner and outside parts of companies. Though lots of literatures have given comprehensive analysis on single classifier FDP method, ensemble method for FDP just emerged in recent years and needs to be further studied. Support vector machine (SVM) shows promising performance in FDP when compared with other single classifier methods. The contribution of this paper is to propose a new FDP method based on SVM ensemble, whose candidate single classifiers are trained by SVM algorithms with different kernel functions on different feature subsets of one initial dataset. SVM kernels such as linear, polynomial, RBF and sigmoid, and the filter feature selection/extraction methods of stepwise multi discriminant analysis (MDA), stepwise logistic regression (logit), and principal component analysis (PCA) are applied. The algorithm for selecting SVM ensemble's base classifiers from candidate ones is designed by considering both individual performance and diversity analysis. Weighted majority voting based on base classifiers’ cross validation accuracy on training dataset is used as the combination mechanism. Experimental results indicate that SVM ensemble is significantly superior to individual SVM classifier when the number of base classifiers in SVM ensemble is properly set. Besides, it also shows that RBF SVM based on features selected by stepwise MDA is a good choice for FDP when individual SVM classifier is applied.     

基于异构分类器集成的增量学习算法

将集成学习的思想引入到增量学习之中可以显著提升学习效果,近年关于集成式增量学习的研究大多采用加权投票的方式将多个同质分类器进行结合,并没有很好地解决增量学习中的稳定-可塑性难题。针对此提出了一种异构分类器集成增量学习算法。该算法在训练过程中,为使模型更具稳定性,用新数据训练多个基分类器加入到异构的集成模型之中,同时采用局部敏感哈希表保存数据梗概以备待测样本近邻的查找;为了适应不断变化的数据,还会用新获得的数据更新集成模型中基分类器的投票权重;对待测样本进行类别预测时,以局部敏感哈希表中与待测样本相似的数据作为桥梁,计算基分类器针对该待测样本的动态权重,结合多个基分类器的投票权重和动态权重判定待测样本所属类别。通过对比实验,证明了该增量算法有比较高的稳定性和泛化能力。     

Analysis of a Plurality Voting-based Combination of Classifiers

In various studies, it has been demonstrated that combining the decisions of multiple classifiers can lead to better recognition result. Plurality voting is one of the most widely used combination strategies. In this paper, we both theoretically and experimentally analyze the performance of a plurality voting based ensemble classifier. Theoretical expressions for system performance are derived as a function of the model parameters: N (number of classifiers), m (number of classes), and p (probability that a single classifier is correct). Experimental results on the human face recognition problem show that the voting strategy can successfully achieve high detection and identification rates, and, simultaneously, low false acceptance rates.     

Using neural network ensembles for bankruptcy prediction and credit scoring

Bankruptcy prediction and credit scoring have long been regarded as critical topics and have been studied extensively in the accounting and finance literature. Artificial intelligence and machine learning techniques have been used to solve these financial decision-making problems. The multilayer perceptron (MLP) network trained by the back-propagation learning algorithm is the mostly used technique for financial decision-making problems. In addition, it is usually superior to other traditional statistical models. Recent studies suggest combining multiple classifiers (or classifier ensembles) should be better than single classifiers. However, the performance of multiple classifiers in bankruptcy prediction and credit scoring is not fully understood. In this paper, we investigate the performance of a single classifier as the baseline classifier to compare with multiple classifiers and diversified multiple classifiers by using neural networks based on three datasets. By comparing with the single classifier as the benchmark in terms of average prediction accuracy, the multiple classifiers only perform better in one of the three datasets. The diversified multiple classifiers trained by not only different classifier parameters but also different sets of training data perform worse in all datasets. However, for the Type I and Type II errors, there is no exact winner. We suggest that it is better to consider these three classifier architectures to make the optimal financial decision.     

基于小波包和组合分类器的脑电信号分类

为了提高脑思维任务分类精度,提出了一种基于小波包分解和多分类器投票组合的运动想象任务分类方法。该方法利用小波包分解对经过预处理的脑电信号进行分解,提取所有频带上的相对小波包能量特征;根据不同脑思维任务下左右半脑各通道间的差异性对C3、C4两通道求取特定频带上的小波包系数的L-2范数作为特征;采用基于投票策略的组合分类器对两种联合特征进行分类,得到了92.85%的识别精度。实验结果表明,联合特征向量较好地反映了左右手运动想象脑电信号的事件相关去同步（ERD）和事件相关同步（ERS）的本质特性;组合分类器识别效果优于单一分类器。     

多类型分类器融合的文本分类方法研究

传统的文本分类方法大多数使用单一的分类器,而不同的分类器对分类任务的侧重点不同,就使得单一的分类方法有一定的局限性,同时每个特征提取方法对特征词的考虑角度不同。针对以上问题,提出了多类型分类器融合的文本分类方法。该模型使用了word2vec、主成分分析、潜在语义索引以及TFIDF特征提取方法作为多类型分类器融合的特征提取方法。并在多类型分类器加权投票方法中忽略了类别信息的问题,提出了类别加权的分类器权重计算方法。通过实验结果表明,多类型分类器融合方法在二元语料库、多元语料库以及特定语料库上都取得了很好的性能,类别加权的分类器权重计算方法比多类型分类器融合方法在分类性能方面提高了1.19%。     

Risk prediction of product‐harm events using rough sets and multiple classifier fusion: an experimental study of listed companies in China

With the increasing of frequency and destructiveness of product‐harm events, study on enterprise crisis management becomes essentially important, but little literature thoroughly explores the risk prediction method of product‐harm event. In this study, an initial index system for risk prediction was built based on the analysis of the key drivers of the product‐harm event's evolution; ultimately, nine risk‐forecasting indexes were obtained using rough set attribute reduction. With the four indexes of cumulative abnormal returns as the input, fuzzy clustering was used to classify the risk level of a product‐harm event into four grades. In order to control the uncertainty and instability of single classifiers in risk prediction, multiple classifier fusion was introduced and combined with self‐organising data mining (SODM). Further, an SODM‐based multiple classifier fusion (SB‐MCF) model was presented for the risk prediction related to a product‐harm event. The experimental results based on 165 Chinese listed companies indicated that the SB‐MCF model improved the average predictive accuracy and reduced variation degree simultaneously. The statistical analysis demonstrated that the SB‐MCF model significantly outperformed six widely used single classification models (e.g. neural networks, support vector machine, and case‐based reasoning) and other six commonly used multiple classifier fusion methods (e.g. majority voting, Bayesian method, and genetic algorithm).     

基于PSO拓展的多分类器加权集成方法

在多分类器集成时,每个基分类器的效能不同,如每个权值都相同,则会影响基分类器发挥作用。基于此,提出基于PSO拓展的多分类器加权集成方法BCPSO。该方法采用随机子空间生成各个独立的子分类器,输出结果通过各分类器加权投票组合规则集成。实验结果表明,该方法有效可行,具有较高的分类正确率。     

AdaBoost ensemble for financial distress prediction: An empirical comparison with data from Chinese listed companies

Due to the important role of financial distress prediction (FDP) for enterprises, it is crucial to improve the accuracy of FDP model. In recent years, classifier ensemble has shown promising advantage over single classifier, but the study on classifier ensemble methods for FDP is still not comprehensive enough and leaves to be further explored. This paper constructs AdaBoost ensemble respectively with single attribute test (SAT) and decision tree (DT) for FDP, and empirically compares them with single DT and support vector machine (SVM). After designing the framework of AdaBoost ensemble method for FDP, the article describes AdaBoost algorithm as well as SAT and DT algorithm in detail, which is followed by the combination mechanism of multiple classifiers. On the initial sample of 692 Chinese listed companies and 41 financial ratios, 30 times of holdout experiments are carried out for FDP respectively one year, two years, and three years in advance. In terms of experimental results, AdaBoost ensemble with SAT outperforms AdaBoost ensemble with DT, single DT classifier and single SVM classifier. As a conclusion, the choice of weak learner is crucial to the performance of AdaBoost ensemble, and AdaBoost ensemble with SAT is more suitable for FDP of Chinese listed companies.     

COMBINING MULTIPLE CLASSIFIERS USING DEMPSTER'S RULE FOR TEXT CATEGORIZATION

In this paper we investigate the combination of four machine learning methods for text categorization using Dempster's rule of combination. These methods include Support Vector Machine (SVM), kNN (Nearest Neighbor), kNN model-based approach (kNNM), and Rocchio. We first present a general representation of the outputs of different classifiers, in particular, modeling it as a piece of evidence by using a novel evidence structure called focal element triplet. Furthermore, we investigate an effective method for combining pieces of evidence derived from classifiers generated by a 10-fold cross-validation. Finally, we evaluate our methods on the 20-newsgroup and Reuters-21578 benchmark data sets and perform the comparative analysis with majority voting in combining multiple classifiers along with the previous result. Our experimental results show that the best combined classifier can improve the performance of the individual classifiers and Dempster's rule of combination outperforms majority voting in combining multiple classifiers.     

A Theoretical Analysis of the Limits of Majority Voting Errors for Multiple Classifier Systems

: A robust character of combining diverse classifiers using a majority voting has recently been illustrated in the pattern recognition literature. Furthermore, negatively correlated classifiers turned out to offer further improvement of the majority voting performance even comparing to the idealised model with independent classifiers. However, negatively correlated classifiers represent a very unlikely situation in real-world classification problems, and their benefits usually remain out of reach. Nevertheless, it is theoretically possible to obtain a 0% majority voting error using a finite number of classifiers at error levels lower than 50%. We attempt to show that structuring classifiers into relevant multistage organisations can widen this boundary, as well as the limits of majority voting error, even more. Introducing discrete error distributions for analysis, we show how majority voting errors and their limits depend upon the parameters of a multiple classifier system with hardened binary outputs (correct/incorrect). Moreover, we investigate the sensitivity of boundary distributions of classifier outputs to small discrepancies modelled by the random changes of votes, and propose new more stable patterns of boundary distributions. Finally, we show how organising classifiers into different structures can be used to widen the limits of majority voting errors, and how this phenomenon can be effectively exploited. Received: 17 November 2000, Received in revised form: 27 November 2001, Accepted: 29 November 2001 ID="A1" Correspondence and offprint requests to: D. Ruta, Applied Computing Research Unit, Division of Computer and Information Systems, University of Paisley, High Street, Paisley PA1 2BE, UK. Email: ruta-ci0@paisley.ac.uk     

基于多分类器集成的维吾尔文联机手写字母识别

多分类器组合能够在一定程度上弥补单个分类器的缺陷,因此它在模式识别中得到了广泛应用。深入调研国内外联机手写识别技术的研究动态,结合维吾尔文字母的独特书写风格,研究了基于多分类器集成的维吾尔语联机手写字母识别。利用5种不同的特征提取方法构造了5个独立的维吾尔语字母分类识别器,采用了等权投票和不等权投票等两种策略将5种维吾尔语字母分类识别器进行了有效组合。其中,单分类器采用了基于动态时间弯折（DTW）匹配距离的最近邻分类方法。实验结果表明,提出的集成策略的识别率明显高于单分类器的识别率,而且为特征的综合集成提供了多种有效途径。     

基于多分类器集成的“北京一号”小卫星遥感影像分类研究

将多分类器集合应用于"北京一号"小卫星多光谱遥感数据土地覆盖分类,首先构建分类器集合,应用最小距离分类、最大似然分类、支持向量机(SVM)、BP神经网络、RBF神经网络和决策树等进行土地覆盖分类,然后利用Bagging、Boosting、投票法、证据理论和模糊积分法等分类器集成方法,得到综合不同分类器输出的最终分类结果。试验表明,多分类器集成能够有效提高"北京一号"小卫星土地覆盖分类的精度,具有广泛的应用前景。     

动态加权投票的多分类器聚合

在使用多分类器系统时,一种流行的方法是采用简单的多数投票策略来聚合多分类器。然而,当各个独立的分类器的性能不统一时,这种简单的多数投票规则会对分类结果造成负面影响。引入一种新的动态加权函数来聚合多个分类器,动态加权函数通过增加分类结果距离样本最近的分类器的权值来提高分类器的性能。在UCI机器学习数据库中的几个现实问题数据集上的实验结果表明动态加权的多分类器聚合方法比简单的多数投票方法能取得更好的分类结果。