A comparative study of classifier ensembles for bankruptcy prediction

The aim of bankruptcy prediction in the areas of data mining and machine learning is to develop an effective model which can provide the higher prediction accuracy. In the prior literature, various classification techniques have been developed and studied, in/with which classifier ensembles by combining multiple classifiers approach have shown their outperformance over many single classifiers. However, in terms of constructing classifier ensembles, there are three critical issues which can affect their performance. The first one is the classification technique actually used/adopted, and the other two are the combination method to combine multiple classifiers and the number of classifiers to be combined, respectively. Since there are limited, relevant studies examining these aforementioned disuses, this paper conducts a comprehensive study of comparing classifier ensembles by three widely used classification techniques including multilayer perceptron (MLP) neural networks, support vector machines (SVM), and decision trees (DT) based on two well-known combination methods including bagging and boosting and different numbers of combined classifiers. Our experimental results by three public datasets show that DT ensembles composed of 80–100 classifiers using the boosting method perform best. The Wilcoxon signed ranked test also demonstrates that DT ensembles by boosting perform significantly different from the other classifier ensembles. Moreover, a further study over a real-world case by a Taiwan bankruptcy dataset was conducted, which also demonstrates the superiority of DT ensembles by boosting over the others.     

Improving experimental studies about ensembles of classifiers for bankruptcy prediction and credit scoring

Previous studies about ensembles of classifiers for bankruptcy prediction and credit scoring have been presented. In these studies, different ensemble schemes for complex classifiers were applied, and the best results were obtained using the Random Subspace method. The Bagging scheme was one of the ensemble methods used in the comparison. However, it was not correctly used. It is very important to use this ensemble scheme on weak and unstable classifiers for producing diversity in the combination. In order to improve the comparison, Bagging scheme on several decision trees models is applied to bankruptcy prediction and credit scoring. Decision trees encourage diversity for the combination of classifiers. Finally, an experimental study shows that Bagging scheme on decision trees present the best results for bankruptcy prediction and credit scoring.     

Combining cluster analysis with classifier ensembles to predict financial distress

The ability to accurately predict business failure is a very important issue in financial decision-making. Incorrect decision-making in financial institutions is very likely to cause financial crises and distress. Bankruptcy prediction and credit scoring are two important problems facing financial decision support. As many related studies develop financial distress models by some machine learning techniques, more advanced machine learning techniques, such as classifier ensembles and hybrid classifiers, have not been fully assessed. The aim of this paper is to develop a novel hybrid financial distress model based on combining the clustering technique and classifier ensembles. In addition, single baseline classifiers, hybrid classifiers, and classifier ensembles are developed for comparisons. In particular, two clustering techniques, Self-Organizing Maps (SOMs) and k-means and three classification techniques, logistic regression, multilayer-perceptron (MLP) neural network, and decision trees, are used to develop these four different types of bankruptcy prediction models. As a result, 21 different models are compared in terms of average prediction accuracy and Type I & II errors. By using five related datasets, combining Self-Organizing Maps (SOMs) with MLP classifier ensembles performs the best, which provides higher predication accuracy and lower Type I & II errors.     

基于神经网络集成的软件故障预测及实验分析

软件系统故障预测是软件测试过程中软件可靠性研究的重点之一。利用软件系统测试过程中前期的故障相关信息进行建模,预测后期的软件故障信息,以便于后期测试和验证资源的合理分配。根据软件测试过程中已知的软件故障时间序列,利用非齐次泊松分布过程、神经网络、神经网络集成等方法对其进行建模。通过对三个实例分别建模,其预测平均相对误差G-O模型依次为3.02%、5.88%和6.58%,而神经网络集成模型为0.19%、1.88%和1.455%,实验结果表明神经网络集成模型具有更精确的预测能力。     

A two-stage hybrid credit scoring model using artificial neural networks and multivariate adaptive regression splines

The objective of the proposed study is to explore the performance of credit scoring using a two-stage hybrid modeling procedure with artificial neural networks and multivariate adaptive regression splines (MARS). The rationale under the analyses is firstly to use MARS in building the credit scoring model, the obtained significant variables are then served as the input nodes of the neural networks model. To demonstrate the effectiveness and feasibility of the proposed modeling procedure, credit scoring tasks are performed on one bank housing loan dataset using cross-validation approach. As the results reveal, the proposed hybrid approach outperforms the results using discriminant analysis, logistic regression, artificial neural networks and MARS and hence provides an alternative in handling credit scoring tasks.     

Functional-link net with fuzzy integral for bankruptcy prediction

The classification ability of a single-layer perceptron could be improved by considering some enhanced features. In particular, this form of neural networks is called a functional-link net. In the output neuron's activation function, such as the sigmoid function, an inner product of a connection weight vector with an input vector is computed. However, since the input features are not independent of each other for the enhanced pattern, an assumption of the additivity is not reasonable. This paper employs a non-additive technique, namely the fuzzy integral, to aggregate performance values for an input pattern by interpreting each of the connection weights as a fuzzy measure of the corresponding feature. A learning algorithm with the genetic algorithm is then designed to automatically find connection weights. The sample data for bankruptcy analysis obtained from Moody's Industrial Manuals is considered to examine the classification ability of the proposed method. The results demonstrate that the proposed method performs well in comparison with traditional functional-link net and multivariate techniques.     

An artificial immune classifier for credit scoring analysis

The primary concern of the rating policies for a banking industry is to develop a more objective, accurate and competitive scoring model to avoid losses from potential bad debt. This study proposes an artificial immune classifier based on the artificial immune network (named AINE-based classifier) to evaluate the applicants’ credit scores. Two experimental credit datasets are used to show the accuracy rate of the artificial immune classifier. The ten-fold cross-validation method is applied to evaluate the performance of the classifier. The classifier is compared with other data mining techniques. Experimental results show that for the AINE-based classifier in credit scoring is more competitive than the SVM and hybrid SVM-based classifiers, except the BPN classifier. We further compare our classifier with other three AIS-based classifiers in the benchmark datasets, and show that the AINE-based classifier can rival the AIRS-based classifiers and outperforms the SAIS classifier when the number of attributes and classes increase. Our classifier can provide the credit card issuer with accurate and valuable information of credit scoring analyses to avoid making incorrect decisions that result in the loss of applicants’ bad debt.     

A hybrid neural network approach for credit scoring

The development of an effective credit scoring model has become a very important issue as the credit industry is confronted with ever‐intensifying competition and aggravating bad debt problems. During the past few years, a substantial number of studies in the field of statistics have been conducted to improve the accuracy of credit scoring models. In order to refine the classification and decrease misclassification, this paper presents a two‐stage model. Focusing on classification, the first stage aims at constructing an artificial neural network (ANN)‐based credit scoring model to categorize applicants into the group of accepted (good) credit and the group of rejected (bad) credit. Switching from classification to reassignment, the second stage proceeds to reduce the Type I error by retrieving the originally rejected good credit applicants to conditional acceptance using the Case‐Based Reasoning (CBR) classification technique. The proposed model (RST–ANN–CBR) is applied to a credit card dataset to verify its effectiveness. As the results indicate, the proposed model is able to achieve more accurate credit scoring than four other methods; more importantly, it is validated to recover potentially lost customers and to increase business revenues.     

Artificial neural networks and bankruptcy forecasting: a state of the art

The use of neural networks in finance began by the end of the 1980s and by the beginning of the 1990s, it developed specific applications related to forecasting on the failure of companies. In order to highlight the evolution of this research stream, we have retained and analysed 30 studies in which the authors use neural networks to solve companies’ classification problems (healthy and failing firms). This review of all these works gives us the opportunity to stress upon future trends in bankruptcy forecasting research.

Comparative analysis of data mining methods for bankruptcy prediction

A great deal of research has been devoted to prediction of bankruptcy, to include application of data mining. Neural networks, support vector machines, and other algorithms often fit data well, but because of lack of comprehensibility, they are considered black box technologies. Conversely, decision trees are more comprehensible by human users. However, sometimes far too many rules result in another form of incomprehensibility. The number of rules obtained from decision tree algorithms can be controlled to some degree through setting different minimum support levels. This study applies a variety of data mining tools to bankruptcy data, with the purpose of comparing accuracy and number of rules. For this data, decision trees were found to be relatively more accurate compared to neural networks and support vector machines, but there were more rule nodes than desired. Adjustment of minimum support yielded more tractable rule sets.     

Neural nets versus conventional techniques in credit scoring in Egyptian banking

Neural nets have become one of the most important tools using in credit scoring. Credit scoring is regarded as a core appraised tool of commercial banks during the last few decades. The purpose of this paper is to investigate the ability of neural nets, such as probabilistic neural nets and multi-layer feed-forward nets, and conventional techniques such as, discriminant analysis, probit analysis and logistic regression, in evaluating credit risk in Egyptian banks applying credit scoring models. The credit scoring task is performed on one bank’s personal loans’ data-set. The results so far revealed that the neural nets-models gave a better average correct classification rate than the other techniques. A one-way analysis of variance and other tests have been applied, demonstrating that there are some significant differences amongst the means of the correct classification rates, pertaining to different techniques.     

A data driven ensemble classifier for credit scoring analysis

This study focuses on predicting whether a credit applicant can be categorized as good, bad or borderline from information initially supplied. This is essentially a classification task for credit scoring. Given its importance, many researchers have recently worked on an ensemble of classifiers. However, to the best of our knowledge, unrepresentative samples drastically reduce the accuracy of the deployment classifier. Few have attempted to preprocess the input samples into more homogeneous cluster groups and then fit the ensemble classifier accordingly. For this reason, we introduce the concept of class-wise classification as a preprocessing step in order to obtain an efficient ensemble classifier. This strategy would work better than a direct ensemble of classifiers without the preprocessing step. The proposed ensemble classifier is constructed by incorporating several data mining techniques, mainly involving optimal associate binning to discretize continuous values; neural network, support vector machine, and Bayesian network are used to augment the ensemble classifier. In particular, the Markov blanket concept of Bayesian network allows for a natural form of feature selection, which provides a basis for mining association rules. The learned knowledge is represented in multiple forms, including causal diagram and constrained association rules. The data driven nature of the proposed system distinguishes it from existing hybrid/ensemble credit scoring systems.     

Probabilistic modeling and visualization for bankruptcy prediction

In accounting and finance domains, bankruptcy prediction is of great utility for all of the economic stakeholders. The challenge of accurate assessment of business failure prediction, specially under scenarios of financial crisis, is known to be complicated. Although there have been many successful studies on bankruptcy detection, seldom probabilistic approaches were carried out. In this paper we assume a probabilistic point-of-view by applying Gaussian processes (GP) in the context of bankruptcy prediction, comparing it against the support vector machines (SVM) and the logistic regression (LR). Using real-world bankruptcy data, an in-depth analysis is conducted showing that, in addition to a probabilistic interpretation, the GP can effectively improve the bankruptcy prediction performance with high accuracy when compared to the other approaches. We additionally generate a complete graphical visualization to improve our understanding of the different attained performances, effectively compiling all the conducted experiments in a meaningful way. We complete our study with an entropy-based analysis that highlights the uncertainty handling properties provided by the GP, crucial for prediction tasks under extremely competitive and volatile business environments.     

Hybrid mining approach in the design of credit scoring models

Unrepresentative data samples are likely to reduce the utility of data classifiers in practical application. This study presents a hybrid mining approach in the design of an effective credit scoring model, based on clustering and neural network techniques. We used clustering techniques to preprocess the input samples with the objective of indicating unrepresentative samples into isolated and inconsistent clusters, and used neural networks to construct the credit scoring model. The clustering stage involved a class-wise classification process. A self-organizing map clustering algorithm was used to automatically determine the number of clusters and the starting points of each cluster. Then, the K-means clustering algorithm was used to generate clusters of samples belonging to new classes and eliminate the unrepresentative samples from each class. In the neural network stage, samples with new class labels were used in the design of the credit scoring model. The proposed method demonstrates by two real world credit data sets that the hybrid mining approach can be used to build effective credit scoring models.     

Threshold accepting trained principal component neural network and feature subset selection: Application to bankruptcy prediction in banks

This paper proposes an application of new principal component neural network (PCNN) architecture to bankruptcy prediction problem in commercial banks. Further, a new feature subset selection (FSS) algorithm is proposed. In this architecture, the hidden layer is completely replaced by what is referred to as a ‘principal component layer’. This layer consists of a few selected principal components that perform the function of hidden nodes. Moreover, this study proposes an algorithm based on the threshold accepting (TA) meta-heuristic to train the PCNN. The architecture reduces the number of weights by a great number as there are no formal connections between the input layer and the principal component layer. The efficacy of the algorithm is tested on the Spanish banks dataset and Turkish banks dataset. The results showed high generalization power of PCNN in the 10-fold cross-validation and also the feature subsets selected in each of the examples showed high discriminating power. PCNN is also compared with PCA-TANN and PCA-BPNN, which have PCA as the preprocessor and have one hidden layer each. Further comparisons are also made with TANN and BPNN. All these classifiers are compared with respect to the AUC (area under the receiver operating characteristic (ROC) curve) criterion. ROC curve is drawn for each classifier with sensitivity on the X-axis and one-specificity on the Y-axis. Based on the experiments conducted, it is inferred that the proposed PCNN hybrids outperformed other classifiers in terms of AUC. It is also observed that the proposed feature subset selection algorithm is very stable and powerful.     

Least squares support vector machines ensemble models for credit scoring

Due to recent financial crisis and regulatory concerns of Basel II, credit risk assessment is becoming one of the most important topics in the field of financial risk management. Quantitative credit scoring models are widely used tools for credit risk assessment in financial institutions. Although single support vector machines (SVM) have been demonstrated with good performance in classification, a single classifier with a fixed group of training samples and parameters setting may have some kind of inductive bias. One effective way to reduce the bias is ensemble model. In this study, several ensemble models based on least squares support vector machines (LSSVM) are brought forward for credit scoring. The models are tested on two real world datasets and the results show that ensemble strategies can help to improve the performance in some degree and are effective for building credit scoring models.     

Managing loan customers using misclassification patterns of credit scoring model

A number of credit scoring models have been developed to evaluate credit risk of new loan applicants and existing loan customers, respectively. This study proposes a method to manage existing customers by using misclassification patterns of credit scoring model. We divide two groups of customers, the currently good and bad credit customers, into two subgroups, respectively, according to whether their credit status is misclassified or not by the neural network model. In addition, we infer the characteristics of each subgroup and propose management strategies corresponding to each subgroup.     

构造性层状神经网络集成方法

在分析构造性神经网络集成和层状神经网络集成方法的基础上,提出了一种构造性层状神经网络集成方法。该方法自动确定神经网络集成中成员神经网络的数目,以及成员神经网络的结构等。集成在保证成员神经网络精度的同时,又保证了成员网络之间的差异度。用户只需要简单定义一些参数,就可以构造出性能较好的神经网络集成。     

Credit scoring and rejected instances reassigning through evolutionary computation techniques

The credit industry is concerned with many problems of interest to the computation community. This study presents a work involving two interesting credit analysis problems and resolves them by applying two techniques, neural networks (NNs) and genetic algorithms (GAs), within the field of evolutionary computation. The first problem is constructing NN-based credit scoring model, which classifies applicants as accepted (good) or rejected (bad) credits. The second one is better understanding the rejected credits, and trying to reassign them to the preferable accepted class by using the GA-based inverse classification technique. Each of these problems influences on the decisions relating to the credit admission evaluation, which significantly affects risk and profitability of creditors. From the computational results, NNs have emerged as a computational tool that is well-matched to the problem of credit classification. Using the GA-based inverse classification, creditors can suggest the conditional acceptance, and further explain the conditions to rejected applicants. In addition, applicants can evaluate the option of minimum modifications to their attributes.     

Incremental construction of classifier and discriminant ensembles

We discuss approaches to incrementally construct an ensemble. The first constructs an ensemble of classifiers choosing a subset from a larger set, and the second constructs an ensemble of discriminants, where a classifier is used for some classes only. We investigate criteria including accuracy, significant improvement, diversity, correlation, and the role of search direction. For discriminant ensembles, we test subset selection and trees. Fusion is by voting or by a linear model. Using 14 classifiers on 38 data sets, incremental search finds small, accurate ensembles in polynomial time. The discriminant ensemble uses a subset of discriminants and is simpler, interpretable, and accurate. We see that an incremental ensemble has higher accuracy than bagging and random subspace method; and it has a comparable accuracy to AdaBoost, but fewer classifiers.