A new genetic feature selection with neural network ensemble

A neural network ensemble is a learning paradigm in which a finite collection of neural networks is trained for the same task. Ensembles generally show better classification and generalization performance than a single neural network does. In this paper, a new feature selection method for a neural network ensemble is proposed for pattern classification. The proposed method selects an adequate feature subset for each constituent neural network of the ensemble using a genetic algorithm. Unlike the conventional feature selection method, each neural network is only allowed to have some (not all) of the considered features. The proposed method can therefore be applied to huge-scale feature classification problems. Experiments are performed with four databases to illustrate the performance of the proposed method.     

Software reliability prediction by soft computing techniques

In this paper, ensemble models are developed to accurately forecast software reliability. Various statistical (multiple linear regression and multivariate adaptive regression splines) and intelligent techniques (backpropagation trained neural network, dynamic evolving neuro-fuzzy inference system and TreeNet) constitute the ensembles presented. Three linear ensembles and one non-linear ensemble are designed and tested. Based on the experiments performed on the software reliability data obtained from literature, it is observed that the non-linear ensemble outperformed all the other ensembles and also the constituent statistical and intelligent techniques.     

Hybridization of autoregressive integrated moving average (ARIMA) with probabilistic neural networks (PNNs)

Autoregressive integrated moving average (ARIMA) models are one of the most important time series models applied in financial market forecasting over the past three decades. Improving forecasting especially time series forecasting accuracy is an important yet often difficult task facing forecasters. Both theoretical and empirical findings have indicated that integration of different models can be an effective way of improving upon their predictive performance, especially when the models in the ensemble are quite different. In the literature, several hybrid techniques have been proposed by combining different time series models together, in order to yield results that are more accurate. In this paper, a new hybrid model of the autoregressive integrated moving average (ARIMA) and probabilistic neural network (PNN), is proposed in order to yield more accurate results than traditional ARIMA models. In proposed model, the estimated values of the ARIMA model are modified based on the distinguished trend of the ARIMA residuals and optimum step length, which are respectively obtained from a probabilistic neural network and a mathematical programming model. Empirical results with three well-known real data sets indicate that the proposed model can be an effective way in order to construct a more accurate hybrid model than ARIMA model. Therefore, it can be used as an appropriate alternative model for forecasting tasks, especially when higher forecasting accuracy is needed.     

Predicting bank financial failures using neural networks,support vector machines and multivariate statistical methods: A comparative analysis in the sample of savings deposit insurance fund (SDIF) transferred banks in Turkey

Bank failures threaten the economic system as a whole. Therefore, predicting bank financial failures is crucial to prevent and/or lessen the incoming negative effects on the economic system. This is originally a classification problem to categorize banks as healthy or non-healthy ones. This study aims to apply various neural network techniques, support vector machines and multivariate statistical methods to the bank failure prediction problem in a Turkish case, and to present a comprehensive computational comparison of the classification performances of the techniques tested. Twenty financial ratios with six feature groups including capital adequacy, asset quality, management quality, earnings, liquidity and sensitivity to market risk (CAMELS) are selected as predictor variables in the study. Four different data sets with different characteristics are developed using official financial data to improve the prediction performance. Each data set is also divided into training and validation sets. In the category of neural networks, four different architectures namely multi-layer perceptron, competitive learning, self-organizing map and learning vector quantization are employed. The multivariate statistical methods; multivariate discriminant analysis, k-means cluster analysis and logistic regression analysis are tested. Experimental results are evaluated with respect to the correct accuracy performance of techniques. Results show that multi-layer perceptron and learning vector quantization can be considered as the most successful models in predicting the financial failure of banks.     

A hybrid financial analysis model for business failure prediction

Accounting frauds have continuously happened all over the world. This leads to the need of predicting business failures. Statistical methods and machine learning techniques have been widely used to deal with this issue. In general, financial ratios are one of the main inputs to develop the prediction models. This paper presents a hybrid financial analysis model including static and trend analysis models to construct and train a back-propagation neural network (BPN) model. Further, the experiments employ four datasets of Taiwan enterprises which support that the proposed model not only provides a high predication rate but also outperforms other models including discriminant analysis, decision trees, and the back-propagation neural network alone.     

A neuro-computational intelligence analysis of the ecological footprint of nations

The per capita ecological footprint (EF) is one of the most-widely recognized measures of environmental sustainability. It seeks to quantify the Earth’s biological capacity required to support human activity. This study uses three neuro-computational methodologies: multi-layer perceptron neural network (MLP), probabilistic neural network (PNN) and generalized regression neural network (GRNN) to predict and classify the EF of 140 nations. Accuracy indices are used to assess the prediction and classification accuracy of the three methodologies. The study shows that neuro-computational models outperform traditional statistical techniques such as regression analysis and discriminant analysis in predicting and classifying per capita EF due to their robustness and flexibility of modeling algorithms.     

Fuzzy polynomial neural networks for approximation of the compressive strength of concrete

The main purpose of this paper is to develop fuzzy polynomial neural networks (FPNN) to predict the compressive strength of concrete. Two different architectures of FPNN are addressed (Type1 and Type2) and their training methods are discussed. In this research, the proposed FPNN is a combination of fuzzy neural networks (FNNs) and polynomial neural networks (PNNs). Here, while the FNN demonstrates the premises (If-Part) of the fuzzy model, the PNN is implemented as its consequence (Then-Part). To enhance the performance of the network, back propagation (BP), and list square error (LSE) algorithms are utilized for the tuning of the system.Six different FPNN architectures are constructed, trained, and tested using the experimental data of 458 different concrete mix-designs collected from three distinct sources. The data are organized in a format of six input parameters of concrete ingredients and one output as 28-day compressive strength of the mix-design. Using root means square (RMS) and correlation factors (CFs), the models are evaluated and compared with training and testing data pairs. The results show that FPNN-Type1 has strong potential as a feasible tool for prediction of the compressive strength of concrete mix-design. However, the FPNN-Type2 is recognized as unfeasible model to this purpose.     

A Normalized Probabilistic Expectation-Maximization Neural Network for Minimizing Bayesian Misclassification Cost Risk

In this paper, we propose a normalized semi-supervised probabilistic expectation-maximization neural network (PEMNN) that minimizes Bayesian misclassification cost risk. Using simulated and real-world datasets, we compare the proposed PEMNN with supervised cost sensitive probabilistic neural network (PNN), discriminant analysis (DA), mathematical integer programming (MIP) model and support vector machines (SVM) for different misclassification cost asymmetries and class biases. The results of our experiments indicate that the PEMNN performs better when class data distributions are normal or uniform. However, when class data distribution is exponential the performance of PEMNN deteriorates giving slight advantage to competing MIP, DA, PNN and SVM techniques. For real-world data with non-parametric distributions and mixed decision-making attributes (continuous and categorical), the PEMNN outperforms the PNN.     

电子鼻模式识别算法的比较研究

文中比较了k-近邻法、线性判别分析、反向传播人工神经网络、概率神经网、学习向量量化以及自组织映射6种电子鼻模式识别算法的分类能力.采用了1个定量指标(识别精度)和4个定性指标(运算速度、训练速度、内存容量、抗干扰能力)对不同算法进行了系统比较.研究表明基于神经网络的模式识别算法比基于统计理论的模式识别算法具有更高的识别精度.如果同时考虑定性指标,当训练速度要求不高时,宜采用学习向量量化算法;能满足内存需求前提下,优先推荐采用概率神经网算法.对于选择性高的信号,采用线性判别分析可以达到最佳效果.     

软组合概率神经网络分类器人脸识别方法

概率神经网络分类器具有学习速度快、易于实现的特点,而且其输出是后验概率, 使得分类器的软组合变得容易。利用概率神经网络的这些特点,提出了软组合概率神经网络分类器人脸识别方法,该方法包括3步:(1)对人脸图像做不完全小波包分解;(2)用包含低频成分的小波子空间图像训练概率神经网络分类器;(3)用模糊积分组合训练好的分类器。将该方法与3种基于矩阵子空间的人脸识别方法在JAFFE、YALE、ORL和FERET 4个人脸数据库上进行了实验比较,结果表明,提出的方法在识别精度和CPU时间两方面均优于其他3种方法。     

A study of financial insolvency prediction model for life insurers

The objective of insurer supervision is to monitor the financial solvency of companies and to protect the rights of consumers. Improving the related legislation and regulatory policy are also the goals of supervision. The purpose of this study is to evaluate the financial soundness by using the rating systems of the CAMEL and the risk-based capital (RBC) models. Moreover, it is to explore whether insurers exit a significance difference of financial stability or not between domestic and foreign branch life insurers. This study constructed an efficient insolvency prediction model and showed that the artificial neural network was more excellent for classification than the traditional discriminant method since the artificial neural network’s accurate discrimination rate of 95.2% with a lower Type I error of 0.0274 and Type II error of 0.0769.     

PREDICTIVE MODELING WITH MISSING DATA USING AN AUTOMATIC RELEVANCE DETERMINATION ENSEMBLE: A COMPARATIVE STUDY

The objective of this article is to present an automatic relevance determination ensemble as an effective variable extraction method for insurance datasets with large numbers of variables. Automatic relevance determination is a method that uses a Bayesian neural network and the evidence framework to rank variables in the order of relevance to the target variable. The current approach uses a single Bayesian neural network that searches only for local minima or maxima. In large datasets with numerous variables, this is a concern because we cannot be certain that the outcome is an optimal one. The method used to address this issue in this study is an automatic relevance determination ensemble with various configurations (or structures) of the Bayesian neural networks. Each outcome in the ensemble is determined by using a confidence factor rather than by scrutinizing the most probable weights values or hyperparameters directly. The extraction method is used with the repeated incremental pruning to produce error reduction, logistic discriminant analysis, and k-nearest neighbor models to evaluate the performance. Furthermore, the datasets employed contain escalating missing data to measure the accuracy and resilience of the models when they are used with the proposed ensemble. The ensemble is compared with the principal component analysis method. The results show that with the automatic relevance determination ensemble, the models achieve higher accuracies in performance than when used with the principal component analysis. Furthermore, the resilience and strength of models is higher when using the ensemble, compared with the principal component analysis method.     

Recognition of Western style musical genres using machine learning techniques

This study uses machine learning techniques (ML) to classify and cluster different Western music genres. Three artificial neural network models (multi-layer perceptron neural network [MLP], probabilistic neural network [PNN]) and self-organizing maps neural network (SOM) along with support vector machines (SVM) are compared to two standard statistical methods (linear discriminant analysis [LDA] and cluster analysis [CA]). The variable sets considered are average frequencies, variance frequencies, maximum frequencies, amplitude or loudness of the sound and the median of the location of the 15 highest peaks in the periodogram. The results show that machine learning models outperform traditional statistical techniques in classifying and clustering different music genres due to their robustness and flexibility of modeling algorithms. The study also shows how it is possible to identify various dimensions of music genres by uncovering complex patterns in the multidimensional data.     

Probabilistic neural network training procedure based on Q(0)-learning algorithm in medical data classification

In this article, an iterative procedure is proposed for the training process of the probabilistic neural network (PNN). In each stage of this procedure, the Q(0)-learning algorithm is utilized for the adaptation of PNN smoothing parameter (σ). Four classes of PNN models are regarded in this study. In the case of the first, simplest model, the smoothing parameter takes the form of a scalar; for the second model, σ is a vector whose elements are computed with respect to the class index; the third considered model has the smoothing parameter vector for which all components are determined depending on each input attribute; finally, the last and the most complex of the analyzed networks, uses the matrix of smoothing parameters where each element is dependent on both class and input feature index. The main idea of the presented approach is based on the appropriate update of the smoothing parameter values according to the Q(0)-learning algorithm. The proposed procedure is verified on six repository data sets. The prediction ability of the algorithm is assessed by computing the test accuracy on 10 %, 20 %, 30 %, and 40 % of examples drawn randomly from each input data set. The results are compared with the test accuracy obtained by PNN trained using the conjugate gradient procedure, support vector machine algorithm, gene expression programming classifier, k–Means method, multilayer perceptron, radial basis function neural network and learning vector quantization neural network. It is shown that the presented procedure can be applied to the automatic adaptation of the smoothing parameter of each of the considered PNN models and that this is an alternative training method. PNN trained by the Q(0)-learning based approach constitutes a classifier which can be treated as one of the top models in data classification problems.     

A Data Mining Approach for Retailing Bank Customer Attrition Analysis

Deregulation within the financial service industries and the widespread acceptance of new technologies is increasing competition in the finance marketplace. Central to the business strategy of every financial service company is the ability to retain existing customers and reach new prospective customers. Data mining is adopted to play an important role in these efforts. In this paper, we present a data mining approach for analyzing retailing bank customer attrition. We discuss the challenging issues such as highly skewed data, time series data unrolling, leaker field detection etc, and the procedure of a data mining project for the attrition analysis for retailing bank customers. We use lift as a proper measure for attrition analysis and compare the lift of data mining models of decision tree, boosted naïve Bayesian network, selective Bayesian network, neural network and the ensemble of classifiers of the above methods. Some interesting findings are reported. Our research work demonstrates the effectiveness and efficiency of data mining in attrition analysis for retailing bank.     

Polynomial neural networks architecture: analysis and design

In this study, we introduce and investigate a class of neural architectures of Polynomial Neural Networks (PNNs), discuss a comprehensive design methodology and carry out a series of numeric experiments. Two kinds of PNN architectures, namely a basic PNN and a modified PNN architecture are discussed. Each of them comes with two types such as the generic and the advanced type. The essence of the design procedure dwells on the Group Method of Data Handling. PNN is a flexible neural architecture whose structure is developed through learning. In particular, the number of layers of the PNN is not fixed in advance but becomes dynamically meaning that the network grows over the training period. In this sense, PNN is a self-organizing network. A comparative analysis shows that the proposed PNN are models with higher accuracy than other fuzzy models.     

基于神经网络集成学习算法的金融时间序列预测

本文在传统神经网络(NN)、循环神经网络(RNN)、长短时记忆网络(LSTM)与门控循环单元(GRU)等神经网络时间预测模型基础上, 进一步构建集成学习(EL)时间序列预测模型, 研究神经网络类模型、集成学习模型和传统时间序列模型在股票指数预测上的表现. 本文以16只A股和国际股票市场指数为样本, 比较模型在不同预测期间和不同国家和地区股票市场上的表现.本文主要结论如下: 第一, 神经网络类时间序列预测模型和神经网络集成学习时间序列预测模型在表现上显著稳健优于传统金融时间序列预测模型, 预测性能提高大约35%; 第二, 神经网络类模型和神经网络集成学习模型在中国和美国股票市场上的表现优于其他发达国家和地区的股票市场.     

基于连续沃尔什变换的过程神经元网络训练

为解决过程神经元网络训练涉及的时域聚合运算问题,提出了过程神经元网络的一种学习算法。对网络的过程式输入函数及时变的权函数,实施连续沃尔什变换,用变换后的数据训练网络。经过这种变换,能使网络输入的时域聚合运算大为简化,有效地避免了复杂的积分过程,使过程神经元网络的训练等同于普通网络的训练。     

Integration of independent component analysis and neural networks for ECG beat classification

In this paper, we propose a scheme to integrate independent component analysis (ICA) and neural networks for electrocardiogram (ECG) beat classification. The ICA is used to decompose ECG signals into weighted sum of basic components that are statistically mutual independent. The projections on these components, together with the RR interval, then constitute a feature vector for the following classifier. Two neural networks, including a probabilistic neural network (PNN) and a back-propagation neural network (BPNN), are employed as classifiers. ECG samples attributing to eight different beat types were sampled from the MIT-BIH arrhythmia database for experiments. The results show high classification accuracy of over 98% with either of the two classifiers. Between them, the PNN shows a slightly better performance than BPNN in terms of accuracy and robustness to the number of ICA-bases. The impressive results prove that the integration of independent component analysis and neural networks, especially PNN, is a promising scheme for the computer-aided diagnosis of heart diseases based on ECG.     

BA-PNN-based methods for power transformer fault diagnosis

This paper presents a machine learning-based approach to power transformer fault diagnosis based on dissolved gas analysis (DGA), a bat algorithm (BA), optimizing the probabilistic neural network (PNN). PNN is a radial basis function feedforward neural network based on Bayesian decision theory, which has a strong fault tolerance and significant advantages in pattern classification. However, one challenge still remains: the performance of PNN is greatly affected by its hidden layer element smooth factor which impacts the classification performance. The proposed approach addresses this challenge by deploying the BA algorithm, a kind of bio-inspired algorithm to optimize PNN. Using the real data collected from a transformer system, we conducted the experiments for validating the performance of the developed method. The experimental results demonstrated that BA is an effective algorithm for optimizing PNN smooth factor and BA-PNN can improve the fault diagnosis performance; in turn, and the machine learning-based model (BA-PNN) can significantly enhance the accuracies of power transformer fault diagnosis.