一种基于神经网络集成的决策树构造方法

神经网络集成方法具有比单个神经网络更强的泛化能力，却因为其黑箱性而难以理解；决策树算法因为分类结果显示为树型结构而具有良好的可理解性，泛化能力却比不上神经网络集成。该文将这两种算法相结合，提出一种决策树的构造算法：使用神经网络集成来预处理训练样本，使用C4．5算法处理预处理后的样本并生成决策树。该文在UCI数据上比较了神经网络集成方法、决策树C4．5算法和该文算法，实验表明：该算法具有神经网络集成方法的强泛化能力的优点，其泛化能力明显优于C4．5算法；该算法的最终结果昆示为决策树，显然具有良好的可理解性。     

Degradation process prediction for rotational machinery based on hybrid intelligent model

Bearings are among the most critical and precise components in rotational machinery. The condition and health of bearings play an important role in the functionality and performance of rotational machinery. Since a neural network ensemble approach shows significantly improved generalization performance and outperforms those of a single neural network, one novel selective neural network ensemble model is developed for bearing degradation process prediction. An improved particle swarm optimization with simulated annealing is proposed to select the optimal subset formed by accurate and diverse networks and obtain a better ability to escape from the local optimum. An experimental setup to perform fatigue testing on ball bearings and several simulations are explored in order to validate the developed prediction model. Experimental results show that degradation process prediction based on the explored selective neural network ensemble model provides a means of enhancing the monitoring of ball bearings' condition, and the results of this model are superior in comparison with the results of a single neural network. This selective neural network ensemble model can be used as one excellent predictive maintenance tool in plants.     

基于并行神经网络集成的多步预测方法

神经网络集成通过训练多个神经网络并将其结论进行合成,可以显著地提高学习系统的泛化能力。该文提出了一种基于特征提取的并行神经网络集成多步预测模型ECPNN(ExtractionofCharacteristicsParallelNeuralNetwork)。从单因素时间序列中提取出代表内在机制的特征,采取并行TDNN(Time-delayNeuralNetwork)集成的方式实现时间序列多步预测。实验结果表明了该模型在多步预测方面的可行性和有效性。     

Exploring the behaviour of base classifiers in credit scoring ensembles

Many techniques have been proposed for credit risk assessment, from statistical models to artificial intelligence methods. During the last few years, different approaches to classifier ensembles have successfully been applied to credit scoring problems, demonstrating to be more accurate than single prediction models. However, it is still a question what base classifiers should be employed in each ensemble in order to achieve the highest performance. Accordingly, the present paper evaluates the performance of seven individual prediction techniques when used as members of five different ensemble methods. The ultimate aim of this study is to suggest appropriate classifiers for each ensemble approach in the context of credit scoring. The experimental results and statistical tests show that the C4.5 decision tree constitutes the best solution for most ensemble methods, closely followed by the multilayer perceptron neural network and logistic regression, whereas the nearest neighbour and the naive Bayes classifiers appear to be significantly the worst.     

Combining cluster analysis with classifier ensembles to predict financial distress

The ability to accurately predict business failure is a very important issue in financial decision-making. Incorrect decision-making in financial institutions is very likely to cause financial crises and distress. Bankruptcy prediction and credit scoring are two important problems facing financial decision support. As many related studies develop financial distress models by some machine learning techniques, more advanced machine learning techniques, such as classifier ensembles and hybrid classifiers, have not been fully assessed. The aim of this paper is to develop a novel hybrid financial distress model based on combining the clustering technique and classifier ensembles. In addition, single baseline classifiers, hybrid classifiers, and classifier ensembles are developed for comparisons. In particular, two clustering techniques, Self-Organizing Maps (SOMs) and k-means and three classification techniques, logistic regression, multilayer-perceptron (MLP) neural network, and decision trees, are used to develop these four different types of bankruptcy prediction models. As a result, 21 different models are compared in terms of average prediction accuracy and Type I & II errors. By using five related datasets, combining Self-Organizing Maps (SOMs) with MLP classifier ensembles performs the best, which provides higher predication accuracy and lower Type I & II errors.     

A committee machine with empirical formulas for permeability prediction

This study integrates log-derived empirical formulas and the concept of the committee machine to develop an improved model for predicting permeability. A set of three empirical formulas, such as the Wyllie–Rose, Coates–Dumanoir, and porosity models to correlate reservoir well-logging information with measured core permeability, are used as expert members in a committee machine. A committee machine, a new type of neural network, has a parallel architecture that fuses knowledge by combining the individual outputs of its experts to arrive at an overall output. In this study, an ensemble-based committee machine with empirical formulas (CMEF) is used. This machine combines three individual formulas, each of which performs the same evaluation task. The overall output of each ensemble member is then computed according to the coefficients (weights) of the ensemble averaging method that reflects the contribution of each formula. The optimal combination of weights for prediction is also investigated using a genetic algorithm.We illustrate the method using a case study. Eighty-two data sets composed of well log data and core data were clustered into 41 training sets to construct the model and 41 testing sets to validate the model's predictive ability. A comparison of prediction results from the CMEF model and from three individual empirical formulas showed that the proposed CMEF model for permeability prediction provided the best generalization and performance for validation. This indicated that the CMEF model was more accurate than any one of the individual empirical formulas performing alone.     

Neural network ensemble operators for time series forecasting

The combination of forecasts resulting from an ensemble of neural networks has been shown to outperform the use of a single “best” network model. This is supported by an extensive body of literature, which shows that combining generally leads to improvements in forecasting accuracy and robustness, and that using the mean operator often outperforms more complex methods of combining forecasts. This paper proposes a mode ensemble operator based on kernel density estimation, which unlike the mean operator is insensitive to outliers and deviations from normality, and unlike the median operator does not require symmetric distributions. The three operators are compared empirically and the proposed mode ensemble operator is found to produce the most accurate forecasts, followed by the median, while the mean has relatively poor performance. The findings suggest that the mode operator should be considered as an alternative to the mean and median operators in forecasting applications. Experiments indicate that mode ensembles are useful in automating neural network models across a large number of time series, overcoming issues of uncertainty associated with data sampling, the stochasticity of neural network training, and the distribution of the forecasts.     

基于知识的神经网络在出行方式选择中的应用研究*

针对神经网络和决策树方法在算法上的本质联系和互补优势,将C4.5决策树提取规则的基于知识的神经网络（knowledgebased neural network,KBNN）用于出行方式预测。对居民通勤出行方式选择数据的分析表明,KBNN相比于决策树方法、普通前馈神经网络和多项Logit模型（MNL）有更高的预测精度,方法不仅提高了网络的可解释性,且易于构造、收敛速度更快,实用性较强,为出行方式选择预测提供了新的思路。     

The evaluation of consumer loans using support vector machines

The commencement of the Basel II requirement, popularization of consumer loans and the intense competition in financial market has increased the awareness of the critical delinquency issue for financial institutions in granting loans to potential applicants. In the past few decades, the scheme of artificial neural networks has been successfully applied to the financial field. Recently, the Support Vector Machine (SVM) has emerged as the better neural network in dealing with classification and forecasting problems due to its superior features of generalization performance and global optimum. This study develops a loan evaluation model using SVM to identify potential applicants for consumer loans. In addition to conducting experiments on performance comparison via cross-validation and paired t test, we analyze misclassification errors in terms of Type I and Type II and their effect on selecting network parameters of SVM. The analysis findings facilitate the development of a useful visual decision-support tool. The experimental results using a real-world data set reveal that SVM surpasses traditional neural network models in generalization performance and visualization via the visual tool, which helps decision makers determine appropriate loan evaluation strategies.     

Hybridization of autoregressive integrated moving average (ARIMA) with probabilistic neural networks (PNNs)

Autoregressive integrated moving average (ARIMA) models are one of the most important time series models applied in financial market forecasting over the past three decades. Improving forecasting especially time series forecasting accuracy is an important yet often difficult task facing forecasters. Both theoretical and empirical findings have indicated that integration of different models can be an effective way of improving upon their predictive performance, especially when the models in the ensemble are quite different. In the literature, several hybrid techniques have been proposed by combining different time series models together, in order to yield results that are more accurate. In this paper, a new hybrid model of the autoregressive integrated moving average (ARIMA) and probabilistic neural network (PNN), is proposed in order to yield more accurate results than traditional ARIMA models. In proposed model, the estimated values of the ARIMA model are modified based on the distinguished trend of the ARIMA residuals and optimum step length, which are respectively obtained from a probabilistic neural network and a mathematical programming model. Empirical results with three well-known real data sets indicate that the proposed model can be an effective way in order to construct a more accurate hybrid model than ARIMA model. Therefore, it can be used as an appropriate alternative model for forecasting tasks, especially when higher forecasting accuracy is needed.     

Online tool wear prediction in drilling operations using selective artificial neural network ensemble model

Online tool wear prediction plays a key role in industry automation for higher productivity and product quality. In recent past, several artificial neural network (ANN) models using multiple sensor signals as inputs for prediction as well as classification of tool wear have been proposed. However, a single ANN used in these models is often tries, which could limit their wide applications due to the complicated procedure of constructing a single ANN model. This study proposed a selective ANN ensemble approach DPSOEN, where several selected component ANNs are jointly used to online predict flank wear in drilling operation. DPSOEN provides more simple training and better generalization performance than using single ANN and hence is easier to be used by operators who often are not good at ANN techniques. Two benchmark cases were used to evaluate the performance of DPSOEN in predicting flank wear. It shows improved generalization performance that outperforms those of single ANN and Ensemble ALL approach. The investigation proposed a heuristic approach for applying the DPSOEN-based model as an effective and useful tool to predict tool wear online with potential applications for tool condition monitoring in general. Analysis from this study provides guidelines in developing ANN ensemble-based tool wear prediction systems.     

Ensemble of metamodels with optimized weight factors

Approximate mathematical models (metamodels) are often used as surrogates for more computationally intensive simulations. The common practice is to construct multiple metamodels based on a common training data set, evaluate their accuracy, and then to use only a single model perceived as the best while discarding the rest. This practice has some shortcomings as it does not take full advantage of the resources devoted to constructing different metamodels, and it is based on the assumption that changes in the training data set will not jeopardize the accuracy of the selected model. It is possible to overcome these drawbacks and to improve the prediction accuracy of the surrogate model if the separate stand-alone metamodels are combined to form an ensemble. Motivated by previous research on committee of neural networks and ensemble of surrogate models, a technique for developing a more accurate ensemble of multiple metamodels is presented in this paper. Here, the selection of weight factors in the general weighted-sum formulation of an ensemble is treated as an optimization problem with the desired solution being one that minimizes a selected error metric. The proposed technique is evaluated by considering one industrial and four benchmark problems. The effect of different metrics for estimating the prediction error at either the training data set or a few validation points is also explored. The results show that the optimized ensemble provides more accurate predictions than the stand-alone metamodels and for most problems even surpassing the previously reported ensemble approaches.     

A neural network ensemble method with jittered training data for time series forecasting

Improving forecasting especially time series forecasting accuracy is an important yet often difficult task facing decision makers in many areas. Combining multiple models can be an effective way to improve forecasting performance. Recently, considerable research has been taken in neural network ensembles. Most of the work, however, is devoted to the classification type of problems. As time series problems are often more difficult to model due to issues such as autocorrelation and single realization at any particular time point, more research is needed in this area.In this paper, we propose a jittered ensemble method for time series forecasting and test its effectiveness with both simulated and real time series. The central idea of the jittered ensemble is adding noises to the input data and thus augments the original training data set to form models based on different but related training samples. Our results show that the proposed method is able to consistently outperform the single modeling approach with a variety of time series processes. We also find that relatively small ensemble sizes of 5 and 10 are quite effective in forecasting performance improvement.     

用广义选择性神经网络集成预测MHC-II分子结合肽

MHC II类分子结合肽的预测对于免疫研究和疫苗设计非常重要,然而其结合肽长度的可变性等原因使其预测变得极为困难,提出了一种基于广义选择性神经网络集成的MHC II分子结合肽预测算法,该算法是一种双层集成模型。第一层是用微分进化算法去生成初始神经网络集成池,第二层是从初始神经网络集成池中选择部分组成最终的神经网络集成。实验结果表明广义选择性神经网络集成比传统的选择性神经网络有更好的泛化性能。     

An ensemble of neural networks for weather forecasting

This study presents the applicability of an ensemble of artificial neural networks (ANNs) and learning paradigms for weather forecasting in southern Saskatchewan, Canada. The proposed ensemble method for weather forecasting has advantages over other techniques like linear combination. Generally, the output of an ensemble is a weighted sum, which are weight-fixed, with the weights being determined from the training or validation data. In the proposed approach, weights are determined dynamically from the respective certainties of the network outputs. The more certain a network seems to be of its decision, the higher the weight. The proposed ensemble model performance is contrasted with multi-layered perceptron network (MLPN), Elman recurrent neural network (ERNN), radial basis function network (RBFN), Hopfield model (HFM) predictive models and regression techniques. The data of temperature, wind speed and relative humidity are used to train and test the different models. With each model, 24-h-ahead forecasts are made for the winter, spring, summer and fall seasons. Moreover, the performance and reliability of the seven models are then evaluated by a number of statistical measures. Among the direct approaches employed, empirical results indicate that HFM is relatively less accurate and RBFN is relatively more reliable for the weather forecasting problem. In comparison, the ensemble of neural networks produced the most accurate forecasts.     

A Bayesian regularized artificial neural network for stock market forecasting

In this paper a Bayesian regularized artificial neural network is proposed as a novel method to forecast financial market behavior. Daily market prices and financial technical indicators are utilized as inputs to predict the one day future closing price of individual stocks. The prediction of stock price movement is generally considered to be a challenging and important task for financial time series analysis. The accurate prediction of stock price movements could play an important role in helping investors improve stock returns. The complexity in predicting these trends lies in the inherent noise and volatility in daily stock price movement. The Bayesian regularized network assigns a probabilistic nature to the network weights, allowing the network to automatically and optimally penalize excessively complex models. The proposed technique reduces the potential for overfitting and overtraining, improving the prediction quality and generalization of the network. Experiments were performed with Microsoft Corp. and Goldman Sachs Group Inc. stock to determine the effectiveness of the model. The results indicate that the proposed model performs as well as the more advanced models without the need for preprocessing of data, seasonality testing, or cycle analysis.     

基于神经网络集成的软件故障预测及实验分析

软件系统故障预测是软件测试过程中软件可靠性研究的重点之一。利用软件系统测试过程中前期的故障相关信息进行建模,预测后期的软件故障信息,以便于后期测试和验证资源的合理分配。根据软件测试过程中已知的软件故障时间序列,利用非齐次泊松分布过程、神经网络、神经网络集成等方法对其进行建模。通过对三个实例分别建模,其预测平均相对误差G-O模型依次为3.02%、5.88%和6.58%,而神经网络集成模型为0.19%、1.88%和1.455%,实验结果表明神经网络集成模型具有更精确的预测能力。     

基于小波神经网络的上市公司财务危机预测①

提出了基于小波神经网络的上市公司财务危机预测模型,分析了公司财务指标的选取方法。小波神经网络的训练采用自适应调整学习率及动量系数的方法,以避免陷入局部极小值。与多元统计方法、Logit及Probit模型进行比较,结果表明,该方法预测精度高,第一类错误及第二类错误显著减小。     

A consensual modeling of the expert systems applied to microwave devices

In this work, a consensual approach is developed for modeling RF/microwave devices. In the proposed method, multiple individual models generated by an expert system ensemble are combined by a consensus rule that results in a consistent and improved generalization outputting with the highest possible reliability and accuracy. Here, the expert system ensemble is basically constructed by the competitor and diverse regressors which in our case are back‐propagation artificial neural network (ANN), support vector (SV) regression machine, k‐nearest neighbor and least squares algorithms that perform generalization independently from each other. In the case of excessive data, to reduce the amount of the data, the expert system ensemble of regressors can be shown to be trained by a subset consisting of the SVs. Main feature of the consensual modeling can be put forward as due to diversity in generalization process of each member of the ensemble, the resulted consensus model will effectively identify and encode more aspects of the nonlinear relationship between the independent and the dependent variables than will a single model. Thus, in the consensual modeling, an enhanced single model is built by combining the most successful sides of the competitor and the diverse contributors. Finally, consensual modeling is demonstrated typically for the two devices: the first is a passive device modeling which is synthesis of the conductor‐backed coplanar waveguide with upper shielding and the second is an active device modeling which is the noise modeling of a microwave transistor. © 2010 Wiley Periodicals, Inc. Int J RF and Microwave CAE, 2010.     

基于遗传算法优化小波神经网络的短期天然气负荷预测

天然气负荷预测对于燃气经营企业尤其重要,对保证天然气管网的用气量、优化管网的调度具有重要意义.传统的天然气预测模型预测精度低、模型泛化程度低.为了克服模型缺陷,提出了一种基于遗传算法优化小波神经网络的天然气负荷预测模型.通过遗传算法对小波神经网络的阈值以及网络连接权值等参数进行优化,从而建立预测效果最好的模型,通过企业提供的历史门站数据对预测模型进行验证.仿真结果表明,使用遗传算法优化网络参数的小波神经网络提高了模型的预测精度,具有一定的工程应用价值.