Neural network ensemble operators for time series forecasting

The combination of forecasts resulting from an ensemble of neural networks has been shown to outperform the use of a single “best” network model. This is supported by an extensive body of literature, which shows that combining generally leads to improvements in forecasting accuracy and robustness, and that using the mean operator often outperforms more complex methods of combining forecasts. This paper proposes a mode ensemble operator based on kernel density estimation, which unlike the mean operator is insensitive to outliers and deviations from normality, and unlike the median operator does not require symmetric distributions. The three operators are compared empirically and the proposed mode ensemble operator is found to produce the most accurate forecasts, followed by the median, while the mean has relatively poor performance. The findings suggest that the mode operator should be considered as an alternative to the mean and median operators in forecasting applications. Experiments indicate that mode ensembles are useful in automating neural network models across a large number of time series, overcoming issues of uncertainty associated with data sampling, the stochasticity of neural network training, and the distribution of the forecasts.     

Improving artificial neural networks’ performance in seasonal time series forecasting

In this study, an artificial neural network (ANN) structure is proposed for seasonal time series forecasting. The proposed structure considers the seasonal period in time series in order to determine the number of input and output neurons. The model was tested for four real-world time series. The results found by the proposed ANN were compared with the results of traditional statistical models and other ANN architectures. This comparison shows that the proposed model comes with lower prediction error than other methods. It is shown that the proposed model is especially convenient when the seasonality in time series is strong; however, if the seasonality is weak, different network structures may be more suitable.     

基于混沌与改进BP神经网络的电价预测方法

结合混沌分析理论和BP神经网络,提出在混沌相空间建立BP神经网络模型.对美国加州边际电价进行预测,并对预测结果进行分析,取得了满意的结果.     

Artificial neural networks for infectious diarrhea prediction using meteorological factors in Shanghai (China)

Infectious diarrhea is an important public health problem around the world. Meteorological factors have been strongly linked to the incidence of infectious diarrhea. Therefore, accurately forecast the number of infectious diarrhea under the effect of meteorological factors is critical to control efforts. In recent decades, development of artificial neural network (ANN) models, as predictors for infectious diseases, have created a great change in infectious disease predictions. In this paper, a three layered feed-forward back-propagation ANN (BPNN) model trained by Levenberg–Marquardt algorithm was developed to predict the weekly number of infectious diarrhea by using meteorological factors as input variable. The meteorological factors were chosen based on the strongly relativity with infectious diarrhea. Also, as a comparison study, the support vector regression (SVR), random forests regression (RFR) and multivariate linear regression (MLR) also were applied as prediction models using the same dataset in addition to BPNN model. The 5-fold cross validation technique was used to avoid the problem of overfitting in models training period. Further, since one of the drawbacks of ANN models is the interpretation of the final model in terms of the relative importance of input variables, a sensitivity analysis is performed to determine the parametric influence on the model outputs. The simulation results obtained from the BPNN confirms the feasibility of this model in terms of applicability and shows better agreement with the actual data, compared to those from the SVR, RFR and MLR models. The BPNN model, described in this paper, is an efficient quantitative tool to evaluate and predict the infectious diarrhea using meteorological factors.     

Hybrid partial least squares and neural network approach for short-term electrical load forecasting

Intelligent systems and methods such as the neural network (NN) are usually used in electric power systems for short-term electrical load forecasting. However, a vast amount of electrical load data is often redundant, and linearly or nonlinearly correlated with each other. Highly correlated input data can result in erroneous prediction results given out by an NN model. Besides this, the determination of the topological structure of an NN model has always been a problem for designers. This paper presents a new artificial intelligence hybrid procedure for next day electric load forecasting based on partial least squares (PLS) and NN. PLS is used for the compression of data input space, and helps to determine the structure of the NN model. The hybrid PLS-NN model can be used to predict hourly electric load on weekdays and weekends. The advantage of this methodology is that the hybrid model can provide faster convergence and more precise prediction results in comparison with abductive networks algorithm. Extensive testing on the electrical load data of the Puget power utility in the USA confirms the validity of the proposed approach.     

Hybrid partial least squares and neural network approach for short-term electrical load forecasting

Intelligent systems and methods such as the neural network （NN） are usually used in electric power systems for short-term electrical load forecasting. However, a vast amount of electrical load data is often redundant, and linearly or nonlinearly correlated with each other. Highly correlated input data can result in erroneous prediction results given out by an NN model. Besides this, the determination of the topological structure of an NN model has always been a problem for designers. This paper presents a new artificial intelligence hybrid procedure for next day electric load forecasting based on partial least squares （PLS） and NN. PLS is used for the compression of data input space, and helps to determine the structure of the NN model. The hybrid PLS-NN model can be used to predict hourly electric load on weekdays and weekends. The advantage of this methodology is that the hybrid model can provide faster convergence and more precise prediction results in comparison with abductive networks algorithm. Extensive testing on the electrical load data of the Puget power utility in the USA confirms the validity of the proposed approach.     

Artificial neural networks: a review of commercial hardware

Artificial neural networks (ANN) became a common solution for a wide variety of problems in many fields, such as control and pattern recognition to name but a few. Many solutions found in these and other ANN fields have reached a hardware implementation phase, either commercial or with prototypes. The most frequent solution for the implementation of ANN consists of training and implementing the ANN within a computer. Nevertheless this solution might be unsuitable because of its cost or its limited speed. The implementation might be too expensive because of the computer and too slow when implemented in software. In both cases dedicated hardware can be an interesting solution.

The necessity of dedicated hardware might not imply building the hardware since in the last two decades several commercial hardware solutions that can be used in the implementation have reached the market.

Unfortunately not every integrated circuit will fit the needs: some will use lower precision, some will implement only certain types of networks, some don’t have training built in and the information is not easy to find.

This article is confined to reporting the commercial chips that have been developed specifically for ANN, leaving out other solutions.

This option has been made because most of the other solutions are based on cards which are built either with these chips, Digital Signal Processors or Reduced Instruction Set Computers.     

Combinatorial time series forecasting based on clustering algorithms and neural networks

Time series analysis utilising more than a single forecasting approach is a procedure originated many years ago as an attempt to improve the performance of the individual model forecasts. In the literature there is a wide range of different approaches but their success depends on the forecasting performance of the individual schemes. A clustering algorithm is often employed to distinguish smaller sets of data that share common properties. The application of clustering algorithms in combinatorial forecasting is discussed with an emphasis placed on the formulation of the problem so that better forecasts are generated. Additionally, the hybrid clustering algorithm that assigns data depending on their distance from the hyper-plane that provides their optimal modelling is applied. The developed cluster-based combinatorial forecasting schemes were examined in a single-step ahead prediction of the pound-dollar daily exchange rate and demonstrated an improvement over conventional linear and neural based combinatorial schemes.     

Review of pulse-coupled neural networks

This paper reviews the research status of pulse-coupled neural networks (PCNN) in the past decade. Considering there are too many publications about the PCNN, we summarize main approaches and point out interesting parts of the PCNN researches rather than contemplate to go into details of particular algorithms or describe results of comparative experiments. First, the current status of the PCNN and some modified models are briefly introduced. Second, we review the PCNN applications in the field of image processing (e.g. image segmentation, image enhancement, image fusion, object and edge detection, pattern recognition, etc.), then applications in other fields also are mentioned. Subsequently, some existing problems are summarized, while we give some suggestions for the solutions to some puzzles. Finally, the trend of the PCNN is pointed out.     

基于神经网络集成的汽车牌照识别

对基于神经网络集成的汽车牌照识别的原理和方法进行了研究,并着重分析了现有技术的积极因素和潜在问题,提出了一种基于神经网络集成进行车牌文字识别的方法.在特征提取时采用了多种特征提取的方法,对提取的每种特征构建一个BP神经网络分别进行训练.最终待识别的字符将被神经网络集成进行识别.实践证明,利用该方法比单个神经网络识别有更高的识别率,具有较高的使用价值.     

An investigation and comparison of artificial neural network and time series models for Chinese food grain price forecasting

This paper compares the predictive performance of ARIMA, artificial neural network and the linear combination models for forecasting wheat price in Chinese market. Empirical results show that the combined model can improve the forecasting performance significantly in contrast with its counterparts in terms of the error evaluation measurements. However, as far as turning points and profit criterions are concerned, the ANN model is best as well as at capturing a significant number of turning points. The results are conflicting when implementing dissimilar forecasting criteria (the quantitative and the turning points measurements) to evaluate the performance of three models. The ANN model is overall the best model, and can be used as an alternative method to model Chinese future food grain price.     

Optimal signal multi-resolution by genetic algorithms to support artificial neural networks for exchange-rate forecasting

Detecting the features of significant patterns from historical data is crucial for good performance in time-series forecasting. Wavelet analysis, which processes information effectively at different scales, can be very useful for feature detection from complex and chaotic time series. In particular, the specific local properties of wavelets can be useful in describing the signals with discontinuous or fractal structure in financial markets. It also allows the removal of noise-dependent high frequencies, while conserving the signal bearing high frequency terms of the signal. However, one of the most critical issues to be solved in the application of the wavelet analysis is to choose the correct wavelet thresholding parameters. If the threshold is small or too large, the wavelet thresholding parameters will tend to overfit or underfit the data. The threshold has so far been selected arbitrarily or by a few statistical criteria.

This study proposes an integrated thresholding design of the optimal or near-optimal wavelet transformation by genetic algorithms (GAs) to represent a significant signal most suitable in artificial neural network models. This approach is applied to Korean won/US dollar exchange-rate forecasting. The experimental results show that this integrated approach using GAs has better performance than the other three wavelet thresholding algorithms (cross-validation, best basis selection and best level tree).     

Classification error of multilayer perceptron neural networks

In subject classification, artificial neural networks (ANNS) are efficient and objective classification methods. Thus, they have been successfully applied to the numerous classification fields. Sometimes, however, classifications do not match the real world, and are subjected to errors. These problems are caused by the nature of ANNS. We discuss these on multilayer perceptron neural networks. By studying of these problems, it helps us to have a better understanding on its classification.     

A high order fuzzy time series forecasting model based on adaptive expectation and artificial neural networks

Many fuzzy time series approaches have been proposed in recent years. These methods include three main phases such as fuzzification, defining fuzzy relationships and, defuzzification. Aladag et al. [2] improved the forecasting accuracy by utilizing feed forward neural networks to determine fuzzy relationships in high order fuzzy time series. Another study for increasing forecasting accuracy was made by Cheng et al. [6]. In their study, they employ adaptive expectation model to adopt forecasts obtained from first order fuzzy time series forecasting model. In this study, we propose a novel high order fuzzy time series method in order to obtain more accurate forecasts. In the proposed method, fuzzy relationships are defined by feed forward neural networks and adaptive expectation model is used for adjusting forecasted values. Unlike the papers of Cheng et al. [6] and Liu et al. [14], forecast adjusting is done by using constraint optimization for weighted parameter. The proposed method is applied to the enrollments of the University of Alabama and the obtained forecasting results compared to those obtained from other approaches are available in the literature. As a result of comparison, it is clearly seen that the proposed method significantly increases the forecasting accuracy.     

基于VLRBP神经网络的汇率预测

为了提高汇率预测的准确性,分别使用VLRBP神经网络模型和GRNN模型及ARIMA模型对欧元汇率时间序列进行建模和预测,通过实证分析发现基于VLRBP的神经网络对于含有大量非线性成分的欧元汇率时间序列的预测比较准确.在分析了最速下降BP学习算法的缺点后,提出利用VLRBP学习算法来解决神经网络振荡和收敛速度过慢的缺陷,并取得较好的效果.同时,为了提高VLRBP网络的泛化性能,提出在训练VLRBP神经网络时应用浴盆曲线方法选取隐层神经元个数和滑动窗口尺寸,试验结果表明该方法适合神经网络模型.     

Improvement of time forecasting models using a novel hybridization of bootstrap and double bootstrap artificial neural networks

Hybrid models such as the Artificial Neural Network-Autoregressive Integrated Moving Average (ANN–ARIMA) model are widely used in forecasting. However, inaccuracies and inefficiency remain in evidence. To yield the ANN–ARIMA with a higher degree of accuracy, efficiency and precision, the bootstrap and the double bootstrap methods are commonly used as alternative methods through the reconstruction of an ANN–ARIMA standard error. Unfortunately, these methods have not been applied in time series-based forecasting models. The aims of this study are twofold. First, is to propose the hybridization of bootstrap model and that of double bootstrap mode called Bootstrap Artificial Neural Network-Autoregressive Integrated Moving Average (B-ANN–ARIMA) and Double Bootstrap Artificial Neural Network-Autoregressive Integrated Moving Average (DB-ANN–ARIMA), respectively. Second, is to investigate the performance of these proposed models by comparing them with ARIMA, ANN and ANN–ARIMA. Our investigation is based on three well-known real datasets, i.e., Wolf’s sunspot data, Canadian lynx data and, Malaysia ringgit/United States dollar exchange rate data. Statistical analysis on SSE, MSE, RMSE, MAE, MAPE and VAF is then conducted to verify that the proposed models are better than previous ARIMA, ANN and ANN–ARIMA models. The empirical results show that, compared with ARIMA, ANNs and ANN–ARIMA models, the proposed models generate smaller values of SSE, MSE, RMSE, MAE, MAPE and VAF for both training and testing datasets. In other words, the proposed models are better than those that we compare with. Their forecasting values are closer to the actual values. Thus, we conclude that the proposed models can be used to generate better forecasting values with higher degree of accuracy, efficiency and, precision in forecasting time series results becomes a priority.     

基于径向基函数(RBF)的安徽省GDP增长模拟与预测

本文运用新型非线性径向基函数RBF神经网络模型，对安徽省国内生产总值(GDP)进行了宏观经济模拟预测分析，结果证明与其它经济计量方法相比较，网络模型新颖，具有较好的预测精度及效果，可广泛应用于各种预测研究，有较高的应用推广价值。     

TACO-miner: An ant colony based algorithm for rule extraction from trained neural networks

Extracting classification rules from data is an important task of data mining and gaining considerable more attention in recent years. In this paper, a new meta-heuristic algorithm which is called as TACO-miner is proposed for rule extraction from artificial neural networks (ANN). The proposed rule extraction algorithm actually works on the trained ANNs in order to discover the hidden knowledge which is available in the form of connection weights within ANN structure. The proposed algorithm is mainly based on a meta-heuristic which is known as touring ant colony optimization (TACO) and consists of two-step hierarchical structure. The proposed algorithm is experimentally evaluated on six binary and n-ary classification benchmark data sets. Results of the comparative study show that TACO-miner is able to discover accurate and concise classification rules.     

An application of pruning in the design of neural networks for real time flood forecasting

We propose the application of pruning in the design of neural networks for hydrological prediction. The basic idea of pruning algorithms, which have not been used in water resources problems yet, is to start from a network which is larger than necessary, and then remove the parameters that are less influential one at a time, designing a much more parameter-parsimonious model. We compare pruned and complete predictors on two quite different Italian catchments. Remarkably, pruned models may provide better generalization than fully connected ones, thus improving the quality of the forecast. Besides the performance issues, pruning is useful to provide evidence of inputs relevance, removing measuring stations identified as redundant (30–40% in our case studies) from the input set. This is a desirable property in the system exercise since data may not be available in extreme situations such as floods; the smaller the set of measuring stations the model depends on, the lower the probability of system downtimes due to missing data. Furthermore, the Authority in charge of the forecast system may decide for real-time operations just to link the gauges of the pruned predictor, thus saving costs considerably, a critical issue in developing countries.

Artificial neural networks for job shop simulation

This paper explores the use of artificial neural networks (ANNs) as a valid alternative to the traditional job-shop simulation approach. Feed forward, multi-layered neural network metamodels were trained through the back-error-propagation (BEP) learning algorithm to provide a versatile job-shop scheduling analysis framework. The constructed neural network architectures were capable of satisfactorily estimating the manufacturing lead times (MLT) for orders simultaneously processed in a four-machine job shop. The MLTs produced by the developed ANN models turned out to be as valid as the data generated from three well-known simulation packages, i.e. Arena, SIMAN, and ProModel. The ANN outputs proved not to be substantially different from the results provided by other valid models such as SIMAN and ProModel when compared against the adopted baseline, Arena. The ANN-based simulations were able to fairly capture the underlying relationship between jobs' machine sequences and their resulting average flowtimes, which proves that ANNs are a viable tool for stochastic simulation metamodeling.