Hybridization of autoregressive integrated moving average (ARIMA) with probabilistic neural networks (PNNs)

Autoregressive integrated moving average (ARIMA) models are one of the most important time series models applied in financial market forecasting over the past three decades. Improving forecasting especially time series forecasting accuracy is an important yet often difficult task facing forecasters. Both theoretical and empirical findings have indicated that integration of different models can be an effective way of improving upon their predictive performance, especially when the models in the ensemble are quite different. In the literature, several hybrid techniques have been proposed by combining different time series models together, in order to yield results that are more accurate. In this paper, a new hybrid model of the autoregressive integrated moving average (ARIMA) and probabilistic neural network (PNN), is proposed in order to yield more accurate results than traditional ARIMA models. In proposed model, the estimated values of the ARIMA model are modified based on the distinguished trend of the ARIMA residuals and optimum step length, which are respectively obtained from a probabilistic neural network and a mathematical programming model. Empirical results with three well-known real data sets indicate that the proposed model can be an effective way in order to construct a more accurate hybrid model than ARIMA model. Therefore, it can be used as an appropriate alternative model for forecasting tasks, especially when higher forecasting accuracy is needed.     

Application of neural networks in forecasting engine systems reliability

This paper presents a comparative study of the predictive performances of neural network time series models for forecasting failures and reliability in engine systems. Traditionally, failure data analysis requires specifications of parametric failure distributions and justifications of certain assumptions, which are at times difficult to validate. On the other hand, the time series modeling technique using neural networks provides a promising alternative. Neural network modeling via feed-forward multilayer perceptron (MLP) suffers from local minima problems and long computation time. The radial basis function (RBF) neural network architecture is found to be a viable alternative due to its shorter training time. Illustrative examples using reliability testing and field data showed that the proposed model results in comparable or better predictive performance than traditional MLP model and the linear benchmark based on Box–Jenkins autoregressive-integrated-moving average (ARIMA) models. The effects of input window size and hidden layer nodes are further investigated. Appropriate design topologies can be determined via sensitivity analysis.     

A new ARIMA-based neuro-fuzzy approach and swarm intelligence for time series forecasting

Time series forecasting is an important and widely interesting topic in the research of system modeling. We propose a new computational intelligence approach to the problem of time series forecasting, using a neuro-fuzzy system (NFS) with auto-regressive integrated moving average (ARIMA) models and a novel hybrid learning method. The proposed intelligent system is denoted as the NFS–ARIMA model, which is used as an adaptive nonlinear predictor to the forecasting problem. For the NFS–ARIMA, the focus is on the design of fuzzy If-Then rules, where ARIMA models are embedded in the consequent parts of If-Then rules. For the hybrid learning method, the well-known particle swarm optimization (PSO) algorithm and the recursive least-squares estimator (RLSE) are combined together in a hybrid way so that they can update the free parameters of NFS–ARIMA efficiently. The PSO is used to update the If-part parameters of the proposed predictor, and the RLSE is used to adapt the Then-part parameters. With the hybrid PSO–RLSE learning method, the NFS–ARIMA predictor may converge in fast learning pace with admirable performance. Three examples are used to test the proposed approach for forecasting ability. The results by the proposed approach are compared to other approaches. The performance comparison shows that the proposed approach performs appreciably better than the compared approaches. Through the experimental results, the proposed approach has shown excellent prediction performance.     

Application of temporal difference learning rules in short‐term traffic flow prediction

Studying dynamic behaviours of a transportation system requires the use of the system mathematical models as well as prediction of traffic flow in the system. Therefore, traffic flow prediction plays an important role in today's intelligent transportation systems. This article introduces a new approach to short‐term daily traffic flow prediction based on artificial neural networks. Among the family of neural networks, multi‐layer perceptron (MLP), radial basis function (RBF) neural network and wavenets have been selected as the three best candidates for performing traffic flow prediction. Moreover, back‐propagation (BP) has been adapted as the most efficient learning scheme in all the cases. It is shown that the coefficients produced by temporal signals improve the performance of the BP learning (BPL) algorithm. Temporal signals provide researchers with a new model of temporal difference BP learning algorithm (TDBPL). The capability and performance of TDBPL algorithm are examined by means of simulation in order to prove that the wavelet theory, with its multi‐resolution ability in comparison to RBF neural networks, is a suitable algorithm in traffic flow forecasting. It is also concluded that despite MLP applications, RBF neural networks do not provide negative forecasts. In addition, the local minimum problems are inevitable in MLP algorithms, while RBF neural networks and wavenet networks do not encounter them.     

HYBRID GREY RELATIONAL ARTIFICIAL NEURAL NETWORK AND AUTO REGRESSIVE INTEGRATED MOVING AVERAGE MODEL FOR FORECASTING TIME-SERIES DATA

The aim of this study is to develop a new hybrid model by combining a linear and nonlinear model for forecasting time-series data. The proposed model (GRANN_ARIMA) integrates nonlinear grey relational artificial neural network (GRANN) and a linear autoregressive integrated moving average (ARIMA) model by combining new features and grey relational analysis to select the appropriate inputs and hybridization succession. To validate the performance of the proposed model, small and large scale data sets are used. The forecasting performance is compared with several models, and these include: individual models (ARIMA, multiple regression, GRANN), several hybrid models (MARMA, MR_ANN, ARIMA_ANN), and an artificial neural network (ANN) trained using a Levenberg Marquardt algorithm. The experiments have shown that the proposed model has outperformed other models with 99.5% forecasting accuracy for small-scale data and 99.84% for large-scale data. The obtained empirical results have proven that the GRANN_ARIMA model can provide a better alternative for time-series forecasting due to its promising performance and capability in handling time-series data for both small- and large-scale data.     

Neural-based electricity load forecasting using hybrid of GA and ACO for feature selection

Due to deregulation of electricity industry, accurate load forecasting and predicting the future electricity demand play an important role in the regional and national power system strategy management. Electricity load forecasting is a challenging task because electric load has complex and nonlinear relationships with several factors. In this paper, two hybrid models are developed for short-term load forecasting (STLF). These models use “ant colony optimization (ACO)” and “combination of genetic algorithm (GA) and ACO (GA-ACO)” for feature selection and multi-layer perceptron (MLP) for hourly load prediction. Weather and climatic conditions, month, season, day of the week, and time of the day are considered as load-influencing factors in this study. Using load time-series of a regional power system, the performance of ACO?+?MLP and GA-ACO?+?MLP hybrid models is compared with principal component analysis (PCA)?+?MLP hybrid model and also with the case of no-feature selection (NFS) when using MLP and radial basis function (RBF) neural models. Experimental results and the performance comparison with similar recent researches in this field show that the proposed GA-ACO?+?MLP hybrid model performs better in load prediction of 24-h ahead in terms of mean absolute percentage error (MAPE).     

Non-parametric regression for space–time forecasting under missing data

As more and more real time spatio-temporal datasets become available at increasing spatial and temporal resolutions, the provision of high quality, predictive information about spatio-temporal processes becomes an increasingly feasible goal. However, many sensor networks that collect spatio-temporal information are prone to failure, resulting in missing data. To complicate matters, the missing data is often not missing at random, and is characterised by long periods where no data is observed. The performance of traditional univariate forecasting methods such as ARIMA models decreases with the length of the missing data period because they do not have access to local temporal information. However, if spatio-temporal autocorrelation is present in a space–time series then spatio-temporal approaches have the potential to offer better forecasts. In this paper, a non-parametric spatio-temporal kernel regression model is developed to forecast the future unit journey time values of road links in central London, UK, under the assumption of sensor malfunction. Only the current traffic patterns of the upstream and downstream neighbouring links are used to inform the forecasts. The model performance is compared with another form of non-parametric regression, K-nearest neighbours, which is also effective in forecasting under missing data. The methods show promising forecasting performance, particularly in periods of high congestion.     

Customer churn prediction by hybrid neural networks

As churn management is a major task for companies to retain valuable customers, the ability to predict customer churn is necessary. In literature, neural networks have shown their applicability to churn prediction. On the other hand, hybrid data mining techniques by combining two or more techniques have been proved to provide better performances than many single techniques over a number of different domain problems. This paper considers two hybrid models by combining two different neural network techniques for churn prediction, which are back-propagation artificial neural networks (ANN) and self-organizing maps (SOM). The hybrid models are ANN combined with ANN (ANN + ANN) and SOM combined with ANN (SOM + ANN). In particular, the first technique of the two hybrid models performs the data reduction task by filtering out unrepresentative training data. Then, the outputs as representative data are used to create the prediction model based on the second technique. To evaluate the performance of these models, three different kinds of testing sets are considered. They are the general testing set and two fuzzy testing sets based on the filtered out data by the first technique of the two hybrid models, i.e. ANN and SOM, respectively. The experimental results show that the two hybrid models outperform the single neural network baseline model in terms of prediction accuracy and Types I and II errors over the three kinds of testing sets. In addition, the ANN + ANN hybrid model significantly performs better than the SOM + ANN hybrid model and the ANN baseline model.     

基于非线性组合模型的交通流预测方法

为开发智能交通系统,提出一种基于RBF和ARIMA网络非线性组合模型的短时交通流预测方法,采用三层结构的RBF网络将2种单一预测方法——RBF和ARIMA网络进行非线性组合,利用实测数据对3类方法进行仿真实验,结果表明,非线性组合模型的预测准确性高于各自单独使用时的准确性,组合模型发挥了2种单一方法各自的优势,是短时交通流预测的有效方法。     

Short-term traffic flow forecasting: parametric and nonparametric approaches via emotional temporal difference learning

Information signal from real case and natural complex dynamical systems such as traffic flow are usually specified by irregular motions. Chaotic nonlinear dynamics approach is now the most powerful tool for scientists to deal with complexities in real cases, and neural networks and neuro-fuzzy models are widely used for their capabilities in nonlinear modeling of chaotic systems more than the traditional methods. As mentioned, the traffic flow conditions caused the forecasting values of traffic flow to lack robustness and accuracy. In this paper, the traffic flow forecasting is analyzed with emotional concepts and multi-agent systems (MASs) points of view as a new method in this field. The findings enabled the researchers to develop a newly object-oriented method of forecasting traffic flow. Its architecture is based on a temporal difference (TD) Q-learning with a neuro-fuzzy structure, which is the nonparametric approach. The performance of TD Q-learning is improved by emotional learning. The proposed method on the present conditions and the action of the system according to the criteria could forecast traffic signals so that the objectives are reached in minimum time. The ability of presented learning algorithm to prospect gains from future actions and obtain rewards from its past experiences allows emotional TD Q-learning algorithm to improve its decisions for the best possible actions. In addition, to study in a more practical situation, the neuro-fuzzy behaviors could be modeled by MAS. The proposed method (intelligent/nonparametric approach) is compared by parametric approach, autoregressive integrated moving average (ARIMA) method, which is implemented by multi-layer perceptron neural networks and called ARIMANN. Here, the ARIMANN is updated by backpropagation and temporal difference backpropagation for the first time. The simulation results revealed that the studied forecaster could discover the optimal forecasting by means of the Q-learning algorithm. Difficult to handle through parametric and classic methods, the real traffic flow signals used for fitting the algorithms is obtained from a two-lane street I-494 in Minnesota City.     

An artificial neural network (p, d, q) model for timeseries forecasting

Artificial neural networks (ANNs) are flexible computing frameworks and universal approximators that can be applied to a wide range of time series forecasting problems with a high degree of accuracy. However, despite all advantages cited for artificial neural networks, their performance for some real time series is not satisfactory. Improving forecasting especially time series forecasting accuracy is an important yet often difficult task facing forecasters. Both theoretical and empirical findings have indicated that integration of different models can be an effective way of improving upon their predictive performance, especially when the models in the ensemble are quite different. In this paper, a novel hybrid model of artificial neural networks is proposed using auto-regressive integrated moving average (ARIMA) models in order to yield a more accurate forecasting model than artificial neural networks. The empirical results with three well-known real data sets indicate that the proposed model can be an effective way to improve forecasting accuracy achieved by artificial neural networks. Therefore, it can be used as an appropriate alternative model for forecasting task, especially when higher forecasting accuracy is needed.     

A hybrid extreme learning machine model with harris hawks optimisation algorithm: an optimised model for product demand forecasting applications

Accurate and real-time product demand forecasting is the need of the hour in the world of supply chain management. Predicting future product demand from historical sales data is a highly non-linear problem, subject to various external and environmental factors. In this work, we propose an optimised forecasting model - an extreme learning machine (ELM) model coupled with the Harris Hawks optimisation (HHO) algorithm to forecast product demand in an e-commerce company. ELM is preferred over traditional neural networks mainly due to its fast computational speed, which allows efficient demand forecasting in real-time. Our ELM-HHO model performed significantly better than ARIMA models that are commonly used in industries to forecast product demand. The performance of the proposed ELM-HHO model was also compared with traditional ELM, ELM auto-tuned using Bayesian Optimisation (ELM-BO), Gated Recurrent Unit (GRU) based recurrent neural network and Long Short Term Memory (LSTM) recurrent neural network models. Different performance metrics, i.e., Root Mean Squared Error (RMSE), Mean Absolute Percentage Error (MAPE) and Mean Percentage Error (MPE) were used for the comparison of the selected models. Horizon forecasting at 3 days and 7 days ahead was also performed using the proposed approach. The results revealed that the proposed approach is superior to traditional product demand forecasting models in terms of prediction accuracy and it can be applied in real-time to predict future product demand based on the previous week’s sales data. In particular, considering RMSE of forecasting, the proposed ELM-HHO model performed 62.73% better than the statistical ARIMA(7,1,0) model, 40.73% better than the neural network based GRU model, 34.05% better than the neural network based LSTM model, 27.16% better than the traditional non-optimised ELM model with 100 hidden nodes and 11.63% better than the ELM-BO model in forecasting product demand for future 3 months. The novelty of the proposed approach lies in the way the fast computational speed of ELMs has been combined with the accuracy gained by tuning hyperparameters using HHO. An increased number of hyperparameters has been optimised in our methodology compared to available models. The majority of approaches to improve the accuracy of ELM so far have only focused on tuning the weights and the biases of the hidden layer. In our hybrid model, we tune the number of hidden nodes, the number of input time lags and even the type of activation function used in the hidden layer in addition to tuning the weights and the biases. This has resulted in a significant increase in accuracy over previous methods. Our work presents an original way of performing product demand forecasting in real-time in industry with highly accurate results which are much better than pre-existing demand forecasting models.

     

A hybrid neural network and ARIMA model for water quality time series prediction

Accurate predictions of time series data have motivated the researchers to develop innovative models for water resources management. Time series data often contain both linear and nonlinear patterns. Therefore, neither ARIMA nor neural networks can be adequate in modeling and predicting time series data. The ARIMA model cannot deal with nonlinear relationships while the neural network model alone is not able to handle both linear and nonlinear patterns equally well. In the present study, a hybrid ARIMA and neural network model is proposed that is capable of exploiting the strengths of traditional time series approaches and artificial neural networks. The proposed approach consists of an ARIMA methodology and feed-forward, backpropagation network structure with an optimized conjugated training algorithm. The hybrid approach for time series prediction is tested using 108-month observations of water quality data, including water temperature, boron and dissolved oxygen, during 1996–2004 at Büyük Menderes river, Turkey. Specifically, the results from the hybrid model provide a robust modeling framework capable of capturing the nonlinear nature of the complex time series and thus producing more accurate predictions. The correlation coefficients between the hybrid model predicted values and observed data for boron, dissolved oxygen and water temperature are 0.902, 0.893, and 0.909, respectively, which are satisfactory in common model applications. Predicted water quality data from the hybrid model are compared with those from the ARIMA methodology and neural network architecture using the accuracy measures. Owing to its ability in recognizing time series patterns and nonlinear characteristics, the hybrid model provides much better accuracy over the ARIMA and neural network models for water quality predictions.     

Forecasting time series using a methodology based on autoregressive integrated moving average and genetic programming

The autoregressive integrated moving average (ARIMA), which is a conventional statistical method, is employed in many fields to construct models for forecasting time series. Although ARIMA can be adopted to obtain a highly accurate linear forecasting model, it cannot accurately forecast nonlinear time series. Artificial neural network (ANN) can be utilized to construct more accurate forecasting model than ARIMA for nonlinear time series, but explaining the meaning of the hidden layers of ANN is difficult and, moreover, it does not yield a mathematical equation. This study proposes a hybrid forecasting model for nonlinear time series by combining ARIMA with genetic programming (GP) to improve upon both the ANN and the ARIMA forecasting models. Finally, some real data sets are adopted to demonstrate the effectiveness of the proposed forecasting model.     

周相似特性下交通流组合预测方法研究

根据交通流量具有周相似的特性,构造了周相似序列。用霍特指数平滑法对周相似序列进行预测,用人工神经网络对残差部分进行预测。将指数平滑法与神经网络法相结合,以便发挥每种方法的优势,获得比单个方法更好的预测结果。实例分析表明,比单独使用ARIMA或单独使用神经网络方法,使用组合方法的预测误差最小,适合于实时的交通流预测。     

Forecasting of short-term traffic-flow based on improved neurofuzzy models via emotional temporal difference learning algorithm

Bounded rationally idea, rather that optimization idea, have result and better performance in decision making theory. Bounded rationality is the idea in decision making, rationality of individuals is limited by the information they have, the cognitive limitations of their minds, and the finite amount of time they have to make decisions. The emotional theory is an important topic presented in this field. The new methods in the direction of purposeful forecasting issues, which are based on cognitive limitations, are presented in this study. The presented algorithms in this study are emphasizes to rectify the learning the peak points, to increase the forecasting accuracy, to decrease the computational time and comply the multi-object forecasting in the algorithms. The structure of the proposed algorithms is based on approximation of its current estimate according to previously learned estimates. The short term traffic flow forecasting is a real benchmark that has been studied in this area. Traffic flow is a good measure of traffic activity. The time-series data used for fitting the proposed models are obtained from a two lane street I-494 in Minnesota City, USA. The research discuss the strong points of new method based on neurofuzzy and limbic system structure such as Locally Linear Neurofuzzy network (LLNF) and Brain Emotional Learning Based Intelligent Controller (BELBIC) models against classical and other intelligent methods such as Radial Basis Function (RBF), Takagi–Sugeno (T–S) neurofuzzy, and Multi-Layer Perceptron (MLP), and the effect of noise on the performance of the models is also considered. Finally, findings confirmed the significance of structural brain modeling beyond the classical artificial neural networks.     

基于FWADE-ELM的短时交通流预测方法

受道路环境和人为因素影响,实际交通系统可视为一个复杂的非线性动力系统,交通流数据具有较强的非线性、时变性和易受随机噪声影响等特征.针对复杂环境下的短时交通流预测问题,提出一种基于烟花差分进化混合算法-极限学习机的短时交通流预测方法.采用奇异谱分析方法滤除原始交通流数据中包含的噪声成分,降噪后的交通流数据用于训练极限学习机(ELM)网络预测模型.进行相空间重构,利用C-C算法确定ELM网络的结构和关键参数.通过融合烟花算法和差分进化算法提出一种烟花差分进化混合算法,可有效提高基本算法的整体优化性能.将改进的混合优化算法用于优化ELM网络的权阈值(结构为9-11-1,维数为110),建立短时交通流预测模型.测试与应用结果表明,所构建的短时交通流预测模型具有较高的预测精度和较强的泛化能力(均方误差为7.75,平均绝对百分比误差为0.086 7),预测值与实际值的拟合程度较好.     

A moving-average filter based hybrid ARIMA–ANN model for forecasting time series data

A suitable combination of linear and nonlinear models provides a more accurate prediction model than an individual linear or nonlinear model for forecasting time series data originating from various applications. The linear autoregressive integrated moving average (ARIMA) and nonlinear artificial neural network (ANN) models are explored in this paper to devise a new hybrid ARIMA–ANN model for the prediction of time series data. Many of the hybrid ARIMA–ANN models which exist in the literature apply an ARIMA model to given time series data, consider the error between the original and the ARIMA-predicted data as a nonlinear component, and model it using an ANN in different ways. Though these models give predictions with higher accuracy than the individual models, there is scope for further improvement in the accuracy if the nature of the given time series is taken into account before applying the models. In the work described in this paper, the nature of volatility was explored using a moving-average filter, and then an ARIMA and an ANN model were suitably applied. Using a simulated data set and experimental data sets such as sunspot data, electricity price data, and stock market data, the proposed hybrid ARIMA–ANN model was applied along with individual ARIMA and ANN models and some existing hybrid ARIMA–ANN models. The results obtained from all of these data sets show that for both one-step-ahead and multistep-ahead forecasts, the proposed hybrid model has higher prediction accuracy.     

基于自组织映射神经网络的局部自回归方法在网络流量预测中的应用

针对网络流量预测,提出一类基于自组织映射（self-organizing map,SOM）神经网络的局部自回归（auto-regressive,AR）方法．根据SOM的联想记忆在时域的推广,在矢量量化临时联想记忆（vector-quantized temporal association memory,VQTAM）建模技术的基础上,给出具有多个局部线性AR模型的AR-SOM方法,基于前K个获胜神经元用权值代替输入向量建立单一时变局部AR模型的K-SOM方法,以及在完成数据向量聚类的同时,更新多个局部AR模型系数的LLM（local linear map）-SOM方法．相对于全局模型,基于SOM神经网络的局部AR方法能够灵活给出有效的监督神经结构,降低了计算复杂度．将本文方法应用于不同的网络流量预测实例中,并与现有方法相比,实验结果表明所提出的方法能有效地改善预测精度,且性能更好．     

自适应差分进化算法优化BP神经网络的时间序列预测

针对BP神经网络学习算法随机初始化连接权值和阈值易使模型陷入局部极小点的缺点,设计了一种自适应差分进化算法优化BP神经网络的混合算法。该混合算法中,差分进化算法采用自适应变异和交叉因子优化BP神经网络的初始权值和阈值,再用预寻优得到的初始权值和阈值训练BP神经网络得到最优的权值和阈值。首先对改进的自适应差分进化算法运用测试函数进行性能测试,然后用一个经典时间序列问题对提出的混合算法进行了检验,并与一般的神经网络、ARIMA预测模型及其它混合预测模型进行了对比,实验结果表明,本文提出的混合算法有效并且明显提高了预测精度。