Multifeedback-Layer Neural Network

The architecture and training procedure of a novel recurrent neural network (RNN), referred to as the multifeedback-layer neural network (MFLNN), is described in this paper. The main difference of the proposed network compared to the available RNNs is that the temporal relations are provided by means of neurons arranged in three feedback layers, not by simple feedback elements, in order to enrich the representation capabilities of the recurrent networks. The feedback layers provide local and global recurrences via nonlinear processing elements. In these feedback layers, weighted sums of the delayed outputs of the hidden and of the output layers are passed through certain activation functions and applied to the feedforward neurons via adjustable weights. Both online and offline training procedures based on the backpropagation through time (BPTT) algorithm are developed. The adjoint model of the MFLNN is built to compute the derivatives with respect to the MFLNN weights which are then used in the training procedures. The Levenberg-Marquardt (LM) method with a trust region approach is used to update the MFLNN weights. The performance of the MFLNN is demonstrated by applying to several illustrative temporal problems including chaotic time series prediction and nonlinear dynamic system identification, and it performed better than several networks available in the literature     

基于LSTM的股票价格预测建模与分析

股价波动是一个高度复杂的非线性系统，其股票的调整不是按照均匀的时间过程推进，具有自身的推进过程。结合LSTM（Long Short-Term Memory）递归神经网络的特性和股票市场的特点，对数据进行插值、小波降噪、归一化等预处理操作后，推送到搭建的不同LSTM层数与相同层数下不同隐藏神经元个数的LSTM网络模型中进行训练与测试。对比评价指标与预测效果找到适宜的LSTM层数与隐藏神经元个数，提高了预测准确率约30%。测试结果表明，该模型计算复杂度小，预测准确率有所提高，不仅能在股票投资前对预测股票走势提供有益的参考，还能帮助投资者在对实际股价有了进一步的认知后构建合适的股票投资策略。     

Pso-Bp 耦合算法在矿井瓦斯突出预测中的应用

矿井瓦斯含量的预测模型是一个多变量、非线性的函数关系,预测模型建立的准确与否决定于各个影响因素之间的相互作用、相互耦合的特性。将神经网络与粒子群算法有机地结合起来,以神经网络理论为基础,利用粒子群算法优化隐含层神经元个数和网络中的连接权值,建立瓦斯含量预测模型,解决了Bp神经网络收敛速度慢、易陷入局部优化的缺陷。并在历史数据的基础上,建立遗传神经网络训练和检验样本集,利用MATLAB进行仿真,结果表明粒子群神经网络模型可靠性强,预测精度高。     

Intelligent stock trading system with price trend prediction and reversal recognition using dual-module neural networks

This article presents an intelligent stock trading system that can generate timely stock trading suggestions according to the prediction of short-term trends of price movement using dual-module neural networks(dual net). Retrospective technical indicators extracted from raw price and volume time series data gathered from the market are used as independent variables for neural modeling. Both neural network modules of thedual net learn the correlation between the trends of price movement and the retrospective technical indicators by use of a modified back-propagation learning algorithm. Reinforcing the temporary correlation between the neural weights and the training patterns, dual modules of neural networks are respectively trained on a short-term and a long-term moving-window of training patterns. An adaptive reversal recognition mechanism that can self-tune thresholds for identification of the timing for buying or selling stocks has also been developed in our system. It is shown that the proposeddual net architecture generalizes better than one single-module neural network. According to the features of acceptable rate of returns and consistent quality of trading suggestions shown in the performance evaluation, an intelligent stock trading system with price trend prediction and reversal recognition can be realized using the proposed dual-module neural networks.     

Artificial neural networks with evolutionary instance selection for financial forecasting

In this paper, I propose a genetic algorithm (GA) approach to instance selection in artificial neural networks (ANNs) for financial data mining. ANN has preeminent learning ability, but often exhibit inconsistent and unpredictable performance for noisy data. In addition, it may not be possible to train ANN or the training task cannot be effectively carried out without data reduction when the amount of data is so large. In this paper, the GA optimizes simultaneously the connection weights between layers and a selection task for relevant instances. The globally evolved weights mitigate the well-known limitations of gradient descent algorithm. In addition, genetically selected instances shorten the learning time and enhance prediction performance. This study applies the proposed model to stock market analysis. Experimental results show that the GA approach is a promising method for instance selection in ANN.     

基于DBN深度学习的期货市场价格预测建模与决策

深度学习通过学习深层非线性网络结构即可实现复杂函数的逼近,可以从大量无标注样本集中学习数据集的本质特征。而深度信念网络(DBN)是由多层随机隐变量组成的贝叶斯概率生成模型,可以作为深度神经网络的预训练环节,为该网络提供初始权重。基于该模型的一个高效学习算法不仅解决了模型训练速度慢的问题,还能产生非常好的参数初始值,极大地提升了模型的建模能力。金融市场是一个多变量非线性系统,通过运用DBN模型进行分析预测可以很好地解决其他预测方法初始权重难以确定的问题。文中以原油期货市场价格预测为例,说明了运用DBN模型进行预测和决策的可行性及有效性。     

基于PCA和IFOA-BP神经网络的股价预测模型

针对基于BP神经网络的股票价格预测模型在价格预测时存在较大误差的问题,在BP神经网络方法的基础上引入了主成分分析方法(PCA)和改进的果蝇算法(IFOA),提出一种基于PCA-IFOA-BP神经网络的股票价格预测模型。通过PCA对股票历史数据进行降维,减少冗余信息;采用改进的果蝇算法优化BP神经网络的初始权值和阈值;建立基于PCA和IFOA-BP神经网络的股票价格预测模型。对上证指数股票价格数据进行仿真验证,仿真结果表明:在股票价格预测中,该模型比BP神经网络、PCA-BP和PCA-FOA-BP的预测精度更高,是一种有效可行的预测方法。     

基于蚁群算法优化回声状态网络的研究

针对输出权值采用最小二乘法的回声状态网络(ESN),在随机选取输入权值和隐层神经元阈值时,存在收敛速度慢、预测精度不稳定等问题,提出了基于蚁群算法优化回声状态网络(ACO-ESN)的算法。该算法将优化回声状态网络的初始输入权值、隐层神经元阈值问题转化为蚁群算法中蚂蚁寻找最佳路径的问题,输出权值采用最小二乘法计算,通过蚁群算法的更新、变异、遗传等操作训练回声状态网络,选择出使回声状态网络预测误差最小的输入权值和阈值,从而提高其预测性能。将ACO-ESN与ELM、I-ELM、OS-ELM、B-ELM等神经网络的仿真结果进行对比,结果验证经过蚁群算法优化的回声状态网络加快了其收敛速度,改善了其预测性能,并增强了隐层神经元的敏感度。     

基于极限学习机的股票价格预测

极限学习机（ Extreme Learning Machine , ELM）是一种新型的单馈层神经网络算法,克服了传统的误差反向传播方法需要多次迭代,算法的计算量和搜索空间大的缺点,只需要设置合适的隐含层节点个数,为输入权和隐含层偏差进行随机赋值,一次完成无需迭代。研究表明股票市场是一个非常复杂的非线性系统,需要用到人工智能理论、统计学理论和经济学理论。本文将极限学习机方法引入股票价格预测中,通过对比支持向量机（ Support Vector Machine , SVM）和误差反传神经网络（ Back Propagation Neural Network , BP神经网络）,分析极限学习机在股票价格预测中的可行性和优势。结果表明极限学习机预测精度高,并且在参数选择及训练速度上具有较明显的优势。     

Research on using genetic algorithms to optimize Elman neural networks

There is a function of dynamic mapping when processing non-linear complex data with Elman neural networks. Because Elman neural network inherits the feature of back-propagation neural network to some extent, it has many defects; for example, it is easy to fall into local minimum, the fixed learning rate, the uncertain number of hidden layer neuron and so on. It affects the processing accuracy. So we optimize the weights, thresholds and numbers of hidden layer neurons of Elman networks by genetic algorithm. It improves training speed and generalization ability of Elman neural networks to get the optimal algorithm model. It has been proved by instance analysis that new algorithm was superior to the traditional model in terms of convergence rate, predicted value error, number of trainings conducted successfully, etc. It indicates the effect of the new algorithm and deserves further popularization.     

小波神经网络模型的确定性预测及应用

针对神经网络模型预测结果的随机性,构建了一种紧致性小波神经网络工具箱。该方法将小波函数移植到BP网络隐层,并采用一种随机确定状态命令获得确定的预测结果。与编程实现的小波神经网络和BP网络比较,该方法适合于大批量数据训练,对数据样本的适应能力和鲁棒性强,尤其对高频随机时间序列有更好的适应能力,具有预测结果确定及实用性强等特点,可显著提高模型的训练速度、预测精度和预测效率。基于小波包变换和小波神经网络的瓦斯涌出量预测实验证明了所提方法的有效性。     

Time series prediction using PSO-optimized neural network and hybrid feature selection algorithm for IEEE load data

Artificial neural networks have been widely used in time series prediction. In this paper, multi-layer feedforward neural networks with optimized structures, using particle swarm optimization (PSO) algorithm, are used for hourly load prediction based on load time series of IEEE Reliability Test System. To have a small and appropriate feature subset, a hybrid method is used for feature selection in this paper. This hybrid method uses the combination of genetic algorithm (GA) and ant colony optimization (ACO) algorithm. The season, day of the week, time of the day and history load are considered as load influencing factors in this study based on the mentioned standard load dataset. The optimized number of neurons in the hidden layers of multi-layer perceptron (MLP) is determined using PSO algorithm. Experimental results show that the proposed hybrid model offers superior performance, in terms of mean absolute percentage error (MAPE), in time series prediction as compared to some recent researches in this field.     

基于模拟退火算法的改进极限学习机

传统的极限学习机作为一种有监督的学习模型,任意对隐藏层神经元的输入权值和偏置进行赋值,通过计算隐藏层神经元的输出权值完成学习过程.针对传统的极限学习机在数据分析预测研究中存在预测精度不足的问题,提出一种基于模拟退火算法改进的极限学习机.首先,利用传统的极限学习机对训练集进行学习,得到隐藏层神经元的输出权值,选取预测结果评价标准.然后利用模拟退火算法,将传统的极限学习机隐藏层输入权值和偏置视为初始解,预测结果评价标准视为目标函数,通过模拟退火的降温过程,找到最优解即学习过程中预测误差最小的极限学习机的隐藏层神经元输入权值和偏置,最后通过传统的极限学习机计算得到隐藏层输出权值.实验选取鸢尾花分类数据和波士顿房价预测数据进行分析.实验发现与传统的极限学习机相比,基于模拟退火改进的极限学习机在分类和回归性能上都更优.     

Predicting time series using neural networks with wavelet-based denoising layers

To avoid the need to pre-process noisy data, two special denoising layers based on wavelet multiresolution analysis have been integrated into layered neural networks. A gradient-based learning algorithm has been developed that uses the same cost function to set both the neural network weights and the free parameters of the denoising layers. The denoising layers, when integrated into feedforward and recurrent neural networks, were validated on three time series prediction problems: the logistic map, a rubber hardness time series, and annual average sunspot numbers. Use of the denoising layers improved the prediction accuracy in both cases.     

基于三次样条权函数神经网络的股价预测

随着经济的发展,股票投资已成为很多人的一种投资理财方式,而股票价格的预测也成为投资者关心和研究的焦点。建立一个运算速度和精确度都比较高的股价预测模型,对于金融投资者具有理论指导意义和实际应用价值。文中针对传统BP算法存在的学习速度慢、容易陷入局部极小值、隐层数不易确定等问题,使用三次样条权函数神经网络建立股价预测模型,克服了传统神经网络的缺点。仿真结果表明,该模型具有较高的预测精度,能够对股市进行有效的预测。     

限定记忆的前向神经网络在线学习算法研究

从理论上分析了隐含层激励函数满足Mercer条件的前向神经网络的数学本质,给出了网络学习的指导方向.提出3种网络在线学习算法,它们通过动态调整网络结构和权值来提高网络在线预测性能.算法完全符合统计学习理论提出的结构风险最小化原则,具有较快的学习收敛速度和良好的抗噪声能力.最后通过具体数值实验验证了上述算法的可行性和优越性.     

股票价格预测的建模与仿真研究

研究股票价格准确预测问题,由于股票价格数据具非线性、随机性等变化规律,同时股票市场与国内外经济政治变化有关,传统股票价格预测方法只能对其线性变化规律进行准确预测,无法反映股票价格非线性部分进行有效建模,导致股价预测精度不高。为了提高股票价格预测精度,提出了一种遗传优化BP神经网络的股票价格预测模型。充分利用BP神经网络良好的非线性映射能力,对股票价格变化规律进行建模,并通过遗传算法对BP神经网络模型参数进行优化,从而获最优股票价格最优预测模型。实验结果表明,相对于传统股票价格预测模型,遗传算法优化BP神经网络的股票价格预测模型拟合程度更好,预测精度更高,为股票价格预测提供了依据。     

Extreme learning machine for interval neural networks

Interval data offer a valuable way of representing the available information in complex problems where uncertainty, inaccuracy, or variability must be taken into account. Considered in this paper is the learning of interval neural networks, of which the input and output are vectors with interval components, and the weights are real numbers. The back-propagation (BP) learning algorithm is very slow for interval neural networks, just as for usual real-valued neural networks. Extreme learning machine (ELM) has faster learning speed than the BP algorithm. In this paper, ELM is applied for learning of interval neural networks, resulting in an interval extreme learning machine (IELM). There are two steps in the ELM for usual feedforward neural networks. The first step is to randomly generate the weights connecting the input and the hidden layers, and the second step is to use the Moore–Penrose generalized inversely to determine the weights connecting the hidden and output layers. The first step can be directly applied for interval neural networks. But the second step cannot, due to the involvement of nonlinear constraint conditions for IELM. Instead, we use the same idea as that of the BP algorithm to form a nonlinear optimization problem to determine the weights connecting the hidden and output layers of IELM. Numerical experiments show that IELM is much faster than the usual BP algorithm. And the generalization performance of IELM is much better than that of BP, while the training error of IELM is a little bit worse than that of BP, implying that there might be an over-fitting for BP.     

改进人工蜂群算法优化RBF神经网络的短时交通流预测

为了提高径向基函数RBF神经网络预测模型对短时交通流的预测准确性,提出了一种基于改进人工蜂群算法优化RBF神经网络的短时交通流预测模型。利用改进人工蜂群算法确定RBF网络隐含层的中心值以及隐含层单元数,然后训练改进的人工蜂群算法RBF神经网络预测模型,并将其应用到某城市4天的短时交通流量数据的验证。将实验结果与传统RBF神经网络预测模型、BP神经网络预测模型和小波神经网络预测模型进行了比较。对比结果表明,该方法对短时交通流具有更高的预测准确性。     

前馈神经网络凸优化算法及其仿真应用

该文利用凸函数共轭性质中的Young不等式构造前馈神经网络优化目标函数。这个优化目标函数若固定权值，对隐层输出来说为凸函数；若固定隐层输出，对权值来说为凸函数。因此，此目标函数不存在局部最小。此目标函数的优化速度快，大大提高了前馈神经网络的学习效率。仿真试验表明，与传统算法如误差反向传播算法或BP算法和含势态因子(Momentum factor)的BP算法及现有的分层优化算法相比，新算法能加快收敛速度，并降低学习误差。利用这种快速算法对矿体进行仿真预测，取得了良好效果。