Artificial neural networks for solving ordinary and partialdifferential equations

We present a method to solve initial and boundary value problems using artificial neural networks. A trial solution of the differential equation is written as a sum of two parts. The first part satisfies the initial/boundary conditions and contains no adjustable parameters. The second part is constructed so as not to affect the initial/boundary conditions. This part involves a feedforward neural network containing adjustable parameters (the weights). Hence by construction the initial/boundary conditions are satisfied and the network is trained to satisfy the differential equation. The applicability of this approach ranges from single ordinary differential equations (ODE), to systems of coupled ODE and also to partial differential equations (PDE). In this article, we illustrate the method by solving a variety of model problems and present comparisons with solutions obtained using the Galerkin finite element method for several cases of partial differential equations. With the advent of neuroprocessors and digital signal processors the method becomes particularly interesting due to the expected essential gains in the execution speed.     

Neural network method: delay and system of delay differential equations

In the present article, delay and system of delay differential equations are treated using feed-forward artificial neural networks. We have solved multiple problems using neural network architectures with different depths. The neural networks are trained using the extreme learning machine algorithm for the satisfaction of delay differential equations and associated initial/boundary conditions. Further, numerical rates of convergence of the proposed algorithm are reported based on variation of error in the obtained solution for different number of training points. Emphasis is on analysing whether deeper network architectures trained with extreme learning machine algorithm can perform better than shallow network architectures for approximating the solutions of delay differential equations.

     

Artificial neural network approach for solving fuzzy differential equations

The current research attempts to offer a novel method for solving fuzzy differential equations with initial conditions based on the use of feed-forward neural networks. First, the fuzzy differential equation is replaced by a system of ordinary differential equations. A trial solution of this system is written as a sum of two parts. The first part satisfies the initial condition and contains no adjustable parameters. The second part involves a feed-forward neural network containing adjustable parameters (the weights). Hence by construction, the initial condition is satisfied and the network is trained to satisfy the differential equations. This method, in comparison with existing numerical methods, shows that the use of neural networks provides solutions with good generalization and high accuracy. The proposed method is illustrated by several examples.     

Weather analysis using ensemble of connectionist learning paradigms

This paper presents a comparative analysis of different connectionist and statistical models for forecasting the weather of Vancouver, Canada. For developing the models, one year's data comprising of daily temperature and wind speed were used. A multi-layered perceptron network (MLPN) and an Elman recurrent neural network (ERNN) were trained using the one-step-secant and Levenberg–Marquardt algorithm. Radial basis function network (RBFN) was employed as an alternative to examine its applicability for weather forecasting. To ensure the effectiveness of neurocomputing techniques, the connectionist models were trained and tested using different datasets. Moreover, ensembles of the neural networks were generated by combining the MLPN, ERNN and RBFN using arithmetic mean and weighted average methods. Subsequently, performance of the connectionist models and their ensembles were compared with a well-established statistical technique. Experimental results obtained have shown RBFN produced the most accurate forecast model compared to ERNN and MLPN. Overall, the proposed ensemble approach produced the most accurate forecast, while the statistical model was relatively less accurate for the weather forecasting problem considered.     

Multilayer perceptron neural networks with novel unsupervised training method for numerical solution of the partial differential equations

In this paper by using MultiLayer Perceptron and Radial Basis Function (RBF) neural networks, a novel method for solving both kinds of differential equation, ordinary and partial differential equation, is presented. From the differential equation and its boundary conditions, the energy function of the network is prepared which is used in the unsupervised training method to update the network parameters. This method was implemented to solve the nonlinear Schrodinger equation in hydrogen atom and triangle-shaped quantum well. Comparison of this method results with analytical solution and two well-known numerical methods, Runge–kutta and finite element, shows the efficiency of Neural Networks with high accuracy, fast convergence and low use of memory for solving the differential equations.     

Application of three unsupervised neural network models to singular nonlinear BVP of transformed 2D Bratu equation

In this paper, numerical techniques are developed for solving two-dimensional Bratu equations using different neural network models optimized with the sequential quadratic programming technique. The original two-dimensional problem is transformed into an equivalent singular, nonlinear boundary value problem of ordinary differential equations. Three neural network models are developed for the transformed problem based on unsupervised error using log-sigmoid, radial basis and tan-sigmoid functions. Optimal weights for each model are trained with the help of the sequential quadratic programming algorithm. Three test cases of the equation are solved using the proposed schemes. Statistical analysis based on a large number of independent runs is carried out to validate the models in terms of accuracy, convergence and computational complexity.     

改进粒子群—BP神经网络模型的短期电力负荷预测

为了准确、快速、高效地预测电网短期负荷,提出了改进的粒子群算法(MPSO),并与BP算法相结合,形成改进的粒子群—BP(MPSO-BP)神经网络算法,用此算法训练神经网络,实现了神经网络参数优化,得到了基于MPSO-BP算法的神经网络模型。综合考虑气象、天气、日期类型等影响负荷的因素,进行电网短期负荷预测。算例分析表明,与传统BP神经网络法和PSO-BP神经网络方法相比,该方法改善了BP神经网络的泛化能力,预测精度高,收敛速度快,对电力系统短期负荷具有良好的预测能力。     

Regression-based weight generation algorithm in neural network for solution of initial and boundary value problems

This paper introduces a new algorithm for solving ordinary differential equations (ODEs) with initial or boundary conditions. In our proposed method, the trial solution of differential equation has been used in the regression-based neural network (RBNN) model for single input and single output system. The artificial neural network (ANN) trial solution of ODE is written as sum of two terms, first one satisfies initial/boundary conditions and contains no adjustable parameters. The second part involves a RBNN model containing adjustable parameters. Network has been trained using the initial weights generated by the coefficients of regression fitting. We have used feed-forward neural network and error back propagation algorithm for minimizing error function. Proposed model has been tested for first, second and fourth-order ODEs. We also compare the results of proposed algorithm with the traditional ANN algorithm. The idea may very well be extended to other complicated differential equations.     

分式过程神经元网络在网络流量预测中的应用

为更好解决网络流量预测问题,依据函数逼近论中分式的函数逼近性质和拟合能力要远远大于线性函数的性质,以及过程神经元网络对时变函数的非线性变换能力,提出一种分式过程神经元网络模型及其学习算法。实验结果证明,该网络模型对具有奇异值过程函数的柔韧逼近性质和在奇异值点附近区域反应的灵敏性优于一般过程神经元网络,以网络实测数据对模型进行训练和流量预测,取得了较好的应用效果。     

Optimization free neural network approach for solving ordinary and partial differential equations

Current work introduces a fast converging neural network-based approach for solution of ordinary and partial differential equations. Proposed technique eliminates the need of time-consuming optimization procedure for training of neural network. Rather, it uses the extreme learning machine algorithm for calculating the neural network parameters so as to make it satisfy the differential equation and associated boundary conditions. Various ordinary and partial differential equations are treated using this technique, and accuracy and convergence aspects of the procedure are discussed.

     

基于人工神经网络的经济预测模型

运用不同改进BP算法来建立和训练人工神经网络经济预测模型,并对GDP进行预测,结果表明:模拟值与实际值吻合较好,基于改进BP神经网络模型预测精度高,模型的通用性和实用性强。     

Local linearization filters for non-linear continuous-discrete state space models with multiplicative noise

In this paper, the local linearization method for the approximate computation of the prediction and filtering estimates of continuous-discrete state space models is extended to the general case of non-linear non-autonomous models with multiplicative noise. The approximate prediction and filter estimates are obtained by applying the optimal linear filter to the piecewise linear state space model that emerges from a local linearization of both the non-linear state equation and the non-linear measurement equation. In addition, the solutions of the differential equations that describe the evolution of the first two conditional moments between observations are obtained, and an algorithm for their numerical computation is also given. The performance of the LL filters is illustrated by mean of numerical experiments.     

Creating efficient nonlinear neural network process models that allow model interpretation

The KBANN (knowledge-based artificial neural networks) approach uses neural networks to refine knowledge that can be written in the form of simple propositional rules. This idea is extended by presenting the MANNIDENT (multivariable artificial neural network identification) algorithm by which the mathematical equations of linear process models determine the topology and initial weights of a network, which is further trained using backpropagation. This method is applied to the task of modelling a non-isothermal CSTR in which a first-order exothermic reaction is occurring. This method produces statistically significant gains in accuracy over both a standard neural network approach and a linear model. Furthermore, using the approximate linear model to initialize the weights of the network produces statistically less variation in accuracy. By structuring the neural network according to the approximate linear model, the model can be readily interpreted.     

Mechanisms for activity spread in a neural field model

Field models of continuous neural networks incorporate nonlocal connectivities as well as finite axonal propagation velocities and lead therefore to delayed integral equations. For special choices of the synaptic footprint it is possible to reduce the integral model to a system of partial differential equations. One example is that of the inhomogeneous damped wave equation in one space dimension derived by Jirsa and Haken for exponential synaptic footprint. We show that this equation can be put into the form of a conservation law with nonlinear source, and explore numerically this representation. We find two mechanisms for the spread of the activity from an initially excited region.     

一种用于机场气象预测的模糊神经网络模型

针对民用机场多因素气象预测问题的复杂性,该文构建出一种基于粗糙集的模糊神经网络模型。采用粗糙集理论约简属性,挖掘潜在规则,在此基础上建立模糊神经网络模型,并根据规则的统计性质和离散化结果初始化网络参数,采用BP算法训练网络。实例验证,该模型在收敛速度与预测精度上优于传统的神经网络模型。     

Intelligent computing for numerical treatment of nonlinear prey–predator models

In this study, a new computing paradigm is presented for evaluation of dynamics of nonlinear prey–predator mathematical model by exploiting the strengths of integrated intelligent mechanism through artificial neural networks, genetic algorithms and interior-point algorithm. In the scheme, artificial neural network based differential equation models of the system are constructed and optimization of the networks is performed with effective global search ability of genetic algorithm and its hybridization with interior-point algorithm for rapid local search. The proposed technique is applied to variants of nonlinear prey–predator models by taking different rating factors and comparison with Adams numerical solver certify the correctness for each scenario. The statistical studies have been conducted to authenticate the accuracy and convergence of the design methodology in terms of mean absolute error, root mean squared error and Nash-Sutcliffe efficiency performance indices.     

Bifurcation analysis of a cellular nonlinear network model via neural network approach

In this paper, receptor-based cellular nonlinear network model is studied. By applying neural network method, the ordinary differential equations being equivalent to the partial differential equations of the model are resulted. Also, the bifurcation analysis of the transformed system is presented. To support our theoretical results, some numerical examples are given.     

一种改进的基于深度神经网络的偏微分方程求解方法

偏微分方程求解是计算流体力学等科学与工程领域中数值分析的计算核心。由于物理的多尺度特性和对离散网格质量的敏感性,传统的数值求解方法通常包含复杂的人机交互和昂贵的网格剖分开销,限制了其在许多实时模拟和优化设计问题上的应用效率。提出了一种改进的基于深度神经网络的偏微分方程求解方法TaylorPINN。该方法利用深度神经网络的万能逼近定理和泰勒公式的函数拟合能力,实现了无网格的数值求解过程。在Helmholtz、Klein-Gordon和Navier-Stokes方程上的数值实验结果表明,TaylorPINN能够很好地拟合计算域内时空点坐标与待求函数值之间的映射关系,并提供了准确的数值预测结果。与常用的基于物理信息神经网络方法相比,对于不同的数值问题,TaylorPINN将预测精度提升了3~20倍。     

Solving 5997 differential equations under Microsoft Windows 95 and NT

New simulation software combines Editor Windows for convenient modeling and file manipulation with a Command Window that automatically switches to full-screen color graphics while a simulation runs. Under Windows 95, the new program preempts the CPU, so that Windows does not compromise solution speed during the time-critical simulation runs. Windows multitasking works at all other times. Extra-fast runtime compilation and Pentium-optimized machine code let you quickly try and compare different models programmed in multiple editor windows. You can also program your own special dialog windows for data entry, model changes, and note keeping. Large problems can include up to 6000 first-order differential equations, plus neural networks and fuzzy logic. As an example, we solve the telegrapher's partial differential equation (delay-line equation) by the method of lines. We solve thousands of simultaneous differential equations first in scalar form and then using DESIRE's compact vector notation.     

A novel method of short-term load forecasting based on multiwavelet transform and multiple neural networks

This paper aims to develop a load forecasting method for short-term load forecasting based on multiwavelet transform and multiple neural networks. Firstly, a variable weight combination load forecasting model for power load is proposed and discussed. Secondly, the training data are extracted from power load data through multiwavelet transform. Lastly, the obtained data are trained through a variable weight combination model. BP network, RBF network and wavelet neural network are adopted as the training network, and the trained data from three neural networks are input to a three-layer feedforward neural network for the load forecasting. Simulation results show that accuracy of the combination load forecasting model proposed in the paper is higher than any one single network model and the combination forecast model of three neural networks without preprocessing method of multiwavelet transform.