State observer design for nonlinear systems using neural network

In this paper, an observer design is proposed for nonlinear systems. The Hamilton–Jacobi–Bellman (HJB) equation based formulation has been developed. The HJB equation is formulated using a suitable non-quadratic term in the performance functional to tackle magnitude constraints on the observer gain. Utilizing Lyapunov's direct method, observer is proved to be optimal with respect to meaningful cost. In the present algorithm, neural network (NN) is used to approximate value function to find approximate solution of HJB equation using least squares method. With time-varying HJB solution, we proposed a dynamic optimal observer for the nonlinear system. Proposed algorithm has been applied on nonlinear systems with finite-time-horizon and infinite-time-horizon. Necessary theoretical and simulation results are presented to validate proposed algorithm.     

Fixed-Final-Time-Constrained Optimal Control of Nonlinear Systems Using Neural Network HJB Approach

In this paper, fixed-final time-constrained optimal control laws using neural networks (NNS) to solve Hamilton-Jacobi-Bellman (HJB) equations for general affine in the constrained nonlinear systems are proposed. An NN is used to approximate the time-varying cost function using the method of least squares on a predefined region. The result is an NN nearly -constrained feedback controller that has time-varying coefficients found by a priori offline tuning. Convergence results are shown. The results of this paper are demonstrated in two examples, including a nonholonomic system.     

A model-based deep reinforcement learning method applied to finite-horizon optimal control of nonlinear control-affine system

The Hamilton–Jacobi–Bellman (HJB) equation can be solved to obtain optimal closed-loop control policies for general nonlinear systems. As it is seldom possible to solve the HJB equation exactly for nonlinear systems, either analytically or numerically, methods to build approximate solutions through simulation based learning have been studied in various names like neurodynamic programming (NDP) and approximate dynamic programming (ADP). The aspect of learning connects these methods to reinforcement learning (RL), which also tries to learn optimal decision policies through trial-and-error based learning. This study develops a model-based RL method, which iteratively learns the solution to the HJB and its associated equations. We focus particularly on the control-affine system with a quadratic objective function and the finite horizon optimal control (FHOC) problem with time-varying reference trajectories. The HJB solutions for such systems involve time-varying value, costate, and policy functions subject to boundary conditions. To represent the time-varying HJB solution in high-dimensional state space in a general and efficient way, deep neural networks (DNNs) are employed. It is shown that the use of DNNs, compared to shallow neural networks (SNNs), can significantly improve the performance of a learned policy in the presence of uncertain initial state and state noise. Examples involving a batch chemical reactor and a one-dimensional diffusion-convection-reaction system are used to demonstrate this and other key aspects of the method.     

Near Optimal Output Feedback Control of Nonlinear Discrete-time Systems Based on Reinforcement Neural Network Learning

In this paper, the output feedback based finitehorizon near optimal regulation of nonlinear affine discretetime systems with unknown system dynamics is considered by using neural networks (NNs) to approximate Hamilton-Jacobi-Bellman (HJB) equation solution. First, a NN-based Luenberger observer is proposed to reconstruct both the system states and the control coefficient matrix. Next, reinforcement learning methodology with actor-critic structure is utilized to approximate the time-varying solution, referred to as the value function, of the HJB equation by using a NN. To properly satisfy the terminal constraint, a new error term is defined and incorporated in the NN update law so that the terminal constraint error is also minimized over time. The NN with constant weights and timedependent activation function is employed to approximate the time-varying value function which is subsequently utilized to generate the finite-horizon near optimal control policy due to NN reconstruction errors. The proposed scheme functions in a forward-in-time manner without offline training phase. Lyapunov analysis is used to investigate the stability of the overall closedloop system. Simulation results are given to show the effectiveness and feasibility of the proposed method.      

Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach

The Hamilton-Jacobi-Bellman (HJB) equation corresponding to constrained control is formulated using a suitable nonquadratic functional. It is shown that the constrained optimal control law has the largest region of asymptotic stability (RAS). The value function of this HJB equation is solved for by solving for a sequence of cost functions satisfying a sequence of Lyapunov equations (LE). A neural network is used to approximate the cost function associated with each LE using the method of least-squares on a well-defined region of attraction of an initial stabilizing controller. As the order of the neural network is increased, the least-squares solution of the HJB equation converges uniformly to the exact solution of the inherently nonlinear HJB equation associated with the saturating control inputs. The result is a nearly optimal constrained state feedback controller that has been tuned a priori off-line.     

Bounded robust control of nonlinear systems using neural network�Cbased HJB solution

In this paper, a Hamilton–Jacobi–Bellman (HJB) equation–based optimal control algorithm for robust controller design is proposed for nonlinear systems. The HJB equation is formulated using a suitable nonquadratic term in the performance functional to tackle constraints on the control input. Utilizing the direct method of Lyapunov stability, the controller is shown to be optimal with respect to a cost functional, which includes penalty on the control effort and the maximum bound on system uncertainty. The bounded controller requires the knowledge of the upper bound of system uncertainty. In the proposed algorithm, neural network is used to approximate the solution of HJB equation using least squares method. Proposed algorithm has been applied on the nonlinear system with matched and unmatched type system uncertainties and uncertainties in the input matrix. Necessary theoretical and simulation results are presented to validate proposed algorithm.     

带有时变时滞和同步切换的忆阻神经网络的全局指数稳定性

本文建立并研究了一类具有时变时滞和不同切换机制的忆阻神经网络.利用李雅普诺夫稳定性理论,得到了该神经网络平衡点一致稳定性的充分条件,该充分条件直接有效地反映了时变时滞对稳定性的影响.数值模拟结果验证了理论结果的有效性.     

Asymptotic optimal control of uncertain nonlinear Euler–Lagrange systems

A sufficient condition to solve an optimal control problem is to solve the Hamilton–Jacobi–Bellman (HJB) equation. However, finding a value function that satisfies the HJB equation for a nonlinear system is challenging. For an optimal control problem when a cost function is provided a priori, previous efforts have utilized feedback linearization methods which assume exact model knowledge, or have developed neural network (NN) approximations of the HJB value function. The result in this paper uses the implicit learning capabilities of the RISE control structure to learn the dynamics asymptotically. Specifically, a Lyapunov stability analysis is performed to show that the RISE feedback term asymptotically identifies the unknown dynamics, yielding semi-global asymptotic tracking. In addition, it is shown that the system converges to a state space system that has a quadratic performance index which has been optimized by an additional control element. An extension is included to illustrate how a NN can be combined with the previous results. Experimental results are given to demonstrate the proposed controllers.     

Time-varying two-phase optimization and its application toneural-network learning

In this paper, a time-varying two-phase (TVTP) optimization neural network is proposed based on the two-phase neural network and the time-varying programming neural network. The proposed TVTP algorithm gives exact feasible solutions with a finite penalty parameter when the problem is a constrained time-varying optimization. It can be applied to system identification and control where it has some constraints on weights in the learning of the neural network. To demonstrate its effectiveness and applicability, the proposed algorithm is applied to the learning of a neo-fuzzy neuron model.     

Design and analysis of a general recurrent neural network model for time-varying matrix inversion

Following the idea of using first-order time derivatives, this paper presents a general recurrent neural network (RNN) model for online inversion of time-varying matrices. Different kinds of activation functions are investigated to guarantee the global exponential convergence of the neural model to the exact inverse of a given time-varying matrix. The robustness of the proposed neural model is also studied with respect to different activation functions and various implementation errors. Simulation results, including the application to kinematic control of redundant manipulators, substantiate the theoretical analysis and demonstrate the efficacy of the neural model on time-varying matrix inversion, especially when using a power-sigmoid activation function.     

一种反馈过程神经元网络模型及在动态信号分类中的应用*

针对动态信号模式分类问题,提出了一种反馈过程神经元网络模型和基于该模型的分类方法。这种网络的输入可直接为时变函数,网络的信息传输既有与前馈神经元网络一样的前向流,也有后面各层节点到前层节点的反馈,且可对节点自身反馈输出信息,能直接用于动态信号的模式分类。由于反馈过程神经元网络在对输入样本的学习中增加了神经元输出信息的反馈,可提高网络的学习效率和稳定性。给出了具体学习算法,以时变函数样本集的分类问题为例,实验结果验证了模型和算法的有效性。     

Global robust stability of interval neural networks with multiple time-varying delays

In this paper, the global robust stability is investigated for interval neural networks with multiple time-varying delays. The neural network contains time-invariant uncertain parameters whose values are unknown but bounded in given compact sets. Without assuming both the boundedness on the activation functions and the differentiability on the time-varying delays, a new sufficient condition is presented to ensure the existence, uniqueness, and global robust stability of equilibria for interval neural networks with multiple time-varying delays based on the Lyapunov–Razumikhin technique as well as matrix inequality analysis. Several previous results are improved and generalized, and an example is given to show the effectiveness of the obtained results.     

A recurrent neural network for solving Sylvester equation with time-varying coefficients

Presents a recurrent neural network for solving the Sylvester equation with time-varying coefficient matrices. The recurrent neural network with implicit dynamics is deliberately developed in the way that its trajectory is guaranteed to converge exponentially to the time-varying solution of a given Sylvester equation. Theoretical results of convergence and sensitivity analysis are presented to show the desirable properties of the recurrent neural network. Simulation results of time-varying matrix inversion and online nonlinear output regulation via pole assignment for the ball and beam system and the inverted pendulum on a cart system are also included to demonstrate the effectiveness and performance of the proposed neural network.     

Lyapunov stabilization of the nonlinear control systems via the neural networks

In this paper, we have used a neural network to obtain an approximate solution to the value function of the HJB (Hamilton–Jacobi–Bellman) equation. Then, we have used it to stabilize the affine control nonlinear systems. The requisite control input is generated as the output of a neural network, which is trained off-line. We have designed two various neural networks, in which learning algorithm in the first one it is the steepest descent method, and in the second one is in its unconstrained optimization method. The proposed methods are compared with the traditional methods, and sometimes our methods are proved to be more efficient than the traditional methods. Numerical examples indicate the effectiveness of the proposed neural networks.     

Performance Analysis of Gradient Neural Network Exploited for Online Time-Varying Matrix Inversion

This technical note presents theoretical analysis and simulation results on the performance of a classic gradient neural network (GNN), which was designed originally for constant matrix inversion but is now exploited for time-varying matrix inversion. Compared to the constant matrix-inversion case, the gradient neural network inverting a time-varying matrix could only approximately approach its time-varying theoretical inverse, instead of converging exactly. In other words, the steady-state error between the GNN solution and the theoretical/exact inverse does not vanish to zero. In this technical note, the upper bound of such an error is estimated firstly. The global exponential convergence rate is then analyzed for such a Hopfield-type neural network when approaching the bound error. Computer-simulation results finally substantiate the performance analysis of this gradient neural network exploited to invert online time-varying matrices.     

基于时变RBF网络的非线性时变系统建模

在常规RBF神经网络中采用时变权值,将其应用于非线性时变系统的建模。采用减聚类算法确定网络隐含层神经元数与基函数中心参数,以迭代学习最小二乘算法修正神经网络时变权值,给出时变RBF网络的学习算法。分析表明,迭代学习最小二乘权值修正算法保证了网络时变权值的有界性,迭代误差收敛于零。仿真结果验证了该方法在非线性时变系统建模方面的有效性。     

基于参数实时最优整定的智能PID控制器研究

在参数时变系统中,为了解决PID参数不易实时调整问题,提出了基于PID控制律的智能控制方法;其主要思想是以PID的控制律作为神经网络输入输出模型,以PID的3个参数作为神经网络权值,通过对PID的控制模型进行实时在线训练,获得PID的最佳参数,从而实现对参数时变系统的最优控制;研究结果表明,基于PID控制模型的神经网络优化方法在处理非线性和时变系统时具有很强的鲁棒性,因而是一种有效的智能控制方法。     

Existence and global asymptotic stability of periodic solution for discrete and distributed time-varying delayed neural networks with discontinuous activations

In this paper, we study a general class of neural networks with discrete and distributed time-varying delays, whose neuron activations are discontinuous and may be unbounded or nonmonotonic. By using the Leray-Schauder alternative theorem in multivalued analysis, matrix theory and generalized Lyapunov-like approach, we obtain some sufficient conditions ensuring the existence, uniqueness and global asymptotic stability of the periodic solution. Moreover, when all the variable coefficients and time delays are real constants, we discuss the global convergence in finite time of the neural network dynamical system. Our results extend previous works not only on discrete and distributed time-varying delayed neural networks with continuous or even Lipschitz continuous activations, but also on discrete and distributed time-varying delayed neural networks with discontinuous activations. Two numerical examples are given to illustrate the effectiveness of our main results.     

一种深度小波过程神经网络及在时变信号分类中的应用

针对多通道非线性时变信号分类问题,提出一种基于稀疏自编码器的深度小波过程神经网络(SAE-DWPNN)。通过构建一种多输入/多输出的小波过程神经网络(WPNN),实现对时变信号的多尺度分解和对过程分布特征的初步提取;通过在WPNN隐层之后叠加一个SAE深度网络,对所提取的信号特征进行高层次的综合和表示,并基于softmax分类器实现对时变信号的分类。SAE-DWPNN将现有过程神经网络扩展为深度结构,同时将深度SAE网络在信息处理机制上扩展到时间域,扩展了两类模型的信息处理能力。该网络可提取多通道时序信号的分布特征及其结构特征,并保持样本特征的多样性,提高了对信号时频特性和结构特征的分析能力。文中分析了SAE-DWPNN的性质,给出了综合训练算法。以基于12导联ECG信号的7种心血管疾病分类诊断为例,实验结果验证了模型和算法的有效性。     

动态自适应模块化神经网络结构设计

针对全连接前馈神经网络不能有效应对时变系统的问题, 提出一种动态自适应模块化神经网络结构. 该网络采用减法聚类算法在线辨识工况数据的空间分布, 利用RBF 神经元实现对数据样本空间的划分, 并结合模糊策略将不同子样本空间的数据动态分配给不同的子网络, 最后对各子网络的输出进行集成. 该模块化网络中子网络数量和子网络规模都能根据所学时变任务动态自适应调整. 通过对不同时变系统的预测表明了该网络能够有效跟踪时变系统.