Dynamic neural network methods applied to on-vehicle idle speedcontrol

The application of neural network techniques to the control of nonlinear dynamical systems has been the subject of substantial interest and research in recent years. In our own work, we have concentrated on extending the dynamic gradient formalism as established by Narendra and Parthasarathy (1990, 1991), and on employing it for applications in the control of nonlinear systems, with specific emphasis on automotive subsystems. The results we have reported to date, however, have been based exclusively upon simulation studies. In this paper, we establish that dynamic gradient training methods can be successfully used for synthesizing neural network controllers directly on instances of real systems. In particular we describe the application of dynamic gradient methods for training a time-lagged recurrent neural network feedback controller for the problem of engine idle speed control on an actual vehicle, discuss hardware and software issues, and provide representative experimental results     

An approach to inverse nonlinear control using neural networks

In this paper, we present a strategy for controlling a class of nonlinear dynamical systems using techniques based on neural networks. The proposed strategy essentially exploits the property of neural networks in being able to approximate arbitrary nonlinear maps when suitable learning strategies are applied. For the closed-loop control, such a network is used in conjunction with a technique of inverse nonlinear control to form what we call an inverse nonlinear controller using neural networks, abbreviated as the INC/NN controller. Properties of the controller are discussed, and it is shown that the proposed INC/NN controller allows the closed-loop error dynamics to be specified directly through a set of controller gains. Extensions of the basic INC/NN controller to incorporate integral control action, to higher order systems, and to a class of nonlinear multi-input multi-output dynamical systems are also indicated. Finally, results of some real-time experiments in applying the INC/NN controller to a position control system which has inherent nonlinearities are presented.     

A recurrent neural-network-based real-time learning controlstrategy applying to nonlinear systems with unknown dynamics

In this paper, the authors present a real-time learning control scheme for unknown nonlinear dynamical systems using recurrent neural networks (RNNs). Two RNNs, based on the same network architecture, are utilized in the learning control system. One is used to approximate the nonlinear system, and the other is used to mimic the desired system response output. The learning rule is achieved by combining the two RNNs to form the neural network control system. A generalized real-time iterative learning algorithm is developed and used to train the RNNs. The algorithm is derived by means of two-dimensional (2-D) system theory that is different from the conventional algorithms that employ the steepest optimization to minimize a cost function. This paper shows that an RNN using the real-time iterative learning algorithm can approximate any trajectory tracking to a very high degree of accuracy. The proposed learning control scheme is applied to numerical problems, and simulation results are included. The results are very promising, and this paper suggests that the 2-D system theory-based RNN learning algorithm provides a new dimension in real-time neural control systems     

Intelligent modeling, observation, and control for nonlinearsystems

We present identification methods for nonlinear mechatronic systems. First, we consider a system consisting of a known linear part and an unknown static nonlinearity. With this approach, using an intelligent observer, it is possible to identify the nonlinear characteristic and to estimate all unmeasurable system states. The identification result of the nonlinearity and the estimated system states are used to improve the controller performance. Secondly, the first approach is extended to systems where both the linear parameters and the nonlinear characteristic are unknown. This is achieved by implementing the intelligent observer as a structured recurrent neural network     

Kalman Filter-Trained Recurrent Neural Equalizers for Time-Varying Channels

Recurrent neural networks have been successfully applied to communications channel equalization because of their modeling capability for nonlinear dynamic systems. Major problems of gradient-descent learning techniques commonly employed to train recurrent neural networks are slow convergence rates and long training sequences required for satisfactory performance. This paper presents decision-feedback equalizers using a recurrent neural network trained with Kalman-filtering algorithms. The main features of the proposed recurrent neural equalizers, using the extended Kalman filter and the unscented Kalman filter, are fast convergence and good performance using relatively short training symbols. Experimental results for various time-varying channels are presented to evaluate the performance of the proposed approaches over a conventional recurrent neural equalizer.     

Approximation of dynamical time-variant systems by continuous-time recurrent neural networks

This paper studies the approximation ability of continuous-time recurrent neural networks to dynamical time-variant systems. It proves that any finite time trajectory of a given dynamical time-variant system can be approximated by the internal state of a continuous-time recurrent neural network. Given several special forms of dynamical time-variant systems or trajectories, this paper shows that they can all be approximately realized by the internal state of a simple recurrent neural network.     

A delay damage model selection algorithm for NARX neural networks

Recurrent neural networks have become popular models for system identification and time series prediction. Nonlinear autoregressive models with exogenous inputs (NARX) neural network models are a popular subclass of recurrent networks and have been used in many applications. Although embedded memory can be found in all recurrent network models, it is particularly prominent in NARX models. We show that using intelligent memory order selection through pruning and good initial heuristics significantly improves the generalization and predictive performance of these nonlinear systems on problems as diverse as grammatical inference and time series prediction     

Algorithmic and architectural optimizations for computationally efficient particle filtering.

In this paper, we analyze the computational challenges in implementing particle filtering, especially to video sequences. Particle filtering is a technique used for filtering nonlinear dynamical systems driven by non-Gaussian noise processes. It has found widespread applications in detection, navigation, and tracking problems. Although, in general, particle filtering methods yield improved results, it is difficult to achieve real time performance. In this paper, we analyze the computational drawbacks of traditional particle filtering algorithms, and present a method for implementing the particle filter using the Independent Metropolis Hastings sampler, that is highly amenable to pipelined implementations and parallelization. We analyze the implementations of the proposed algorithm, and, in particular, concentrate on implementations that have minimum processing times. It is shown that the design parameters for the fastest implementation can be chosen by solving a set of convex programs. The proposed computational methodology was verified using a cluster of PCs for the application of visual tracking. We demonstrate a linear speed-up of the algorithm using the methodology proposed in the paper.     

基于后向回归网络的自适应多步预测

本文针对一类非线性动态系统,提出了一种新的基于后向回归网络的自适应多步预测方法,并对基于神经网络的自适应预测机理进行了分析。预测器由两个同构的后向回归网络来实现,输入及预测长度由随机读写存贮器单元的取值来控制。计算机仿真结果表明,这种自适应预测方法对一类恒定参数的非线性系统是可行的,可有效地处理系统具有的大时延和随机干扰。     

Nonlinear system identification and prediction using orthogonalfunctions

We describe a systematic scheme for the nonlinear adaptive filtering of signals that are generated by nonlinear dynamical systems. The complete filter consists of three sections: a signal-independent standard orthonormal expansion, a scaling derived from an estimate of the vector probability density function (PDF), and an adaptive linear combiner. The orthonormal property of the expansions has two significant implications for adaptive filtering: first, model order reduction is trivial since the contribution of each term to the mean squared error is directly related to the coefficient in the final linear combiner; and second, consistent and rapid convergence of stochastic gradient algorithms is assured. A technique based on the inverse Fourier transform for obtaining a PDF estimate from the characteristic function is also presented. The prediction and identification performance of this nonlinear structure is examined for a number of signals, and it is contrasted with common radial basis function and linear networks     

自动巡航汽车自适应逆控制的建模与仿真

研究神经网络非线性系统的自适应建模和逆建模策略用于非线性的自动巡航系统的控制及可行性。通过对自适应逆控制方法与现行的反馈控制、模糊控制、PID控制进行对比,并在有干扰的情况下系统需要一定的收敛时间,通过运用Matlab软件进行仿真。根据仿真结果分析,当对象输出没有受到干扰时,其在线辨识对象模型和逆模型有十分好的效果;当对象输出存在一些干扰时,由于干扰的存在,需要一段时间来将两个辨识模型收敛。因此,基于动态神经网络的非线性自适应逆控制系统是十分可行的。     

A neural network control system with parallel adaptive enhancementsapplicable to nonlinear servomechanisms

In this paper, we present a technique for using an additional parallel neural network to provide adaptive enhancements to a basic fixed neural network-based nonlinear control system. This proposed parallel adaptive neural network control system is applicable to nonlinear dynamical systems of the type commonly encountered in many practical position control servomechanisms. Properties of the controller are discussed, and it is shown that if Gaussian radial basis function networks are used for the additional parallel neural network, uniformly stable adaptation is assured and the approximation error converges to zero asymptotically. In the paper, the effectiveness of the proposed parallel adaptive neural network control system is demonstrated in real-time implementation experiments for position control in a servomechanism with asymmetrical loading and changes in the load     

Induced l/sub 2/ and generalized H/sub 2/ filtering for systems with repeated scalar nonlinearities

This paper provides complete results on the filtering problem for a class of nonlinear systems described by a discrete-time state equation containing a repeated scalar nonlinearity as in recurrent neural networks. Both induced l/sub 2/ and generalized H/sub 2/ indexes are introduced to evaluate the filtering performance. For a given stable discrete-time systems with repeated scalar nonlinearities, our purpose is to design a stable full-order or reduced-order filter with the same repeated scalar nonlinearities such that the filtering error system is asymptotically stable and has a guaranteed induced l/sub 2/ or generalized H/sub 2/ performance. Sufficient conditions are obtained for the existence of admissible filters. Since these conditions involve matrix equalities, the cone complementarity linearization procedure is employed to cast the nonconvex feasibility problem into a sequential minimization problem subject to linear matrix inequalities, which can be readily solved by using standard numerical software. If these conditions are feasible, a desired filter can be easily constructed. These filtering results are further extended to discrete-time systems with both state delay and repeated scalar nonlinearities. The techniques used in this paper are very different from those used for previous controller synthesis problems, which enable us to circumvent the difficulty of dilating a positive diagonally dominant matrix. A numerical example is provided to show the applicability of the proposed theories.     

多径衰落非线性动力特征初探

本文应用现场实测数据初步探讨了蜂窝移动通信环境中多径衰落的非线生动力特征，首先通过重构状态空间和分析关联维数验证了多径衰落动力机制具有限维自由度，然后设计了前向BP神经网络来拟合其非线性动力系统，并重构了多径信号。分析和仿真结果表明，与传统的随机模型相比，多径衰落的内在物理机制更多地表现出非线性系统特征。     

非凸投影自适应Hammerstein滤波算法

该文研究了Hammerstein系统参数辨识和非线性系统预测问题,提出一种基于非凸投影的自适应滤波算法。论文将问题归结为具有非凸可行域的约束优化问题,并建立了基于交替方向乘子法(ADMM)和递归最小二乘相结合的算法框架。在该算法框架下,非凸约束优化问题的全局最优解可通过岭回归和欧几里得(Euclid)投影循环计算得到。将提出的算法分别应用于Hammerstein系统的参数辨识、非线性未知系统预测以及非线性声学回声消除,并进行仿真实验,结果显示所提算法具有较好的收敛性和稳定性,能够得到较准确的辨识和预测效果。     

Nonlinear scale-space filtering and multiresolution system

We derive and demonstrate a nonlinear scale-space filter and its application in generating a nonlinear multiresolution system. For each datum in a signal, a neighborhood of weighted data is used for clustering. The cluster center becomes the filter output. The filter is governed by a single scale parameter that dictates the spatial extent of nearby data used for clustering. This, together with the local characteristic of the signal, determines the scale parameter in the output space, which dictates the influences of these data on the output. This filter is thus adaptive and data driven. It provides a mechanism for (a) removing impulsive noise, (b) improved smoothing of nonimpulsive noise, and (c) preserving edges. Comparisons with Gaussian scale-space filtering and median filters are made using real images. Using the architecture of the Laplacian pyramid and this nonlinear filter for interpolation, we construct a nonlinear multiresolution system that has two features: (1) edges are well preserved at low resolutions, and (2) difference signals are small and spatially localized. This filter implicitly presents a new mechanism for detecting discontinuities differing from techniques based on local gradients and line processes. This work shows that scale-space filtering, nonlinear filtering, and scale-space clustering are closely related and provides a framework within which further image processing, image coding, and computer vision problems can be investigated.     

An adaptive Gauss-Newton algorithm for training multilayer nonlinear filters that have embedded memory

We describe herein a new means of training dynamic multilayer nonlinear adaptive filters, orneural networks. We restrict our discussion to multilayer dynamic Volterra networks, which are structured so as to restrict their degrees of computational freedom, based on a priori knowledge about the dynamic operation to be emulated. The networks consist of linear dynamic filters together with nonlinear generalized single-layer subnets. We describe how a Newton-like optimization strategy can be applied to these dynamic architectures and detail a newmodified Gauss-Newton optimization technique. The new training algorithm converges faster and to a smaller value of cost than backpropagation-through-time for a wide range of adaptive filtering applications. We apply the algorithm to modeling the inverse of a nonlinear dynamic tracking system. The superior performance of the algorithm over standard techniques is demonstrated.This work was supported by the Stanford Gravity Probe-B project under NASA contract AS 8-36125.     

Real-time parallel adaptive neural network control for nonlinear servomechanisms—an approach using direct adaptive techniques

In this paper, a parallel adaptive neural network control system applicable to nonlinear dynamical systems of the type commonly encountered in many practical position control servomechanisms is developed. The controller is based on the use of direct adaptive techniques and an approach of using an additional parallel neural network to provide adaptive enhancements to a basic fixed neural network-based nonlinear controller. Properties of the proposed new controller are discussed in the paper and it is shown that if Gaussian radial basis function networks are used for the additional parallel neural network, uniformly stable adaptation is assured and asymptotic tracking of the position reference signal is achieved. The effectiveness of the proposed adaptive neural network control system is demonstrated in real-time implementation experiments for position control in a servomechanism with asymmetrical loading and changes in the load.     

神经网络在地-地导弹惯导系统初始对准中的应用

以地-地导弹的惯导系统为研究对象,分析了传统方法在惯导系统初始对准方面的缺陷.针对惯导系统的非线性及实时性等方面的要求,考虑到神经网络所具有的函数逼近性能,扩展Kalman滤波(EKF)所具有的最优估计性能的特点,提出了基于扩展Kalman滤波的神经网络应用技术.应用扩展Kalman滤波对多层神经网络的非线性离散时间系统进行算法训练,在获得的所有观测数据中找到状态(权值)的最小方差估计.在假定的地理坐标系下,对地-地导弹的惯导系统地面自对准的非线性状态方程,应用Matlab对基于EKF的神经网络方法和传统的Kalman滤波方法进行了仿真,对仿真的结果进行了对比分析.     

非线性贝叶斯滤波算法综述

滤波的目的是从序贯量测中在线、实时地估计和预测出动态系统的状态和误差的统计量。从递归贝叶斯估计的框架出发,对非线性滤波算法作了统一描述,并根据对后验概率密度的近似方法的不同,把非线性滤波划归为3类:基于函数近似的滤波方法、基于确定性采样的滤波方法和基于随机采样的滤波方法。对这些非线性滤波的原理、方法及特点做了分析和评述,最后介绍了非线性滤波研究的新动态,并对其发展作了展望。