Entropy Optimization Filtering for Fault Isolation of Nonlinear Non-Gaussian Stochastic Systems

In this paper, the fault isolation (FI) problem is investigated for nonlinear non-Gaussian systems with multiple faults(or abrupt changes of system parameters) in the presence of noises. By constructing a filter to estimate the states, the FI problem can be reduced to an entropy optimization problem subjected to the non-Gaussian estimation error systems. The design objective for the FI purpose is that the entropy of the estimation error is maximized in the presence of diagnosed fault and is minimized in the presence of the nuisance faults or noises. It is shown that the error dynamics is represented by a nonlinear non-Gaussian stochastic system, for which new relationships are applied to formulate the probability density functions (PDFs) of the stochastic error in terms of the PDFs of the noises and the faults. The Renyi's entropy has been used to simplify the computations in the filtering for the recursive design algorithms. It is noted that the output can be supposed to be immeasurable (but with known stochastic distributions), which is different from the existing results where the output is always measurable for feedback. Finally, simulations are given to demonstrate the effectiveness of the proposed data-driven FI filtering algorithms.     

Dynamic Analysis of Stochastic Recurrent Neural Networks

This paper addresses the issue of pth moment exponential stability of stochastic recurrent neural networks (SRNN) with time-varying interconnections and delays. With the help of the Dini derivative of the expectation of V(t, X(t)) “along” the solution X(t) of the model and the technique of Halanay-type inequality, some novel sufficient conditions on pth moment exponential stability of the trivial solution has been established. Conclusions of the development as presented in this paper have gone beyond some published results and are helpful to design stability of networks when stochastic noise is taken into consideration. An example is also given to illustrate the effectiveness of our results.     

随机Hopfiedld神经网络的稳定性分析

通过利用动态大规模互连系统的分解方法,对由Ｉｔｏ随机微分方程描述的随机Ｈｏｐｆｉｌｅｄ神经网络给出了稳定性分析,这样的神经网络被认为是许多神经元的一个互连,在本文中给出了稳定性结论是以单个神经凶和互连结构的性质来表示的。     

Feedback Approximation of the Stochastic Growth Model by Genetic Neural Networks

A direct numerical optimization method is developed to approximate the one-sector stochastic growth model. The feedback investment policy is parameterized as a neural network and trained by a genetic algorithm to maximize the utility functional over the space of time-invariant investment policies. To eliminate the dependence of training on the initial conditions, at any generation, the same stationary investment policy (the same network) is used to repeatedly solve the problem from differing initial conditions. The fitness of a given policy rule is then computed as the sum of payoffs over all initial conditions. The algorithm performs quite well under a wide set of parameters. Given the general purpose nature of the method, the flexibility of neural network parametrization and the global nature of the genetic algorithm search, it can be easily extended to tackle problems with higher dimensional nonlinearities, state spaces and/or discontinuities.     

基于神经网络的非线性自适应控制*

本文对非线性自适应控制的一个新领域－基于神经网络的非线性自适应控制（以下简称ＮＮＢＮＡＣ）的研究进展进行了综述，讨论了这一领域中存在的几个重要问题，然后指出了与这些问题相关的未来的研究方向。     

Prediction of Hydroforming Characteristics using Random Neural Networks

In recent years, hydroforming has become the topic of a lot of active research. Researchers have been looking for better procedures and prediction tools to improve the quality of the product and reduce the prototyping cost. Similar to any other metal forming process, hydroforming leads to non-homogeneous plastic deformations of the workpiece. In this paper, a model is developed to predict the amount of deformation caused by hydroforming using random neural networks (RNNs). RNNs learn the behavior of a system from the provided input/output data in a manner similar to the way the human brain does. This is different from the usual connectionist neural network (NN) models which are based on simple functional analyses. Experimental data is collected and used in training as well as testing the RNNs. The RNN models have feedforward architectures and use a generalized learning algorithm in the training process. Multi-layer RNNs with as few as six neurons were used to capture the nonlinear correlations between the input and output data collected from an experimental setup. The RNN models were able to predict the center deflection, the thickness variation, as well as the deformed shape of circular plate specimens with good accuracy. Received: February 2004 / Accepted: September 2005     

卷积神经网络表征可视化研究综述

近年来, 深度学习在图像分类、目标检测及场景识别等任务上取得了突破性进展, 这些任务多以卷积神经网络为基础搭建识别模型, 训练后的模型拥有优异的自动特征提取和预测性能, 能够为用户提供“输入–输出”形式的端到端解决方案. 然而, 由于分布式的特征编码和越来越复杂的模型结构, 人们始终无法准确理解卷积神经网络模型内部知识表示, 以及促使其做出特定决策的潜在原因. 另一方面, 卷积神经网络模型在一些高风险领域的应用, 也要求对其决策原因进行充分了解, 方能获取用户信任. 因此, 卷积神经网络的可解释性问题逐渐受到关注. 研究人员针对性地提出了一系列用于理解和解释卷积神经网络的方法, 包括事后解释方法和构建自解释的模型等, 这些方法各有侧重和优势, 从多方面对卷积神经网络进行特征分析和决策解释. 表征可视化是其中一种重要的卷积神经网络可解释性方法, 能够对卷积神经网络所学特征及输入–输出之间的相关关系以视觉的方式呈现, 从而快速获取对卷积神经网络内部特征和决策的理解, 具有过程简单和效果直观的特点. 对近年来卷积神经网络表征可视化领域的相关文献进行了综合性回顾, 按照以下几个方面组织内容: 表征可视化研究的提起、相关概念及内容、可视化方法、可视化的效果评估及可视化的应用, 重点关注了表征可视化方法的分类及算法的具体过程. 最后是总结和对该领域仍存在的难点及未来研究趋势进行了展望.     

基于神经网络的异构网络向量化表示方法

当网络中存在不同类型的对象时,对象与对象之间的关系会变得多种多样,网络的结构也会变得更为复杂。针对网络的异构化问题,提出了一种基于神经网络的异构网络向量化表示方法。针对具有图片和文本两种类型对象的异构网络,采用多层次的卷积网络将图片映射到一个潜在的特征空间,采用全连接的神经网络将文本对象也映射到相同的特征空间。在该特征空间内,图片与图片、文本与文本以及图片和文本之间的相似性采用相同的距离计算方法。在实验中,应用提出的方法进行异构网络的多种应用测试,结果表明提出的方法是有效的。     

Nonlinear Filtering of Non-Gaussian Noise

This paper introduces a new nonlinear filter for a discrete time, linear system which is observed in additive non-Gaussian measurement noise. The new filter is recursive, computationally efficient and has significantly improved performance over other linear and nonlinear schemes. The problem of narrowband interference suppression in additive noise is considered as an important example of non-Gaussian noise filtering. It is shown that the new filter outperforms currently used approaches and at the same time offers simplicity in the design.     

Decomposition Analysis of Random Processes in Time Stochastic Petri Networks

A method of analysis of random processes of change of reachable labels in nonexponential time stochastic Petri networks with restricted prehistory is developed on the basis of representation of these processes as a set of parallel independent subprocesses, each of which is a semi-Markov process. The aim of analysis is the computation of the limiting probability distribution of reachable labels. An example is given to illustrate the method.     

Implementation of Spectral Transformations in the Class of Fast Neural Networks

In the paper, the use of neural networks for the implementation of fast algorithms of spectral transformations is discussed. It is shown that the fast algorithms are particular cases of fast neural networks (FNNs). Methods for parametric tuning FNNs to a given system of basis functions are suggested. Neural network implementations of the fast Walsh and wavelet transformations and the fast Fourier, Vilenkin–Christiansen, and Haar transforms are constructed. The discussions are illustrated by examples.     

Nonlinear Black Box Modelling – Fuzzy Networks versus Neural Networks

Fuzzy networks and neural networks offer two different approaches of nonlinear black box modelling. Efficient identification methods have been developed to calculate the parameters for a given structure and have been applied successfully in many examples. But the applications proposed in the literature usually miss the comparison of the alternative method, so that the selection of the more suitable approach for a given task is difficult. This paper aims to ease the decision for one of the two methodologies by considering one well-known high quality approximator of each network type, and presenting a fair comparison. For this purpose, two mathematical and three complex technical examples of nonlinear systems are considered. Generally, fuzzy networks and neural networks face the problem of overtraining causing poor validation/generalisation results. A modification of the established identification methods is proposed as a significant improvement for both approaches.     

随机自动学习机在BP神经网络中的应用

BP算法在许多领域中得到了很好的应用,但它有很多局限性。对复杂的问题,BP需要很长的时间训练网络,而且不一定能得到最佳的网络参数,因此找到合适的网络参数是比较困难的。本文将引入随机自动学习机模型来对BP网络的参数进行调整优化。实验证明所提出的方法不仅能提高网络训练的收敛速度,而且避免了训练陷入局部最小点。     

非线性系统的回归网络辨识

针对未知非线性系统的辨识问题,本文提出了一种新型的回归网络模型,证明了该网络模型在一定条件下能够逼近非线性系统的输入输出关系,提出了训练网络前向连接和反向连接权值的动态反向传播算法,伪真结果验证该方法的有效性。     

Area Efficient Pattern Representation of Binary Neural Networks on RRAM

Resistive random access memory (RRAM) has been demonstrated to implement multiply-and-accumulate(MAC) operations using a highly parallel analog fashion,which dramatically accelerates the convolutional neural networks(CNNs).Since CNNs require considerable converters between analog crossbars and digital peripheral circuits,recent studies map the binary neural networks (BNNs) onto RRAM and binarize the weights to {+1,-1}.However,two mainstream representations for BNN weights introduce patterns of redundant 0s and 1s when dealing with negative weights.In this work,we reduce the area of redundant 0s and 1s by proposing a BNN weight representation framework based on the novel pattern representation and a corresponding architecture.First,we spilt the weight matrix into several small matrices by clustering adjacent columns together.Second,we extract 1s' patterns,i.e.,the submatrices only containing 1s,from the small weight matrix,such that each final output can be represented by the sum of several patterns.Third,we map these patterns onto RRAM crossbars,including pattern computation crossbars (PCCs) and pattern accumulation crossbars(PACs).Finally,we compare the pattern representation with two mainstream representations and adopt the more area efficient one.The evaluation results demonstrate that our framework can save over 20％ of crossbar area effectively,compared with two mainstream representations.     

Stochastic Quantization for Learning Accurate Low-Bit Deep Neural Networks

International Journal of Computer Vision - Low-bit deep neural networks (DNNs) become critical for embedded applications due to their low storage requirement yet high computing efficiency. However,...     

Mean-field Theory and Synchronization in Random Recurrent Neural Networks

In this paper, we first present a new mathematical approach, based on large deviation techniques, for the study of a large random recurrent neural network with discrete time dynamics. In particular, we state a mean field property and a law of large numbers, in the most general case of random models with sparse connections and several populations. Our results are supported by rigorous proofs. Then, we focus our interest on large size dynamics, in the case of a model with excitatory and inhibitory populations. The study of the mean field system and of the divergence of individual trajectories allows to define different dynamical regimes in the macroscopic parameters space, which include chaos and collective synchronization phenomenons. At last, we look at the behavior of a particular finite-size system submitted to gaussian static inputs. The system adapts its dynamics to the input signal, and spontaneously produces dynamical transitions from asynchronous to synchronous behaviors, which correspond to the crossing of a bifurcation line in the macroscopic parameters space.     

Attractor and Stochastic Boundedness for Stochastic Infinite Delay Neural Networks with Markovian Switching

First, we establish the stochastic LaSalle theorem for stochastic infinite delay differential equations with Markovian switching, from which some criterias on attraction are obtained. Then, by employing Lyapunov method and LaSalle-type theorem established above, we obtain some sufficient conditions ensuring the attractor and stochastic boundedness for stochastic infinite delay neural networks with Markovian switching. Finally, an example is also discussed to illustrate the efficiency of the obtained results.