脉频调制神经网络VLSI的设计及应用

本文提出了一种用于故障诊断识别的改进脉冲频率调制(PFM)VLSI神经网络电路,改进了传统的基于软件的机械故障诊断模式,发挥了神经网络超大规模集成电路(VLSI)的优势.利用单层感知器网络、场效应管电路实现了一种新的数字模拟混合突触乘法/加法器电路,而且该神经网络电路的突触权值不需要学习调整,降低了电路的复杂性.以此电路为基础,设计了进行主轴承噪声故障诊断的神经网络故障识别系统.将含有故障信息的原始噪声信号,经过前置信号处理分析、故障特征值提取和神经网络运算,得出VLSI电路输出端电容的电压——代表待识别信号与模板故障信号的“欧氏距离”,进而判断出故障的类别.经过仿真测试,基于硬件的诊断系统的识别性能接近于基于软件的系统.     

A parallel processing VLSI BAM engine

In this paper emerging parallel/distributed architectures are explored for the digital VLSI implementation of adaptive bidirectional associative memory (BAM) neural network. A single instruction stream many data stream (SIMD)-based parallel processing architecture, is developed for the adaptive BAM neural network, taking advantage of the inherent parallelism in BAM. This novel neural processor architecture is named the sliding feeder BAM array processor (SLiFBAM). The SLiFBAM processor can be viewed as a two-stroke neural processing engine, It has four operating modes: learn pattern, evaluate pattern, read weight, and write weight. Design of a SLiFBAM VLSI processor chip is also described. By using 2-mum scalable CMOS technology, a SLiFBAM processor chip with 4+4 neurons and eight modules of 256x5 bit local weight-storage SRAM, was integrated on a 6.9x7.4 mm(2) prototype die. The system architecture is highly flexible and modular, enabling the construction of larger BAM networks of up to 252 neurons using multiple SLiFBAM chips.     

Neurocomputing hardware: present and future

This paper discusses the current state of the art of industrial neurocomputing, and then speculates on its future.Three examples of commercial neuro-silicon are presented: the Adaptive Solutions CNAPS system, the Intel ETANN chip, and the Synaptics OCR chip.We then speculate on where commercial neurocomputing hardware is going. In particular we propose that commercial systems will evolve in the direction of capturing more contextual, knowledge level information. Some results of an industrial handwritten character recognition system created at Apple Computers will be presented which demonstrate the power of adding contextual knowledge to neural network based recognition. Also discussed will be some of the possible directions required for neural network algorithms needed to capture such knowledge and utilize it effectively, as well as results from experiments on capturing contextual knowledge using several different neural network algorithms.Finally, the issues involved in designing VLSI architectures for the efficient emulation of sparsely activated, sparsely connected contextual networks will be discussed. There are fundamental cost/performance limits when emulating such sparse structures in both the digital and analog domain.     

Fast pattern recognition using a neurocomputer

This paper presents a new pattern recognition system based on moment invariants using a neurocomputer. The new pattern recognition system consists of a CCD video camera, an image processing system named FDM, a monitor, two stand lights, an NEC PC-9801 microcomputer and a RICOH RN-2000 neurocomputer; these two different types of computers can be considered to constitute an artificial brain. Experimental studies to recognize five dynamic patterns of Japanese chestnuts were performed. From the studies, a high speed of both learning and recognition has been achieved compared with the former pattern recognition system based on the software of artificial neural networks developed by us. This work was presented, in part, at the International Symposium on Artificial Life and Robotics, Oita, Japan, February 18–20, 1996     

线性神经网络及在系统辨识中的初步应用

该文从基本的智能控制技术——神经网络(NN)技术出发，探讨了神经网络用于系统辨识与建模的基本理论，分析了线性系统神经网络建模的规律，提出了一种利用线性神经网络进行系统辨识的方法。该辨识方法显示出很强的处理问题的能力，无需辨别系统阶次，辨识结构简单，收敛速度快，仿真结果表明这种方法的有效性和可行性。该文共分为四部分，第一部分介绍了神经网络用于系统辨识的特征，第二部分讲述了线性神经网络的工作原理，包括线性神经网络的模型、传递函数、学习规则及训练过程，第三部分讲述了线性神经网络进行系统辨识的仿真实例，第四部分对上述内容作了简要小结。     

A VLSI Image Processing Architecture Dedicated to Real-Time Quality Control Analysis in an Industrial Plant

In this paper, we present a VLSI architecture for real-time image processing in quality control industrial applications: automation of the visual inspection phase of mechanical parts treated by the Fluorescent Magnetic Particle Inspection method for structural-defect detection. The VLSI architecture implements a highly constrained neural network tailored for this specific application: the multi-layer perceptron with strictly local connections. The learning of the weights is performed off line by using the adaptive simulated-annealing algorithm. The neural network has been trained on real plant data: recognition results of the training and classification tasks compare favorably with those obtained by expert human operators.The VLSI architecture receives as input the image (taken on-line on the plant) of a mechanical part and it will find out if at least one structural surface defect is present. The VLSI architecture was optimized, through a set of transformations on the high-level VHDL specifications of the neural network algorithm, to reach real-time operating conditions. Following the proposed approach and the designed architecture, we designed and successfully tested a custom VLSI chip for the real-time implementation of the recognition task.     

确定学习与基于数据的建模及控制

确定学习运用自适应控制和动力学系统的概念与方法, 研究未知动态环境下的知识获取、表达、存储和利用等问题. 针对产生周期或回归轨迹的连续 非线性动态系统, 确定学习可以对其未知系统动态进行局部准确建模, 其基本要 素包括: 1)使用径向基函数(Radial basis function, RBF)神经网络; 2)对于周期(或回归)状态轨迹 满足部分持续激励条件; 3)在周期(或回归)轨迹的邻域内实现对非线性系统动态的局部准确神经网络逼近(局部准确建模); 4)所学的知识以时不变且空间分布的方式表达、以常值神经网络权值的方式存储, 并可在动态环境下用于动态模式的快速识别或者闭环神经网络控制. 本文针对离散动态系统, 扩展了确定学习理论, 提出一个根据时态数据序列对离散动态系统进行建模与控制的框架. 首先, 运用确定学习原理和离散系统的自适应辨识方法, 实现对产生时态数据的离散非线性系统的未知动态进行局部准确的神经网络建模, 并利用此建模结果对时态数据序列进行时不变表达. 其次, 提出时态数据序列的基于动力学的相似性定义, 以及对离散动态系统产生的时态数据序列(亦可称为动态模式)进行快速识别方法. 最后, 针对离散非线性控制系统, 实现了基于时态数据序列对控制系统动态的闭环辨识(局部准确建模). 所学关于闭环动态的知识可用于基于模式的智能控制. 本文表明确定学习可以为时态数据挖掘的研究提供新的途径, 并为基于数据的建模与控制等问题提供新的研究思路.     

An analog VLSI recurrent neural network learning a continuous-timetrajectory

Real-time algorithms for gradient descent supervised learning in recurrent dynamical neural networks fail to support scalable VLSI implementation, due to their complexity which grows sharply with the network dimension. We present an alternative implementation in analog VLSI, which employs a stochastic perturbation algorithm to observe the gradient of the error index directly on the network in random directions of the parameter space, thereby avoiding the tedious task of deriving the gradient from an explicit model of the network dynamics. The network contains six fully recurrent neurons with continuous-time dynamics, providing 42 free parameters which comprise connection strengths and thresholds. The chip implementing the network includes local provisions supporting both the learning and storage of the parameters, integrated in a scalable architecture which can be readily expanded for applications of learning recurrent dynamical networks requiring larger dimensionality. We describe and characterize the functional elements comprising the implemented recurrent network and integrated learning system, and include experimental results obtained from training the network to represent a quadrature-phase oscillator.     

Neurocomputer control in an artificial brain for tracking moving objects

We developed a new control technique for tracking a moving object using a neurocomputer. The control is produced by the RICOH neurocomputer RN-2000, which is able to learn various control laws instantly, in order to track a moving object within a predetermined range of errors. The system for tracking consists of a new information processing system which is a primitive artificial brain (denoted the ABrain). This paper descrbes the new neurocomputer control technique used in the primitive ABrain and presents the results obtained from the tracking experiments. This work was presented, in part, at International Symposium on Artificial Life and Robotics, Oita, Japan, February 18–20, 1996     

Applying Learning by Examples for Digital Design Automation

This paper describes a new learning by example mechanism and its application for digital circuit design automation. This mechanism uses finite state machines to represent the inferred models or designs. The resultant models are easy to be implemented in hardware using current VLSI technologies. Our simulation results show that it is often possible to infer a well-defined deterministic model or design from just one sequence of examples. In addition this mechanism is able to handle sequential task involving long-term dependence. This new learning by example mechanism is used as a design by example system for automatic synthesis of digital circuits. Such systems have not previously been successfully developed mainly because of the lack of mechanism to implement them. From artificial neural network research, it seems possible to apply the knowledge gained from learning by example to form a design by example system. However, one of the problems with neural network approaches is that the resultant models are very difficult to be implemented in hardware using current VLSI technologies. By using the mechanism described in this paper, the resultant models are finite state machines that are well suited for digital designs. Several sequential circuit design examples are simulated and tested. Although our test results show that such a system is feasible for designing simple circuits or small-scale circuit modules, the feasibility of such a system for large-scale circuit design remains to be showed. Both the learning mechanism and the design method show potential and the future research directions are provided.     

Dynamic state and parameter estimation applied to neuromorphic systems

Neuroscientists often propose detailed computational models to probe the properties of the neural systems they study. With the advent of neuromorphic engineering, there is an increasing number of hardware electronic analogs of biological neural systems being proposed as well. However, for both biological and hardware systems, it is often difficult to estimate the parameters of the model so that they are meaningful to the experimental system under study, especially when these models involve a large number of states and parameters that cannot be simultaneously measured. We have developed a procedure to solve this problem in the context of interacting neural populations using a recently developed dynamic state and parameter estimation (DSPE) technique. This technique uses synchronization as a tool for dynamically coupling experimentally measured data to its corresponding model to determine its parameters and internal state variables. Typically experimental data are obtained from the biological neural system and the model is simulated in software; here we show that this technique is also efficient in validating proposed network models for neuromorphic spike-based very large-scale integration (VLSI) chips and that it is able to systematically extract network parameters such as synaptic weights, time constants, and other variables that are not accessible by direct observation. Our results suggest that this method can become a very useful tool for model-based identification and configuration of neuromorphic multichip VLSI systems.     

Architecture and statistical model of a pulse-mode digitalmultilayer neural network

A new architecture and a statistical model for a pulse-mode digital multilayer neural network (DMNN) are presented. Algebraic neural operations are replaced by stochastic processes using pseudo-random pulse sequences. Synaptic weights and neuron states are represented as probabilities and estimated as average rates of pulse occurrences in corresponding pulse sequences. A statistical model of error (or noise) is developed to estimate relative accuracy associated with stochastic computing in terms of mean and variance. The stochastic computing technique is implemented with simple logic gates as basic computing elements leading to a high neuron-density on a chip. Furthermore, the use of simple logic gates for neural operations, the pulse-mode signal representation, and the modular design techniques lead to a massively parallel yet compact and flexible network architecture, well suited for VLSI implementation. Any size of a feedforward network can be configured where processing speed is independent of the network size. Multilayer feedforward networks are modeled and applied to pattern classification problems such as encoding and character recognition.     

A generic systolic array building block for neural networks withon-chip learning

Neural networks require VLSI implementations for on-board systems. Size and real-time considerations show that on-chip learning is necessary for a large range of applications. A flexible digital design is preferred here to more compact analog or optical realizations. As opposed to many current implementations, the two-dimensional systolic array system presented is an attempt to define a novel computer architecture inspired by neurobiology. It is composed of generic building blocks for basic operations rather than predefined neural models. A full custom VLSI design of a first prototype has demonstrated the efficacy of this design. A complete board dedicated to Hopfield's model has been designed using these building blocks. Beyond the very specific application presented, the underlying principles can be used for designing efficient hardware for most neural network models.     

一种多变量模糊神经网络解耦控制器的设计

为提高多变量、非线性和强耦合系统的动态特性和解耦能力,根据解耦原理和神经网络思想,提出一种两级串联结构的自适应模糊神经网络解耦控制器.前级是基于智能权函数规则的自调整模糊控制器,后级是基于动态耦合特性的自适应神经网络解耦控制器.同时从理论上证明了学习算法的收敛性.仿真实例表明,所提出的解耦控制器具有良好的鲁棒性和自适应解耦能力,是解决多变量、非线性和强耦合问题的一种简便有效的控制算法.     

基于神经网络非线性系统辨识和控制的研究

本文提出了由静态的前馈网络和稳定的滤波器构成的非线性系统的辨识模型，在神经网络固有的逼近误差存在的情况下，从理论上讨论了神经网络应用于辨识控制过程中系统的稳定性问题，最后研究了在非线性系统的轨迹跟踪过程中增加滑动控制来偿神经网络的逼近误差，从而提高系统跟踪性能。     

Modeling and sensitivity analysis of neural networks

This paper investigates the use of neural networks for the identification of linear time invariant dynamical systems. Two classes of networks, namely the multilayer feedforward network and the recurrent network with linear neurons, are studied. A notation based on Kronecker product and vector-valued function of matrix is introduced for neural models. It permits to write a feedforward network as a one step ahead predictor used in parameter estimation. A special attention is devoted to system theory interpretation of neural models. Sensitivity analysis can be formulated using derivatives based on the above-mentioned notation.     

Periodic stabilization of look-ahead filters in VLSI implementation

A linear look-ahead filter is a model of digital dynamical systems and infinite impulse response filters for fast pipeline processing in very large scale integration (VLSI) implementation of the digital systems. Two essential problems to be dealt with in the design of look-ahead filters are stability and computational complexity of the filter. In this paper, a new periodic scheme is proposed to stabilize the d-step look-ahead filter and provide minimum amount of computation in digital implementation of the filter.     

Intelligent control of exit temperature in a gas-fuel can-type combustor

The dynamical characteristics of a gas-fuel can-type combustor are highly nonlinear and are too complicated to be modeled precisely. Consequently, it is very difficult to control the exit temperature in a combustor using a conventional feedback controller. This paper investigates the models, describing the dynamics of exit temperature for a gas-fuel can-type combustor, and designs the intelligent controllers, based on the characteristics of the constructed models, to control the exit temperature in the combustor. An identified neural network (INN) was utilized to construct the dynamical models because of its powerful learning and handling ability for nonlinear systems. According to the open-loop responses of the investigated models, two controllers, a self-tuning fuzzy proportional–integral–derivative controller and a neural network controller, were developed for the exit temperature control. Experiments were conducted to evaluate the constructed models and the designed controllers.     

Application of the ANNA neural network chip to high-speed characterrecognition

A neural network with 136000 connections for recognition of handwritten digits has been implemented using a mixed analog/digital neural network chip. The neural network chip is capable of processing 1000 characters/s. The recognition system has essentially the same rate (5%) as a simulation of the network with 32-b floating-point precision.     

High-order neural network structures for identification ofdynamical systems

Several continuous-time and discrete-time recurrent neural network models have been developed and applied to various engineering problems. One of the difficulties encountered in the application of recurrent networks is the derivation of efficient learning algorithms that also guarantee the stability of the overall system. This paper studies the approximation and learning properties of one class of recurrent networks, known as high-order neural networks; and applies these architectures to the identification of dynamical systems. In recurrent high-order neural networks, the dynamic components are distributed throughout the network in the form of dynamic neurons. It is shown that if enough high-order connections are allowed then this network is capable of approximating arbitrary dynamical systems. Identification schemes based on high-order network architectures are designed and analyzed.