A CMOS analog adaptive BAM with on-chip learning and weightrefreshing

The transconductance-mode (T-mode) approach is extended to implement analog continuous-time neural network hardware systems to include on-chip Hebbian learning and on-chip analog weight storage capability. The demonstration vehicle used is a 5+5-neuron bidirectional associative memory (BAM) prototype fabricated in a standard 2-mum double-metal double-polysilicon CMOS process. Mismatches and nonidealities in learning neural hardware are not supposed to be critical if on-chip learning is available, because they will be implicitly compensated. However, mismatches in the learning circuits themselves cannot always be compensated. This mismatch is specially important if the learning circuits use transistors operating in weak inversion. The authors estimate the expected mismatch between learning circuits in the BAM network prototype and evaluate its effect on the learning performance, using theoretical computations and Monte Carlo HSPICE simulations. These theoretical predictions are verified using experimentally measured results on the test vehicle prototype.     

Specification and implementation of a digital Hopfield-typeassociative memory with on-chip training

The definition of the requirements for the design of a neural network associative memory, with on-chip training, in standard digital CMOS technology is addressed. Various learning rules that can be integrated in silicon and the associative memory properties of the resulting networks are investigated. The relationships between the architecture of the circuit and the learning rule are studied in order to minimize the extra circuitry required for the implementation of training. A 64-neuron associative memory with on-chip training has been manufactured, and its future extensions are outlined. Beyond the application to the specific circuit described, the general methodology for determining the accuracy requirements can be applied to other circuits and to other autoassociative memory architectures.     

A VLSI recurrent network of integrate-and-fire neurons connected by plastic synapses with long-term memory.

Electronic neuromorphic devices with on-chip, on-line learning should be able to modify quickly the synaptic couplings to acquire information about new patterns to be stored (synaptic plasticity) and, at the same time, preserve this information on very long time scales (synaptic stability). Here, we illustrate the electronic implementation of a simple solution to this stability-plasticity problem, recently proposed and studied in various contexts. It is based on the observation that reducing the analog depth of the synapses to the extreme (bistable synapses) does not necessarily disrupt the performance of the device as an associative memory, provided that 1) the number of neurons is large enough; 2) the transitions between stable synaptic states are stochastic; and 3) learning is slow. The drastic reduction of the analog depth of the synaptic variable also makes this solution appealing from the point of view of electronic implementation and offers a simple methodological alternative to the technological solution based on floating gates. We describe the full custom analog very large-scale integration (VLSI) realization of a small network of integrate-and-fire neurons connected by bistable deterministic plastic synapses which can implement the idea of stochastic learning. In the absence of stimuli, the memory is preserved indefinitely. During the stimulation the synapse undergoes quick temporary changes through the activities of the pre- and postsynaptic neurons; those changes stochastically result in a long-term modification of the synaptic efficacy. The intentionally disordered pattern of connectivity allows the system to generate a randomness suited to drive the stochastic selection mechanism. We check by a suitable stimulation protocol that the stochastic synaptic plasticity produces the expected pattern of potentiation and depression in the electronic network.     

Modular neuro-chip with on-chip learning and adjustable learning parameters

A modular analog neuro-chip with on-chip learning capability is described. Two popular learning algorithms, error back-propagation and Hebbian learning, are incorporated with adjustable learning parameters. This analog neuro-chip has a fully modular structure for easy multi-chip expansion. The numbers of synapses and neurons can be expanded by simple pin-to-pin connections without additional circuits. For effective learning, the learning rate, sigmoid slope, and ratio of Hebbian learning term to error back-propagation term can be controlled externally by digital signals. The chip is fabricated and successfully trained with gray-scale patterns as well as XOR problem.     

Recurrent correlation associative memories

A model for a class of high-capacity associative memories is presented. Since they are based on two-layer recurrent neural networks and their operations depend on the correlation measure, these associative memories are called recurrent correlation associative memories (RCAMs). The RCAMs are shown to be asymptotically stable in both synchronous and asynchronous (sequential) update modes as long as their weighting functions are continuous and monotone nondecreasing. In particular, a high-capacity RCAM named the exponential correlation associative memory (ECAM) is proposed. The asymptotic storage capacity of the ECAM scales exponentially with the length of memory patterns, and it meets the ultimate upper bound for the capacity of associative memories. The asymptotic storage capacity of the ECAM with limited dynamic range in its exponentiation nodes is found to be proportional to that dynamic range. Design and fabrication of a 3-mm CMOS ECAM chip is reported. The prototype chip can store 32 24-bit memory patterns, and its speed is higher than one associative recall operation every 3 mus. An application of the ECAM chip to vector quantization is also described.     

基于约束区域的连续时间联想记忆神经网络

传统的联想记忆神经网络模型是根据联想记忆点设计权值。文中提出一种根据联想记忆点设计基于约束区域的神经网络模型,它保证了渐近稳定的平衡点集与样要点集相同,不渐近稳定的平衡点恰为实际的拒识状态,并且吸引域分布合理。它具有学习和遗忘能力,还具有记忆容量大和电路可实现优点,是理想的联想记忆器。     

双积分CMOS图像传感器时序控制电路研究

用简单的模拟电路实现双积分技术可以大幅地提高CMOS图像传感器的动态范围.采用列共用采样保持电路的双积分电路和一次采样技术像素相同,仅增加了面积很小的处理电路,在分析该结构工作原理的基础上,提出了高效的时序控制方法.该算法优化了长积分、短积分和信号处理的时序分配,获得了较高的帧频.该算法用可综合的Verilog语言实现,功能仿真和FPGA验证,结果证明了可行性.     

Analog weight adaptation hardware

The demands on offsets in analog weight adaptation circuitry are very high for onchip learning feed-forward neural networks using a back-propagation type of learning rule. Exceeding of the specifications for weight adaptation offsets prevents the weights from converging to their optimum, which leads to a significantly degraded learning behavior. This letter presents a circuit, including a tuning system, that minimizes weight adaptation offsets and that can be used to implement analog on-chip back-propagation learning feed-forward neural networks.     

A GCM neural network using cubic logistic map for information processing

A new chaotic neural network described by a modified globally coupled map (GCM) model with cubic logistic map is proposed, which is called CL-GCM model. Its rich dynamical behaviors over a wide range of parameters and the dynamics mechanism of neurons are demonstrated in detail. Furthermore, the network with delay coupling can be precisely controlled to any specified-periodic orbit by feedback control or modulated parameter control with variable threshold. The results of simulations and experiments suggest that the network is controlled successfully. The controlled CL-GCM model exhibits excellent associative memory performance which appears it can output unique fixed pattern or periodic patterns with specified period which contain the stored pattern closest to the initial pattern.

     

采用计数器存储权值的人工神经网络的实现

文中提出了一种采用计数器存储权值的人工神经网络的实现方案。数字权值采用计数器存储,突触电路和神经元电路用模拟电路来实现。数字权值经脉冲宽度调制电路转换为脉冲信号作为模拟突触电路的输入信号。因而权值可以长期存储,对权值的修改易于实现,突触神经元电路结构简单,融合了人工神经网络模拟实现和数字实现的优点。对于智能计算机的实现具有重要的意义。     

A novel continuous-time neural network for realizing associativememory

A novel neural network is proposed in this paper for realizing associative memory. The main advantage of the neural network is that each prototype pattern is stored if and only if as an asymptotically stable equilibrium point. Furthermore, the basin of attraction of each desired memory pattern is distributed reasonably (in the Hamming distance sense), and an equilibrium point that is not asymptotically stable is really the state that cannot be recognized. The proposed network also has a high storage as well as the capability of learning and forgetting, and all its components can be implemented. The network considered is a very simple linear system with a projection on a closed convex set spanned by the prototype patterns. The advanced performance of the proposed network is demonstrated by means of simulation of a numerical example.     

Computing motion using analog VLSI vision chips: An experimental comparison among different approaches

We have designed, built and tested a number of analog CMOS VLSI circuits for computing 1-D motion from the time-varying intensity values provided by an array of on-chip phototransistors. We present experimental data for two such circuits and discuss their relative performance. One circuit approximates the correlation model while a second chip uses resistive grids to compute zero-crossings to be tracked over time by a separate digital processor. Both circuits integrate image acquisition with image processing functions and compute velocity in real time. For comparison, we also describe the performance of a simple motion algorithm using off-the-shelf digital components. We conclude that analog circuits implementing various correlation-like motion algorithms are more robust than our previous analog circuits implementing gradient-like motion algorithms.     

A real-time experiment using a 50-neuron CMOS analog silicon chipwith on-chip digital learning

The authors present initial results of a pattern/character recognition and association experiment using a newly fabricated 50-neuron CMOS analog silicon chip with digital on-chip learning. Attention is given to the circuit architecture, the VLSI chips, and the interface circuitry.     

Analog implementation of a Kohonen map with on-chip learning.

Kohonen maps are self-organizing neural networks that classify and quantify n-dimensional data into a one- or two-dimensional array of neurons. Most applications of Kohonen maps use simulations on conventional computers, eventually coupled to hardware accelerators or dedicated neural computers. The small number of different operations involved in the combined learning and classification process, however, makes the Kohonen model particularly suited to a dedicated VLSI implementation, taking full advantage of the parallelism and speed that can be obtained on the chip. A fully analog implementation of a one-dimensional Kohonen map, with on-chip learning and refreshment of on-chip analog synaptic weights, is proposed. The small number of transistors in each cell allows a high degree of parallelism in the operations, which greatly improves the computation speed compared to other implementations. The storage of analog synaptic weights, based on the principle of current copiers, is emphasized. It is shown that this technique can be used successfully for the realization of VLSI Kohonen maps.     

Application of Feed-forward Artificial Neural Networks to the Identification of Defective Analog Integrated Circuits

This paper presents a new approach for detecting defects in analog integrated circuits using a feed-forward neural network trained by the resilient error back-propagation method. A feed-forward neural network has been used for detecting faults in a simple analog CMOS circuit by representing the differences observed in power supply current of fault-free and faulty circuits. The identification of defects was performed in time and frequency domains, followed by a comparison of results achieved in both domains. We show that resilient back-propagation neural networks can be a very efficient and versatile approach for identifying defective analog circuits. Moreover, this approach is not limited to the supply current analysis, because it also offers monitoring of other circuit parameters. The type of defects detected by the resilient backpropagation neural networks, as well as other possible applications of this approach, are discussed.     

A CMOS feedforward neural-network chip with on-chip parallel learning for oscillation cancellation

The paper presents a mixed signal CMOS feedforward neural-network chip with on-chip error-reduction hardware for real-time adaptation. The chip has compact on-chip weighs capable of high-speed parallel learning; the implemented learning algorithm is a genetic random search algorithm: the random weight change (RWC) algorithm. The algorithm does not require a known desired neural network output for error calculation and is suitable for direct feedback control. With hardware experiments, we demonstrate that the RWC chip, as a direct feedback controller, successfully suppresses unstable oscillations modeling combustion engine instability in real time.     

Current-mode subthreshold MOS circuits for analog VLSI neuralsystems

An overview of the current-mode approach for designing analog VLSI neural systems in subthreshold CMOS technology is presented. Emphasis is given to design techniques at the device level using the current-controlled current conveyor and the translinear principle. Circuits for associative memory and silicon retina systems are used as examples. The design methodology and how it relates to actual biological microcircuits are discussed.     

The modulation of learning state in a biological associative memory: an in vitro,in vivo,and in computo study of object recognition in mammalian olfactory cortex

During learning of overlapping input patterns in an associative memory, recall of previously stored patterns can interfere with the learning of new patterns. Most associative memory models avoid this difficulty by ignoring the effect of previously modified connections during learning, by clamping network activity to the patterns to be learned. Through the interaction of experimental and modeling techniques, we now have evidence to suggest that a somewhat analogous approach may have been taken by biology within the olfactory cerebral cortex. Specifically we have recently discovered that the naturally occurring neuromodulator acetylcholine produces a variety of effects on cortical cells and circuits which, when taken together, can prevent memory interference in a biologically realistic memory model. Further, it has been demonstrated that these biological mechanisms can actually improve the memory storage performance of previously published abstract neural network associative memory models.     

Analog implementation of ANN with inherent quadratic nonlinearity of the synapses

In real-life applications of multilayer neural networks, the scale of integration, processing speed, and manufacturability are of key importance. A simple analog-signal synapse model is implemented on a standard 0.35 /spl mu/m CMOS process requiring no floating-gate capability. A neural-matrix of 2176 analog current-mode synapses arranged in eight layers of 16 neurons with 16 inputs each is constructed for the purpose of a fingerprint feature extraction application. Synapse weights are stored on the analog storage capacitors, and synapse nonlinearity with respect to weight is investigated. The capability of the synapse to operate in feedforward and learning modes is studied and demonstrated. The effect of the synapse's inherent quadratic nonlinearity on learning convergence and on the optimization of vector direction is analyzed. Transistor-level analog simulations verify the hardware circuit. System-level MatLab simulations verify the synapse mathematical model. The conclusion reached is that the proposed implementation is very suitable for large-scale artificial neural networks - especially if on-chip integration with other products on a standard CMOS process is required.     

A Reconfigurable Bit-Serial VLSI Systolic Array Neuro-Chip

A dynamically reconfigurable bit-serial systolic array implemented in 1.2-μm double-metal P-well CMOS is described. This processor array is proposed as the central computational unit in the Reconfigurable Systolic Array (RSA) neuro-computer and performance estimates suggest that a 64 IC system (containing a total of 1024 usable processors) can achieve a learning rate of 1134 MCUPS on the NETtalk problem. The architecture employs reconfiguration techniques for both fault-tolerance and functionality, and allows a number of neural network models (in both the recall and learning phases) from associative memory networks, supervised networks, and unsupervised networks to be supported.