Context-free and context-sensitive dynamics in recurrent neural networks

Continuous-valued recurrent neural networks can learn mechanisms for processing context-free languages. The dynamics of such networks is usually based on damped oscillation around fixed points in state space and requires that the dynamical components are arranged in certain ways. It is shown that qualitatively similar dynamics with similar constraints hold for aⁿbⁿcⁿ , a context-sensitive language. The additional difficulty with aⁿbⁿcⁿ , compared with the context-free language aⁿbⁿ , consists of 'counting up' and 'counting down' letters simultaneously. The network solution is to oscillate in two principal dimensions, one for counting up and one for counting down. This study focuses on the dynamics employed by the sequential cascaded network, in contrast to the simple recurrent network, and the use of backpropagation through time. Found solutions generalize well beyond training data, however, learning is not reliable. The contribution of this study lies in demonstrating how the dynamics in recurrent neural networks that process context-free languages can also be employed in processing some context-sensitive languages (traditionally thought of as requiring additional computation resources). This continuity of mechanism between language classes contributes to our understanding of neural networks in modelling language learning and processing.     

Connectionist simulation of quantification skills

The study of numerical abilities, and how they are acquired, is being used to explore the continuity between ontogenesis and environmental learning. One technique that proves useful in this exploration is the artificial simulation of numerical abilities with neural networks, using different learning paradigms to explore development. A neural network simulation of subitization, sometimes referred to as visual enumeration, and of counting, a recurrent operation, has been developed using the so-called multi-net architecture. Our numerical ability simulations use two or more neural networks combining supervised and unsupervised learning techniques to model subitization and counting. Subitization has been simulated using networks employing unsupervised self-organizing learning, the results of which agree with infant subitization experiments and are comparable with supervised neural network simulations of subitization reported in the literature. Counting has been simulated using a multi-net system of supervised static and recurrent backpropagation networks that learn their individual tasks within an unsupervised, competitive framework. The developmental profile of the counting simulation shows similarities to that of children learning to count and demonstrates how neural networks can learn how to be combined together in a process modelling development.     

Kinematic motor learning

This paper focuses on adaptive motor control in the kinematic domain. Several motor-learning strategies from the literature are adopted to kinematic problems: ‘feedback-error learning’, ‘distal supervised learning’, and ‘direct inverse modelling’ (DIM). One of these learning strategies, DIM, is significantly enhanced by combining it with abstract recurrent neural networks. Moreover, a newly developed learning strategy (‘learning by averaging’) is presented in detail. The performance of these learning strategies is compared with different learning tasks on two simulated robot setups (a robot-camera-head and a planar arm). The results indicate a general superiority of DIM if combined with abstract recurrent neural networks. Learning by averaging shows consistent success if the motor task is constrained by special requirements.     

A systematic comparison of flat and standard cascade-correlation using a student–teacher network approximation task

Cascade-correlation (cascor) networks grow by recruiting hidden units to adjust their computational power to the task being learned. The standard cascor algorithm recruits each hidden unit on a new layer, creating deep networks. In contrast, the flat cascor variant adds all recruited hidden units on a single hidden layer. Student–teacher network approximation tasks were used to investigate the ability of flat and standard cascor networks to learn the input–output mapping of other, randomly initialized flat and standard cascor networks. For low-complexity approximation tasks, there was no significant performance difference between flat and standard student networks. Contrary to the common belief that standard cascor does not generalize well due to cascading weights creating deep networks, we found that both standard and flat cascor generalized well on problems of varying complexity. On high-complexity tasks, flat cascor networks had fewer connection weights and learned with less computational cost than standard networks did.     

Modifying the Generalized Delta Rule to Train Networks of Non-monotonic Processors for Pattern Classification

A modification of the generalized delta rule is described that is capable of training multilayer networks of value units, i.e. units defined by a particular non-monotonic activation function, the Gaussian, For simple problems of pattern classification, this rule produces networks with several advantages over standard feedforward networks: they require fewer processing units and can be trained much more quickly. Though superficially similar, there are fundamental differences between the networks trained by this new learning rule and radial basis function networks. These differences suggest that value unit networks may be better suited for learning some pattern classification tasks and for answering general questions related to the organization of neurophysiological systems.     

The classification game: combining supervised learning and language evolution

We study the emergence of shared representations in a population of agents engaged in a supervised classification task, using a model called the classification game. We connect languages with tasks by treating the agents’ classification hypothesis space as an information channel. We show that by learning through the classification game, agents can implicitly perform complexity regularisation, which improves generalisation. Improved generalisation also means that the languages that emerge are well adapted to the given task. The improved language-task fit springs from the interplay of two opposing forces: the dynamics of collective learning impose a preference for simple representations, while the intricacy of the classification task imposes a pressure towards representations that are more complex. The push–pull of these two forces results in the emergence of a shared representation that is simple but not too simple. Our agents use artificial neural networks to solve the classification tasks they face, and a simple counting algorithm to learn a language as a form-meaning mapping. We present several experiments to demonstrate that both compositional and holistic languages can emerge in our system. We also demonstrate that the agents avoid overfitting on noisy data, and can learn some very difficult tasks through interaction, which they are unable to learn individually. Further, when the agents use simple recurrent networks to solve temporal classification tasks, we see the emergence of a rudimentary grammar, which does not have to be explicitly learned.     

A Local Learning Algorithm for Dynamic Feedforward and Recurrent Networks

Most known learning algorithms for dynamic neural networks in non-stationary environments need global computations to perform credit assignment. These algorithms either are not local in time or not local in space. Those algorithms which are local in both time and space usually cannot deal sensibly with ‘hidden units’. In contrast, as far as we can judge, learning rules in biological systems with many ‘hidden units’ are local in both space and time. In this paper we propose a parallel on-line learning algorithms which performs local computations only, yet still is designed to deal with hidden units and with units whose past activations are ‘hidden in time’. The approach is inspired by Holland's idea of the bucket brigade for classifier systems, which is transformed to run on a neural network with fixed topology. The result is a feedforward or recurrent ‘neural’ dissipative system which is consuming ‘weight-substance’ and permanently trying to distribute this substance onto its connections in an appropriate way. Simple experiments demonstrating the feasibility of the algorithm are reported.     

Hybridization and Specialization of Real-time Recurrent Learning-based Neural Networks

In this article, three different methods for hybridization and specialization of real-time recurrent learning (RTRL)-based neural networks (NNs) are presented. The first approach consists of combining recurrent networks with feedforward networks. The second approach continues with the combination of multiple recurrent NNs. The last approach introduces the combination of connectionist systems with instructionist artificial intelligence techniques. Two examples are added to demonstrate properties and advantages of these techniques. The first example is a process diagnosis task where a hybrid NN is connected to a knowledge-based system. The second example is a NN consisting of different recurrent modules that is used to handle missing sensor data in a process modelling task.     

Dynamic Update of the Reinforcement Function During Learning

During the last decade, numerous contributions have been made to the use of reinforcement learning in the robot learning field. They have focused mainly on the generalization, memorization and exploration issues-mandatory for dealing with real robots. However, it is our opinion that the most difficult task today is to obtain the definition of the reinforcement function (RF). A first attempt in this direction was made by introducing a method-the update parameters algorithm (UPA)-for tuning a RF in such a way that it would be optimal during the exploration phase. The only requirement is to conform to a particular expression of RF. In this article, we propose Dynamic-UPA, an algorithm able to tune the RF parameters during the whole learning phase (exploration and exploitation). It allows one to undertake the so-called exploration versus exploitation dilemma through careful computation of the RF parameter values by controlling the ratio between positive and negative reinforcement during learning. Experiments with the mobile robot Khepera in tasks of synthesis of obstacle avoidance and wall-following behaviors validate our proposals.     

基于CMAC神经网络的液压电梯实时控制研究

本文在ＣＭＡＣ的基础上提出一种实时控制算法,算法以时间为ＣＭＡＣ的输入矢量,在结构上与ＰＤ控制复合构成控制器。文中对电梯速度曲线的离线和在线两种学习方法进行了讨论和实验,结果表明采用离线学习的ＣＭＡＣ控制器较ＣＭＡＣ在线学习控制器和常规的ＰＩＤ具有更优良的控制品质。     

A Connectionist Model of Choice and Reaction Time in Absolute Identification

A connectionist architecture is developed that can be used for modeling choice probabilities and reaction times in identification tasks. The architecture consists of a feedforward network and a decoding module, and learning is by mean-variance back-propagation, an extension of the standard back-propagation learning algorithm. We suggest that the new learning procedure leads to a better model of human learning in simple identification tasks than does standard back-propagation. Choice probabilities are modeled by the input-output relations of the network and reaction times are modeled by the time taken for the network, particularly the decoding module, to achieve a stable state. In this paper, the model is applied to the identification of unidimensional stimuli; applications to the identification of multidimensional stimuli—visual displays and words—is mentioned and presented in more detail in other papers. The strengths and weaknesses of this connectionist approach vis-à-vis other approaches are discussed     

基于模糊神经网络的冷连轧机轧制力预测

建立了一种基于数学模型和模糊神经网络共同作用的冷连轧机轧制力预测模型,通过数学模型描述轧制接触面积,模糊网络预测轧制单位压力.提出将Hough变换应用于神经网络的参数确定,从而使最终设计的网络具有最佳结构参数.试验研究证明了所设计模型具有较强的泛化能力和鲁棒性,大大提高了轧制力的预报精度.     

The Behaviour of Affine Boolean Sequential Networks

Whereas a significant amount of empirical work has been carried out in the area of logical neural networks, the fundamental theoretical basis is underdeveloped. This paper provides an analysis of recurrent nets of affine Boolean functions and defines the relation between their behaviour and that of linear Boolean nets. It is shown that the only effect on the unlabelled state structure of adding any number of inverters to a linear Boolean net is a restricted modification of cycle lengths.     

A constrained neural learning rule for eliminating the border effect in online self-organising maps

The self-organising map (SOM) is a concise and powerful algorithm for clustering and visualisation of high-dimensional data. However, this robust algorithm still suffers from the border effect. Most of the approaches proposed to eliminate this effect use a borderless topological structure. We prefer to keep the original topological structure of the SOM for visualisation. A novel approach is proposed for the elimination of the border effect from the perspective of self-organising learning. Based on an assumption that the best matching unit (BMU) should be the most active unit, the approach proposes that the BMU should move more towards its associated input sample than its neighbours in the fine-tuned learning stage. Our constrained approach emphasises the effect of the lateral connections and neutralises the effect on the distance between the input sample and units. This approach is able to make units of the map stretch wider than the traditional SOM and thus the border effect is alleviated. Our proposed approach is proved to satisfy the requirements of the topologically ordered neural networks and is evaluated by both qualitative and quantitative criteria. All experiments conclude that performance is improved if the proposed constrained learning rule is used.     

An efficient MLP-learning algorithm using parallel tangent gradient and improved adaptive learning rates

Poor convergence is a common problem of gradient-based multi-layer perceptron (MLP)-learning algorithms. It is claimed that using a deflecting direction like momentum and adaptive learning rates (ALRs) can improve the convergence performance. For a more reliable and faster MLP learning, we introduce the parallel tangent gradient with adaptive learning rates (PTGALR) algorithm that uses parallel tangent deflecting direction instead of the momentum. Moreover, we use two independent variable learning rates, one for the gradient descent and the other for accelerating direction through the parallel tangent. Also, we propose an improved ALR computation algorithm that calculates the learning rates with a dispensable error oscillation. This adaptation algorithm has two outputs: one is the applied learning rate and the other is used for a better learning rate estimation in the next iteration. Moreover, the proposed ALR computation algorithm models the error function as a one-dimensional quadratic function of the learning rate when it is needed. The implementation results of PTGALR for some well-known binary and real MLP problems show higher and faster convergence with lower oscillations than the similar adaptive learning algorithms.     

Neural networks with a self-refreshing memory: Knowledge transfer in sequential learning tasks without catastrophic forgetting

We explore a dual-network architecture with self-refreshing memory (Ans and Rousset 1997) which overcomes catastrophic forgetting in sequential learning tasks. Its principle is that new knowledge is learned along with an internally generated activity reflecting the network history. What mainly distinguishes this model from others using pseudorehearsal in feedforward multilayer networks is a reverberating process used for generating pseudoitems. This process, which tends to go up to network attractors from random activation, is more suitable for capturing optimally the deep structure of previously learned knowledge than a single feedforward pass of activity. The proposed mechanism for ?transporting memory? without loss of information between two different brain structures could be viewed as a neurobiologically plausible means for consolidation in long-term memory. Knowledge transfer is explored with regard to learning speed, ability to generalize and vulnerability to network damages. We show that transfer is more efficient when two related tasks are sequentially learned than when they are learned concurrently. With a self-refreshing memory network knowledge can be saved for a long time and therefore reused in subsequent acquisitions.     

仿人脑多巴胺调控机制的机器人视觉自发育学习算法研究

针对传统机器人学习离线、任务特定、无法在学习过程中扩展智能，且实时性、适应性差等问题，借鉴认知机器人、神经生物学、认知科学等思想，提出一种仿人脑多巴胺调控机制的机器人视觉自发育学习算法，模拟人脑海马与前额叶神经回路，用多巴胺调控学习进程，实现仿人类的视觉学习能力。实验结果表明：该算法能有效地实现视觉图像的自发育学习，能完成非特定任务，且识别率高，实时性好，适应性强。     

Behavioural Aspects of Combining Backpropagation Learning and Self-organizing Maps

Backpropagation learning (BP) is known for its serious limitations in generalizing knowledge from certain types of learning material. In this paper, we describe a new learning algorithm, BP-SOM, which overcomes some of these limitations as is shown by its application to four benchmark tasks. BP-SOM is a combination of a multi-layered feedforward network (MFN) trained with BP and Kohonen's self-organizing maps (SOMs). During the learning process, hidden-unit activations of the MFN are presented as learning vectors to SOMs trained in parallel. The SOM information is used when updating the connection weights of the MFN in addition to standard error backpropagation. The effect of the augmented error signal is that, during learning, clusters of hiddenunit activation patterns of instances associated with the same class tend to become highly similar. In a number of experiments, BP-SOM is shown (i) to improve generalization performance (i.e. avoid overfitting); (ii) to increase the amount of hidden units that can be pruned without loss of generalization performance and (iii) to provide a means for automatic rule extraction from trained networks. The results are compared with results achieved by two other learning algorithms for MFNs: conventional BP and BP augmented with weight decay. From the experiments and the comparisons, we conclude that the hybrid BP-SOM architecture, in which supervised and unsupervised and learning co-operate in finding adequate hidden-layer representations, successfully combines the advantages of supervised and unsupervised learning.     

Modeling development of natural multi-sensory integration using neural self-organisation and probabilistic population codes

Humans and other animals have been shown to perform near-optimally in multi-sensory integration tasks. Probabilistic population codes (PPCs) have been proposed as a mechanism by which optimal integration can be accomplished. Previous approaches have focussed on how neural networks might produce PPCs from sensory input or perform calculations using them, like combining multiple PPCs. Less attention has been given to the question of how the necessary organisation of neurons can arise and how the required knowledge about the input statistics can be learned. In this paper, we propose a model of learning multi-sensory integration based on an unsupervised learning algorithm in which an artificial neural network learns the noise characteristics of each of its sources of input. Our algorithm borrows from the self-organising map the ability to learn latent-variable models of the input and extends it to learning to produce a PPC approximating a probability density function over the latent variable behind its (noisy) input. The neurons in our network are only required to perform simple calculations and we make few assumptions about input noise properties and tuning functions. We report on a neurorobotic experiment in which we apply our algorithm to multi-sensory integration in a humanoid robot to demonstrate its effectiveness and compare it to human multi-sensory integration on the behavioural level. We also show in simulations that our algorithm performs near-optimally under certain plausible conditions, and that it reproduces important aspects of natural multi-sensory integration on the neural level.     

Function Optimization using Connectionist Reinforcement Learning Algorithms

Any non-associative reinforcement learning algorithm can be viewed as a method for performing function optimization through (possibly noise-corrupted) sampling of function values. We describe the results of simulations in which the optima of several deterministic functions studied by Ackley were sought using variants of REINFORCE algorithms. Some of the algorithms used here incorporated additional heuristic features resembling certain aspects of some of the algorithms used in Ackley's studies. Differing levels of performance were achieved by the various algorithms investigated, but a number of them performed at a level comparable to the best found in Ackley's studies on a number of the tasks, in spite of their simplicity. One of these variants, called REINFORCE/MENT, represents a novel but principled approach to reinforcement learning in nontrivial networks which incorporates an entropy maximization strategy. This was found to perform especially well on more hierarchically organized tasks.