Flexible feature discovery and structural information control

In this paper, we propose a new information theoretic method called structural information control for flexible feature discovery. The new method has three distinctive characteristics, which traditional competitive learning fails to offer. First, the new method can directly control competitive unit activation patterns, whereas traditional competitive learning does not have any means to control them. Thus, with the new method, it is possible to extract salient features not discovered by traditional methods. Second, competitive units compete witheach other by maximizing their information content about input patterns. Consequently, this information maximization makes it possible to control flexibly competition processes. Third, in structural information control, it is possible to define many different kinds of information content, and we can choose a specific type of information according to a given objective. When applied to competitive learning, structural information can be used to control the number of dead or spare units, and to extract macro as well as micro features of input patterns in explicit ways. We first applied this method to simple pattern classification to demonstrate that information can be controlled and that different neuron firing patterns can be generated. Second, a dipole problem was used to show that structural information could provide representations similar to those by the conventional competitive learning methods. Finally, we applied the method to a language acquisition problem in which networks must flexibly discover some linguistic rules by changing structural information. Especially, we attempted to examine the effect of the information parameter to control the number of dead neurons, and thus to examine how macro and micro features in input patterns can explicitly be discovered by structural information.     

Improving feature extraction performance of greedy network-growing algorithm by inverse euclidean distance

We propose here a new computational method for the information-theoretic method, called the greedy network-growing algorithm, to facilitate a process of information acquisition. We have so far used the sigmoidal activation function for competitive unit outputs. The method can effectively suppress many competitive units by generating strongly negative connections. However, because methods with the sigmoidal activation function are not very sensitive to input patterns, we have observed that in some cases final representations obtained by the method do not necessarily faithfully describe input patterns. To remedy this shortcoming, we employ the inverse of distance between input patterns and connection weights for competitive unit outputs. As the distance becomes smaller, competitive units are more strongly activated. Thus, winning units tend to represent input patterns more faithfully than in the previous method with the sigmoidal activation function. We applied the new method to artificial data analysis and animal classification. Experimental results confirmed that more information can be acquired and more explicit features can be extracted by our new method.     

Teacher-directed learning: information-theoretic competitive learning in supervised multi-layered networks

In this paper, we propose a new type of efficient learning method called teacher-directed learning. The method can accept training patterns and correlated teachers, and we need not back-propagate errors between targets and outputs into networks. Information flows always from an input layer to an output layer. In addition, connections to be updated are those from an input layer to the first competitive layer. All other connections can take fixed values. Learning is realized as a competitive process by maximizing information on training patterns and correlated teachers. Because information is maximized, information is compressed into networks in simple ways, which enables us to discover salient features in input patterns. We applied this method to the vertical and horizontal lines detection problem, the analysis of US–Japan trade relations and a fairly complex syntactic analysis system. Experimental results confirmed that teacher information in an input layer forces networks to produce correct answers. In addition, because of maximized information in competitive units, easily interpretable internal representations can be obtained.     

Information theoretic competitive learning in self-adaptive multi-layered networks

In this paper, we propose self-adaptive multi-layered networks in which information in each processing layer is always maximized. Using these multi-layered networks, we can solve complex problems and discover salient features that single-layered networks fail to extract. In addition, this successive information maximization enables networks gradually to extract important features. We applied the new method to the Iris data problem, the vertical-horizontal lines detection problem, a phonological data analysis problem and a medical data problem. Experimental results confirmed that information can repeatedly be maximized in multi-layered networks and that the networks can extract features that cannot be detected by single-layered networks. In addition, features extracted in successive layers are cumulatively combined to detect more macroscopic features.     

Analysis of Hidden Representations by Greedy Clustering

The hidden layer of backpropagation neural networks (NNs) holds the key to the networks' success in solving pattern classification problems. The units in the hidden layer encapsulate the network's internal representations of the outside world described by the input data. this paper, the hidden representations of trained networks are investigated by means simple greedy clustering algorithm. This clustering algorithm is applied to networks have been trained to solve well-known problems: the monks problems, the 5-bit problem and the contiguity problem. The results from applying the algorithm to problems with known concepts provide us with a better understanding of NN learning. These also explain why NNs achieve higher predictive accuracy than that of decision-tree methods. The results of this study can be readily applied to rule extraction from Production rules are extracted for the parity and the monks problems, as well as benchmark data set: Pima Indian diabetes diagnosis. The extracted rules from the Indian diabetes data set compare favorably with rules extracted from ARTMAP NNs terms of predictive accuracy and simplicity.     

Complex Connectionist Dynamics and the Limited Complexity of the Resulting Functions

The paper demonstrates how algorithmic information theory can be elegantly used as a powerful tool for analyzing the dynamics in connectionist systems. It is shown that simple structures of connectionist systems-even if they are very large-are unable significantly to ease the problem of learning complex functions. Also, the development of new learning algorithms would not essentially change this situation. Lower and upper bounds are given for the number of examples needed to learn complex concepts. The bounds are proved with respect to the notion of probably approximately correct learning. It is proposed to use algorithmic information theory for further studies on network dynamics.     

Associative Memory Content-ADDRESSED Memory Fault-TOLERANCE Multiple Associative Recall Adjustable-PRECISION Memory Binary Mappings Information Retrieval Address-BASED Memory High-CAPACITY Memory Interference

This paper presents an approach to design of a neural architecture for both associative (content-addressed) and address-based memories. Several interesting properties of the memory module are mathematically analyzed in detail. When used as an associative memory, the proposed neural memory module supports recall from partial input patterns, (sequential) multiple recalls and fault-tolerance. When used as an address-based memory, the memory module can provide working space for dynamic representations for symbol processing and shared message-passing among neural network modules within an integrated neural network system. It also provides for real-time update of memory contents by one-shot learning without interference with other stored patterns.     

Molten salt phase diagram evaluation by pattern recognition:Part Ⅰ Divalent rare earth halide and alkali metal halide binary systems

At present CALPHAD (CALculation of PHAse Diagram) technique is not capable of predicting whether there exists intermediate compound, much less predicting the formulae, the number, and the melting congruence of intermediate compounds. To solve this problem, a new approach called the phase diagram evaluation by pattern recognition (PDEPR) was improved. The micro-parameters, such as the radius and the electronegativity of the element, were used as original features and then they were transformed and spanned to the different features in multi-dimensional space.Then a set of classifying functions were obtained to predict the information of intermediate compounds in REX2-AX systems (RE-rare earth element; A-Li, Na, K, Rb, and Cs; X F, Cl, Br, and 1). It is comparatively important for the design of materials.     

Molten salt phase diagram evaluation by pattern recognition：Part I Divalent rare earth halide and alkali metal halide binary systems

At present CALPHAD (CALculation of PHAse Diagram) technique is not capable of predicting whether there exists intermediate compound, much less predicting the formulae, the number, and the melting congruence of intermediate compounds. To solve this problem, a new approach called the phase diagram evaluation by pattern recognition (PDEPR) was improved. The micro-parameters, such as the radius and the electronegativity of the element, were used as original features and then they were transformed and spanned to the different features in multi-dimensional space. Then a set of classifying functions were obtained to predict the information of intermediate compounds in REX2-AX systems (RE-rare earth element; A--Li, Na, K, Rb, and Cs; X--F, CI, Br, and I). It is comparatively important for the design of materials.     

Improving visualisation and prediction performance of supervised self-organising map by modified contradiction resolution

We have previously proposed a new type of information-theoretic method where a neuron is evaluated by itself (self-evaluation) and by its surrounding neurons (outer-evaluation). If contradiction between different types of evaluation exists, it is reduced as much as possible. In the present paper, we try to separate self- and outer-evaluation more explicitly and introduce the importance of neurons. First, we separate self- and outer-evaluation to enhance the characteristics shared by the two types of evaluation. Second, we introduce the importance of neurons in evaluation. By using a limited number of important neurons in evaluation, we expect the main characteristics in input patterns to emerge. We applied this contradiction resolution to two types of data, namely, the Senate data and the Euro-yen exchange rates. In both data sets, experimental results confirmed that improved prediction performance was obtained. Prediction performance was better than that obtained by the conventional self-organising map (SOM) and radial basis function networks. In addition, final representations obtained by contradiction resolution were easier to interpret than those given by the conventional SOM. Experimental results confirmed that improved interpretation and visualisation were accompanied by improved prediction performance.     

Pseudo-recurrent Connectionist Networks: An Approach to the 'Sensitivity-Stability' Dilemma

In order to solve the 'sensitivity-stability' problem-and its immediate correlate, the problem of sequential learning-it is crucial to develop connectionist architectures that are simultaneously sensitive to, but not excessively disrupted by, new input. French (1992) suggested that to alleviate a particularly severe form of this disruption, catastrophic forgetting, it was necessary for networks to separate dynamically their internal representations during learning. McClelland et al. (1995) went even further. They suggested that nature's way of implementing this obligatory separation was the evolution of two separate areas of the brain, the hippocampus and the neocortex. In keeping with this idea of radical separation, a 'pseudo-recurrent' memory model is presented here that partitions a connectionist network into two functionally distinct, but continually interacting areas. One area serves as a final-storage area for representations; the other is an early-processing area where new representations are first learned by the system. The final-storage area continually supplies internally generated patterns (pseudopatterns; Robins, 1995), which are approximations of its content, to the early-processing area, where they are interleaved with the new patterns to be learned. Transfer of the new learning is done either by weight-copying from the early-processing area to the final-storage area or by pseudopattern transfer. A number of experiments are presented that demonstrate the effectiveness of this approach, allowing, in particular, effective sequential learning with gradual forgetting in the presence of new input. Finally, it is shown that the two interacting areas automatically produce representational compaction and it is suggested that similar representational streamlining may exist in the brain.     

A Dynamic Neural Substrate and Automatic Perception Switching

This paper presents DENN, a dynamic neural network or neural substrate having a number of abilities that might allow it to play a useful role as a constituent of an artificial cognitive system, handling the task of low-level perceptual processing. DENN can adapt without supervision to new objects, is able to respond to patterns of activation from several objects presented simultaneously to it, and is able to automatically switch its perception between multiple objects. It is based on an ideal neural substrate as conjectured by Dimond (1980), having the twin capabilities of autonomous learning and memory, capabilities emerging due to the use of autonomous neurons. DENN has a pyramidal architecture and its neurons have topologically organized receptive fields. Through training, the neurons become feature detectors, with the higher level neurons responding to more complex features. The neurons respond to a retinal input with an oscillatory output whose frequency depends only on their own input. Due to developing phase differences, the higher level neurons can move out of phase relative to each other. Therefore, different inputs are recognized cyclically-a process we term 'automatic perception switching'. Experiments verified the system's ability of automatic perception switching, investigated its response to randomized images, and compared the performance of adaptive and non-adaptive versions of the neural substrate.     

Manipulating space: modelling the role of transient dynamics in inattentional blindness

According to Noë’s enactive theory of perception, sensorimotor knowledge allows us to predict the sensory outcomes of our actions. This paper suggests that tuning input filters with such predictions may be the cause of sustained inattentional blindness. Most models of learning capture statistically salient regularities in and between data streams. Such analysis is, however, severely limited by both the problem of marginal regularity and the credit assignment problem. A neurocomputational reservoir system can be used to alleviate these problems without training by enhancing the separability of regularities in input streams. However, as the regularities made separable vary with the state of the reservoir, feedback in the form of predictions of future sensory input can both enhance expected discriminations and hinder unanticipated ones. This renders the model blind to features not made separable in the regions of state space the reservoir is manipulated towards. This is demonstrated in a computational model of sustained inattentional blindness, leading to predictions about human behaviour that have yet to be tested.     

Optimized tool path generation based on dynamic programming for five-axis flank milling of rule surface

This paper presents a computation scheme that generates optimized tool path for five-axis flank milling of ruled surface. Tool path planning is transformed into a matching problem between two point sets in 3D space, sampled from the boundary curves of the machined surface. Each connection in the matching corresponds to a possible tool position. Dynamic programming techniques are applied to obtain the optimal combination of tool positions with the objective function as machining error. The error estimation considers both the deviation induced by the cutter at discrete positions and the one between them. The path planning problem is thus solved in a systematic manner by formulizing it as a mathematical programming task. In addition, the scheme incorporates several optimization parameters that allow generating new patterns of tool motion. Implementation results obtained from simulation and experiment indicate that our method produces better machining quality. This work provides a concise but effective approach for machining error control in five-axis flank milling.     

Temporal sequence detection with spiking neurons: towards recognizing robot language instructions

We present an approach for recognition and clustering of spatio temporal patterns based on networks of spiking neurons with active dendrites and dynamic synapses. We introduce a new model of an integrate-and-fire neuron with active dendrites and dynamic synapses (ADDS) and its synaptic plasticity rule. The neuron employs the dynamics of the synapses and the active properties of the dendrites as an adaptive mechanism for maximizing its response to a specific spatio-temporal distribution of incoming action potentials. The learning algorithm follows recent biological evidence on synaptic plasticity. It goes beyond the current computational approaches which are based only on the relative timing between single pre- and post-synaptic spikes and implements a functional dependence based on the state of the dendritic and somatic membrane potentials around the pre- and post-synaptic action potentials. The learning algorithm is demonstrated to effectively train the neuron towards a selective response determined by the spatio-temporal pattern of the onsets of input spike trains. The model is used in the implementation of a part of a robotic system for natural language instructions. We test the model with a robot whose goal is to recognize and execute language instructions. The research in this article demonstrates the potential of spiking neurons for processing spatio-temporal patterns and the experiments present spiking neural networks as a paradigm which can be applied for modelling sequence detectors at word level for robot instructions.     

Self-organised manifold learning and heuristic charting via adaptive metrics

Classical metric and non-metric multidimensional scaling (MDS) variants represent the well-known manifold learning (ML) methods which enable construction of low-dimensional representation (projections) of high-dimensional data inputs. However, their use is limited to the cases when data are inherently reducible to low dimensionality. In general, drawbacks and limitations of these, as well as pure, MDS variants become more apparent when the exploration (learning) is exposed to the structured data of high intrinsic dimension. As we demonstrate on artificial as well as real-world datasets, the over-determination problem can be solved by means of the hybrid and multi-component discrete-continuous multi-modal optimisation heuristics. A remarkable feature of the approach is that projections onto 2D are constructed simultaneously with the data categorisation compensating in part for the loss of original input information. We observed that the optimisation module integrated with ML modelling, metric learning and categorisation leads to a nontrivial mechanism resulting in heuristic charting of data.     

焊接电弧信息测试分析系统

以焊接电流、电弧电压、电弧光谱为信息源，开发了一套可进行信号采集、信号处理、信息提取的焊接电弧信息测试分析系统，并可实现同步高速摄影。研制的电弧电压、焊接电流传感器具有良好的强弱电隔离性能，良好的输出信号品质，很好地保护了采集卡及计算机。设计的电弧光谱传感器信噪比高，可得到品质良好的电弧光谱信号。基于Visualbasic语言开发的软件系统可采集多路焊接电弧信号，并可对采集的信号进行瞬时分析和统计分析，以提取有价值的焊接信息。     

神经网络信息融合技术在岩石挖掘机系统故障诊断中的应用

将信息融合技术与神经网络结合起来，充分利用检测到的各种故障征兆信息准确诊断岩石挖掘机系统的故障。采用混合式数据融合方法，将数据级、特征级和决策级融合通过神经网络的方法综合在一起，解决输入信息不对称性问题，使得小数据量和大数据量的信息融合成为可能。     

Validation of an information model for inspection with CMM

Application of Simultaneous Engineering concept can be made at two levels: the first one refers to project work and administration methods, reorganizing teams; the second level refers to the use of computer systems which not only transmits product and process information among them, but also allows them to share it [S. Bloor, J. Owen, Product Data Exchange, UCL Press Limited, London, 1995]. Nowadays, several research groups are working on this second alternative.In this context, the definition of a product information model is very useful: a mechanism able to provide product information to the activities in the production cycle [P. Gu, K. Chang, Product Modelling Using STEP, Computer Aided Design, 27(3) (1995) 163–179; P.R. Wilson, Information Modelling, IEEE Computer Graphics and Applications, 7(12) (1987) 65–67]. This model should satisfy the different points of view that the product lifecycle activities have about the same information [O.J. Canciglieri, R.I.M. Young, A multi-viewpoint reasoning system in design for injection moulding, International Conference on Design and Production of Dies and Molds, Istambul, 19–21 Junio, 1997, pp. 195–205; T. de Martino, B. Falcidieno, S. Habinger, Design and engineering process integration through a multiple view intermediate modeller in a distributed object-oriented system environment, Computer Aided Design, 30(6) (1998) 437–452; Product Data Representation and Exchange. Part 1. Overview and Fundamental Principles, ISO 10303 Part 1, ISO, 1993]: functionality, geometry, manufacturability or inspectionability.STEP standard [Product Data Representation and Exchange. Part 1. Overview and Fundamental Principles, ISO 10303 Part 1, ISO, 1993] provides a way to solve the problem of information integration along the cycle. However, this standard still does not consider the dimensional inspection process integration. This paper shows a solution to integrate the dimensional inspection process in the production cycle using coordinate measuring machines. The integration is achieved by means of an information model implemented in a product central database, accessible to every activity in the cycle. The information model has been defined according to STEP standard philosophy, in such a way that interaction with other information models related to other activities in the cycle is possible.A new framework called IFCIA is presented. Here, the model has been tested taking as reference the information objects included in the previously developed model. This framework was already introduced in Refs. [J. Barreiro, J.E. Labarga, A. Vizán, J. Ríos, Information model for the integration of the inspection activity in a concurrent engineering framework, International Journal of Machine Tools and Manufacture, 43(8) (2003) 797–809; J. Barreiro, J.E. Labarga, A. Vizán, J. Ríos, Functional model for the development of an inspection integration framework, International Journal of Machine Tools and Manufacturing, 43(15) (2003) 1621–1632] but it is explained in detail now. The framework architecture and the difficulties that arise to integrate dimensional measuring equipments in the cycle are shown.     

Pattern recognition of weld defects detected by radiographic test

In recent years there has been a marked advance in the research for the development of an automatized system to analyze weld defects detected by radiographs. This work describes a study of nonlinear pattern classifiers, implemented by artificial neural networks, to classify weld defects existent in radiographic weld beads, aiming principally to increase the percentage of defect recognition success obtained with the linear classifiers. Radiographic patterns from International Institute of Welding (IIW) were used. Geometric features of defect classes were used as input data of the classifiers. Using a novel approach for this area of research, a criterion of neural relevance was applied to evaluate the discrimination capacity of the classes studied by the features used, aiming to prove that the quality of the features is more important than the quantity of features used. Well known for other applications, but still not exploited in weld defect recognition, the analytical techniques of the principal nonlinear discrimination components, also developed by neural networks, are presented to show the classification problem in two dimensions, as well as evaluating the classification performance obtained with these techniques. The results prove the efficiency of the techniques for the data used.