Multilayer perceptron, fuzzy sets, and classification

A fuzzy neural network model based on the multilayer perceptron, using the backpropagation algorithm, and capable of fuzzy classification of patterns is described. The input vector consists of membership values to linguistic properties while the output vector is defined in terms of fuzzy class membership values. This allows efficient modeling of fuzzy uncertain patterns with appropriate weights being assigned to the backpropagated errors depending upon the membership values at the corresponding outputs. During training, the learning rate is gradually decreased in discrete steps until the network converges to a minimum error solution. The effectiveness of the algorithm is demonstrated on a speech recognition problem. The results are compared with those of the conventional MLP, the Bayes classifier, and other related models.     

Functional-link net with fuzzy integral for bankruptcy prediction

The classification ability of a single-layer perceptron could be improved by considering some enhanced features. In particular, this form of neural networks is called a functional-link net. In the output neuron's activation function, such as the sigmoid function, an inner product of a connection weight vector with an input vector is computed. However, since the input features are not independent of each other for the enhanced pattern, an assumption of the additivity is not reasonable. This paper employs a non-additive technique, namely the fuzzy integral, to aggregate performance values for an input pattern by interpreting each of the connection weights as a fuzzy measure of the corresponding feature. A learning algorithm with the genetic algorithm is then designed to automatically find connection weights. The sample data for bankruptcy analysis obtained from Moody's Industrial Manuals is considered to examine the classification ability of the proposed method. The results demonstrate that the proposed method performs well in comparison with traditional functional-link net and multivariate techniques.     

在FCS环境中神经网络模糊控制的研究

本文提出了在ＦＣＳ环境中的一种模糊控制方法，该方法采用了模糊ＩＦ．ＴＨＥＮ规则和数字数据两种方法进行学习在ＦＣＳ环境中，系统的正确性不但依赖于其计算出的逻辑结果，而且还依赖于该结果的计算时间。针对上述特征，提出了一种能够控制模糊输入矢量的神经网络体系结构，它将一个模糊输入矢量映射到其模糊输出中。     

Iterative inversion of fuzzified neural networks

The inversion of a neural network is a process of computing inputs that produce a given target when fed into the neural network. The inversion algorithm of crisp neural networks is based on the gradient descent search in which a candidate inverse is iteratively refined to decrease the error between its output and the target. In this paper. we derive an inversion algorithm of fuzzified neural networks from that of crisp neural networks. First, we present a framework of learning algorithms of fuzzified neural networks and introduce the idea of adjusting schemes for fuzzy variables. Next, we derive the inversion algorithm of fuzzified neural networks by applying the adjusting scheme for fuzzy variables to total inputs in the input layer. Finally, we make three experiments on the parity-three problem, examine the effect of the size of training sets on the inversion, and investigate how the fuzziness of inputs and targets of training sets affects the inversion     

Solving fully fuzzy polynomials using feed-back neural networks

Recently, there has been a considerable amount of interest and practice in solving many problems of several applied fields by fuzzy polynomials. In this paper, we have designed an artificial fuzzified feed-back neural network. With this design, we are able to find a solution of fully fuzzy polynomial with degree n. This neural network can get a fuzzy vector as an input, and calculates its corresponding fuzzy output. It is clear that the input–output relation for each unit of fuzzy neural network is defined by the extension principle of Zadeh. In this work, a cost function is also defined for the level sets of fuzzy output and fuzzy target. Next a learning algorithm based on the gradient descent method will be defined that can adjust the fuzzy connection weights. Finally, our approach is illustrated by computer simulations on numerical examples. It is worthwhile to mention that application of this method in fluid mechanics has been shown by an example.     

A new backpropagation learning algorithm for layered neural networks with nondifferentiable units

We propose a digital version of the backpropagation algorithm (DBP) for three-layered neural networks with nondifferentiable binary units. This approach feeds teacher signals to both the middle and output layers, whereas with a simple perceptron, they are given only to the output layer. The additional teacher signals enable the DBP to update the coupling weights not only between the middle and output layers but also between the input and middle layers. A neural network based on DBP learning is fast and easy to implement in hardware. Simulation results for several linearly nonseparable problems such as XOR demonstrate that the DBP performs favorably when compared to the conventional approaches. Furthermore, in large-scale networks, simulation results indicate that the DBP provides high performance.     

Fuzzy rough granular neural networks, fuzzy granules, and classification

We introduce a fuzzy rough granular neural network (FRGNN) model based on the multilayer perceptron using a back-propagation algorithm for the fuzzy classification of patterns. We provide the development strategy of the network mainly based upon the input vector, initial connection weights determined by fuzzy rough set theoretic concepts, and the target vector. While the input vector is described in terms of fuzzy granules, the target vector is defined in terms of fuzzy class membership values and zeros. Crude domain knowledge about the initial data is represented in the form of a decision table, which is divided into subtables corresponding to different classes. The data in each decision table is converted into granular form. The syntax of these decision tables automatically determines the appropriate number of hidden nodes, while the dependency factors from all the decision tables are used as initial weights. The dependency factor of each attribute and the average degree of the dependency factor of all the attributes with respect to decision classes are considered as initial connection weights between the nodes of the input layer and the hidden layer, and the hidden layer and the output layer, respectively. The effectiveness of the proposed FRGNN is demonstrated on several real-life data sets.     

Possibility-based fuzzy neural networks and their application toimage processing

This paper describes the foundations for a class of fuzzy neural networks. Such a network is a composite or two-stage network consisting of a fuzzy network stage and a neural network stage. It exhibits the ability to classify complex feature set vectors with a configuration that is simpler than that needed by a standard neural network, Unlike a standard neural network, this network is able to accept as input a vector of scalar values, or a vector (set) of possibility functions. The first stage of the network is fuzzy based. It has two parts: a parameter computing network (PCN), followed by a converting layer. In the PCN the weights of the nodes are possibility functions, and hence, the output of this network is a fuzzy set. The second part of this stage, which is a single layer network, then converts this fuzzy set into a scalar vector for input to the second stage. The second stage of the network is a standard backpropagation based neural network. In addition to establishing the theoretical foundations for such a network, this paper presents sample applications of the network for classification problems in satellite image processing and seismic lithology pattern recognition.     

Neural networks that learn from fuzzy if-then rules

An architecture for neural networks that can handle fuzzy input vectors is proposed, and learning algorithms that utilize fuzzy if-then rules as well as numerical data in neural network learning for classification problems and for fuzzy control problems are derived. The learning algorithms can be viewed as an extension of the backpropagation algorithm to the case of fuzzy input vectors and fuzzy target outputs. Using the proposed methods, linguistic knowledge from human experts represented by fuzzy if-then rules and numerical data from measuring instruments can be integrated into a single information processing system (classification system or fuzzy control system). It is shown that the scheme works well for simple examples     

Real-coded chaotic quantum-inspired genetic algorithm for training of fuzzy neural networks

In this paper, a novel approach to adjusting the weightings of fuzzy neural networks using a Real-coded Chaotic Quantum-inspired genetic Algorithm (RCQGA) is proposed. Fuzzy neural networks are traditionally trained by using gradient-based methods, which may fall into local minimum during the learning process. To overcome the problems encountered by the conventional learning methods, RCQGA algorithms are adopted because of their capabilities of directed random search for global optimization. It is well known, however, that the searching speed of the conventional quantum genetic algorithms (QGA) is not satisfactory. In this paper, a real-coded chaotic quantum-inspired genetic algorithm (RCQGA) is proposed based on the chaotic and coherent characters of Q-bits. In this algorithm, real chromosomes are inversely mapped to Q-bits in the solution space. Q-bits probability-guided real cross and chaos mutation are applied to the evolution and searching of real chromosomes. Chromosomes consisting of the weightings of the fuzzy neural network are coded as an adjustable vector with real number components that are searched by the RCQGA. Simulation results have shown that faster convergence of the evolution process in searching for an optimal fuzzy neural network can be achieved. Examples of nonlinear functions approximated by using the fuzzy neural network via the RCQGA are demonstrated to illustrate the effectiveness of the proposed method.     

构造性核覆盖算法在图像识别中的应用

构造性神经网络的主要特点是：在对给定的具体数据的处理过程中,能同时给出网络的结构和参数;支持向量机就是先通过引入核函数的非线性变换,然后在这个核空间中求取最优线性分类面,其所求得的分类函数,形式上类似于一个神经网络,而构造性核覆盖算法(简称为CKCA)则是一种将神经网络中的构造性学习方法(如覆盖算法)与支持向量机(SVM)中的核函数法相结合的方法。CKCA方法具有运算量小、构造性强、直观等特点,适于处理大规模分类问题和图像识别问题。为验证CKCA算法的应用效果,利用图像质量不高的车牌字符进行了识别实验,并取得了较好的结果。     

Hierarchical hybrid fuzzy-neural networks for approximation with mixed input variables

Real-world systems usually involve both continuous and discrete input variables. However, in existing learning algorithms of both neural networks and fuzzy systems, these mixed variables are usually treated as continuous without taking into account the special features of discrete variables. It is inefficient to represent each discrete input variable having only a few fixed values by one input neuron with full connection to the hidden layer. This paper proposes a novel hierarchical hybrid fuzzy neural network to represent systems with mixed input variables. The proposed model consists of two levels: the lower level are fuzzy sub-systems each of which aggregates several discrete input variables into an intermediate variable as its output; the higher level is a neural network whose input variables consist of continuous input variables and intermediate variables. For systems or function approximations with mixed variables, it is shown that the proposed hierarchical hybrid fuzzy neural networks outperform standard neural networks in accuracy with fewer parameters, and both provide greater transparency and preserve the universal approximation property (i.e., they can approximate any function with mixed input variables to any degree of accuracy).     

Morphological perceptrons with competitive learning: Lattice-theoretical framework and constructive learning algorithm

A morphological neural network is generally defined as a type of artificial neural network that performs an elementary operation of mathematical morphology at every node, possibly followed by the application of an activation function. The underlying framework of mathematical morphology can be found in lattice theory.With the advent of granular computing, lattice-based neurocomputing models such as morphological neural networks and fuzzy lattice neurocomputing models are becoming increasingly important since many information granules such as fuzzy sets and their extensions, intervals, and rough sets are lattice ordered. In this paper, we present the lattice-theoretical background and the learning algorithms for morphological perceptrons with competitive learning which arise by incorporating a winner-take-all output layer into the original morphological perceptron model. Several well-known classification problems that are available on the internet are used to compare our new model with a range of classifiers such as conventional multi-layer perceptrons, fuzzy lattice neurocomputing models, k-nearest neighbors, and decision trees.     

A fuzzy ensemble of parallel polynomial neural networks with information granules formed by fuzzy clustering

In this paper, we introduce a new category of fuzzy models called a fuzzy ensemble of parallel polynomial neural network (FEP²N²), which consist of a series of polynomial neural networks weighted by activation levels of information granules formed with the use of fuzzy clustering. The two underlying design mechanisms of the proposed networks rely on information granules resulting from the use of fuzzy C-means clustering (FCM) and take advantage of polynomial neural networks (PNNs).The resulting model comes in the form of parallel polynomial neural networks. In the design procedure, in order to estimate the optimal values of the coefficients of polynomial neural networks we use a weighted least square estimation algorithm. We incorporate various types of structures as the consequent part of the fuzzy model when using the learning algorithm. Among the diverse structures being available, we consider polynomial neural networks, which exhibit highly nonlinear characteristics when being viewed as local learning models.We use FCM to form information granules and to overcome the high dimensionality problem. We adopt PNNs to find the optimal local models, which can describe the relationship between the input variables and output variable within some local region of the input space.We show that the generalization capabilities as well as the approximation abilities of the proposed model are improved as a result of using polynomial neural networks. The performance of the network is quantified through experimentation in which we use a number of benchmarks already exploited within the realm of fuzzy or neurofuzzy modeling.     

Fuzzy neural networks-based quality prediction system for sinteringprocess

A hybrid fuzzy neural networks and genetic algorithm (GA) system is proposed to solve the difficult and challenging problem of constructing a system model from the given input and output data to predict the quality of chemical components of the finished sintering mineral. A bidirectional fuzzy neural network (BFNN) is proposed to represent the fuzzy model and realize the fuzzy inference. The learning process of BFNN is divided into off-line and online learning. In off-line learning, the GA is used to train the BFNN and construct a system model based on the training data. During online operation, the algorithm inherited from the principle of backpropagation is used to adjust the network parameters and improve the system precision in each sampling period. The process of constructing a system model is introduced in details. The results obtained from the actual prediction demonstrate that the performance and capability of the proposed system are superior     

基于模糊神经网络的抽油机故障诊断方法

将模糊逻辑与神经网络相结合,构造模糊神经网络,将神经网络输入层的确定性信息模糊化后变成模糊量,将故障征兆参数相对应的隶属度数值作为神经网络的输入,从而使神经网络更加适合设备故障描述,克服了神经网络对不精确信息表达的缺点。提出基于黄金分割法的变步长BP算法来训练神经网络,根据误差变化趋势动态调整学习速率,实现学习步长的自适应调整,提高网络收敛的速度,防止网络训练时陷入局部极小。将训练好的模糊神经网络应用于抽油机设备的故障诊断,取得良好效果。     

Centroid based Multilayer Perceptron Networks

In this study we investigate a hybrid neural network architecture for modelling purposes. The proposed network is based on the multilayer perceptron (MLP) network. However, in addition to the usual hidden layers the first hidden layer is selected to be a centroid layer. Each unit in this new layer incorporates a centroid that is located somewhere in the input space. The output of these units is the Euclidean distance between the centroid and the input. The centroid layer clearly resembles the hidden layer of the radial basis function (RBF) networks. Therefore the centroid based multilayer perceptron (CMLP) networks can be regarded as a hybrid of MLP and RBF networks. The presented benchmark experiments show that the proposed hybrid architecture is able to combine the good properties of MLP and RBF networks resulting fast and efficient learning, and compact network structure.     

Divide-and-conquer learning and modular perceptron networks

A novel modular perceptron network (MPN) and divide-and-conquer learning (DCL) schemes for the design of modular neural networks are proposed. When a training process in a multilayer perceptron falls into a local minimum or stalls in a flat region, the proposed DCL scheme is applied to divide the current training data region into two easier to be learned regions. The learning process continues when a self-growing perceptron network and its initial weight estimation are constructed for one of the newly partitioned regions. Another partitioned region will resume the training process on the original perceptron network. Data region partitioning, weight estimating and learning are iteratively repeated until all the training data are completely learned by the MPN. We evaluated and compared the proposed MPN with several representative neural networks on the two-spirals problem and real-world dataset. The MPN achieved better weight learning performance by requiring much less data presentations during the network training phases, and better generalization performance, and less processing time during the retrieving phase.     

Limitations of multi-layer perceptron networks - steps towards genetic neural networks

In this paper we investigate multi-layer perceptron networks in the task domain of Boolean functions. We demystify the multi-layer perceptron network by showing that it just divides the input space into regions constrained by hyperplanes. We use this information to construct minimal training sets. Despite using minimal training sets, the learning time of multi-layer perceptron networks with backpropagation scales exponentially for complex Boolean functions. But modular neural networks which consist of independentky trained subnetworks scale very well. We conjecture that the next generation of neural networks will be genetic neural networks which evolve their structure. We confirm Minsky and Papert: “The future of neural networks is tied not to the search for some single, universal scheme to solve all problems at once, bu to the evolution of a many-faceted technology of network design.”     

Supervised Learning of Logical Operations in Layered Spiking Neural Networks with Spike Train Encoding

Few algorithms for supervised training of spiking neural networks exist that can deal with patterns of multiple spikes, and their computational properties are largely unexplored. We demonstrate in a set of simulations that the ReSuMe learning algorithm can successfully be applied to layered neural networks. Input and output patterns are encoded as spike trains of multiple precisely timed spikes, and the network learns to transform the input trains into target output trains. This is done by combining the ReSuMe learning algorithm with multiplicative scaling of the connections of downstream neurons. We show in particular that layered networks with one hidden layer can learn the basic logical operations, including Exclusive-Or, while networks without hidden layer cannot, mirroring an analogous result for layered networks of rate neurons. While supervised learning in spiking neural networks is not yet fit for technical purposes, exploring computational properties of spiking neural networks advances our understanding of how computations can be done with spike trains.