A block-diagonal recurrent fuzzy neural network for system identification

A recurrent fuzzy neural network with internal feedback is suggested in this paper. The network is entitled dynamic block-diagonal fuzzy neural network (DBD-FNN), and constitutes a generalized Takagi-Sugeno-Kang fuzzy system, where the consequent parts of the fuzzy rules are small Block-Diagonal Recurrent Neural Networks. The proposed model is applied to a benchmark identification problem, where a dynamic system is to be identified. Additionally, an application of the proposed model to the problem of the analysis of lung sounds is presented. Particularly, a filter based on the DBD-FNN is developed, trained with the RENNCOM method. Extensive experimental and simulation results are given and performance comparisons with a series of other models are conducted, highlighting the modeling characteristics of DBD-FNN as an identification tool and the effectiveness of the proposed separation filter.     

A novel compensation-based recurrent fuzzy neural network and its learning algorithm

Based on detailed study on several kinds of fuzzy neural networks, we propose a novel compensation-based recurrent fuzzy neural network (CRFNN) by adding recurrent element and compensatory element to the conventional fuzzy neural network. Then, we propose a sequential learning method for the structure identification of the CRFNN in order to confirm the fuzzy rules and their correlative parameters effectively. Furthermore, we improve the BP algorithm based on the characteristics of the proposed CRFNN to train the network. By modeling the typical nonlinear systems, we draw the conclusion that the proposed CRFNN has excellent dynamic response and strong learning ability. Supported by the National High-Tech Research and Development Program of China (Grant No. 2006AA05A107) and Special Fund of Jiangsu Province for Technology Transfer (Grant No. BA2007008)     

A TSK-type recurrent fuzzy network for dynamic systems processingby neural network and genetic algorithms

In this paper, a TSK-type recurrent fuzzy network (TRFN) structure is proposed. The proposal calls for the design of TRFN by either neural network or genetic algorithms depending on the learning environment. A recurrent fuzzy network is described which develops from a series of recurrent fuzzy if-then rules with TSK-type consequent parts. The recurrent property comes from feeding the internal variables, derived from fuzzy firing strengths, back to both the network input and output layers. In this configuration, each internal variable is responsible for memorizing the temporal history of its corresponding fuzzy rule. The internal variable is also combined with external input variables in each rule's consequence, which shows an increase in network learning ability. TRFN design under different learning environments is next advanced. For problems where supervised training data is directly available, TRFN with supervised learning (TRFN-S) is proposed, and a neural network (NN) learning approach is adopted for TRFN-S design. An online learning algorithm with concurrent structure and parameter learning is proposed. With flexibility of partition in the precondition part, and outcome of TSK-type, the TRFN-S displays both small network size and high learning accuracy. For problems where gradient information for NN learning is costly to obtain or unavailable, like reinforcement learning, TRFN with Genetic learning (TRFN-G) is put forward. The precondition parts of TRFN-G are also partitioned in a flexible way, and all free parameters are designed concurrently by genetic algorithm. Owing to the well-designed network structure of TRFN, TRFN-G, like TRFN-S, is characterized by high learning accuracy. To demonstrate the superior properties of TRFN, TRFN-S is applied to dynamic system identification and TRFN-G to dynamic system control. By comparing the results to other types of recurrent networks and design configurations, the efficiency of TRFN is verified     

路径规划; 态势评估; 模糊逻辑; 贝叶斯网络

针对非线性动态系统辨识和控制的特点，对4层模糊神经网络进行了优化和改进，形成了动态模糊神经网络，提高了网络的稳定性和对动态系统的辨识能力，同时给出了基于Lyapunov函数稳定收敛定理的各权向量以及权矩阵学习速率的自适应调整算法．应用于非线性动态系统的辨识和控制仿真试验表明，改进后的动态模糊神经网络与模糊神经网络相比，可取得更好的辨识精度和跟踪控制效果。     

Fuzzy identification using fuzzy neural networks with stable learning algorithms

In general, fuzzy neural networks cannot match nonlinear systems exactly. Unmodeled dynamic leads parameters drift and even instability problem. According to system identification theory, robust modification terms must be included in order to guarantee Lyapunov stability. This paper suggests new learning laws for Mamdani and Takagi-Sugeno-Kang type fuzzy neural networks based on input-to-state stability approach. The new learning schemes employ a time-varying learning rate that is determined from input-output data and model structure. Stable learning algorithms for the premise and the consequence parts of fuzzy rules are proposed. The calculation of the learning rate does not need any prior information such as estimation of the modeling error bounds. This offer an advantage compared to other techniques using robust modification.     

A fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks

In this paper, we present a fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks (FAOS-PFNN), where a novel structure learning algorithm incorporating a pruning strategy into new growth criteria is developed. The proposed growing procedure without pruning not only speeds up the online learning process but also facilitates a more parsimonious fuzzy neural network while achieving comparable performance and accuracy by virtue of the growing and pruning strategy. The FAOS-PFNN starts with no hidden neurons and parsimoniously generates new hidden units according to the proposed growth criteria as learning proceeds. In the parameter learning phase, all the free parameters of hidden units, regardless of whether they are newly created or originally existing, are updated by the extended Kalman filter (EKF) method. The effectiveness and superiority of the FAOS-PFNN paradigm is compared with other popular approaches like resource allocation network (RAN), RAN via the extended Kalman filter (RANEKF), minimal resource allocation network (MRAN), adaptive-network-based fuzzy inference system (ANFIS), orthogonal least squares (OLS), RBF-AFS, dynamic fuzzy neural networks (DFNN), generalized DFNN (GDFNN), generalized GAP-RBF (GGAP-RBF), online sequential extreme learning machine (OS-ELM) and self-organizing fuzzy neural network (SOFNN) on various benchmark problems in the areas of function approximation, nonlinear dynamic system identification, chaotic time-series prediction and real-world regression problems. Simulation results demonstrate that the proposed FAOS-PFNN algorithm can achieve faster learning speed and more compact network structure with comparably high accuracy of approximation and generalization.     

A recurrent self-organizing neural fuzzy inference network

A recurrent self-organizing neural fuzzy inference network (RSONFIN) is proposed. The RSONFIN is inherently a recurrent multilayered connectionist network for realizing the basic elements and functions of dynamic fuzzy inference, and may be considered to be constructed from a series of dynamic fuzzy rules. The temporal relations embedded in the network are built by adding some feedback connections representing the memory elements to a feedforward neural fuzzy network. Each weight as well as node in the RSONFIN has its own meaning and represents a special element in a fuzzy rule. There are no hidden nodes initially in the RSONFIN. They are created online via concurrent structure identification and parameter identification. The structure learning together with the parameter learning forms a fast learning algorithm for building a small, yet powerful, dynamic neural fuzzy network. Two major characteristics of the RSONFIN can thus be seen: 1) the recurrent property of the RSONFIN makes it suitable for dealing with temporal problems and 2) no predetermination, like the number of hidden nodes, must be given, since the RSONFIN can find its optimal structure and parameters automatically and quickly. Moreover, to reduce the number of fuzzy rules generated, a flexible input partition method, the aligned clustering-based algorithm, is proposed. Various simulations on temporal problems are done and performance comparisons with some existing recurrent networks are also made. Efficiency of the RSONFIN is verified from these results.     

Prediction and identification using wavelet-based recurrent fuzzy neural networks

This paper presents a wavelet-based recurrent fuzzy neural network (WRFNN) for prediction and identification of nonlinear dynamic systems. The proposed WRFNN model combines the traditional Takagi-Sugeno-Kang (TSK) fuzzy model and the wavelet neural networks (WNN). This paper adopts the nonorthogonal and compactly supported functions as wavelet neural network bases. Temporal relations embedded in the network are caused by adding some feedback connections representing the memory units into the second layer of the feedforward wavelet-based fuzzy neural networks (WFNN). An online learning algorithm, which consists of structure learning and parameter learning, is also presented. The structure learning depends on the degree measure to obtain the number of fuzzy rules and wavelet functions. Meanwhile, the parameter learning is based on the gradient descent method for adjusting the shape of the membership function and the connection weights of WNN. Finally, computer simulations have demonstrated that the proposed WRFNN model requires fewer adjustable parameters and obtains a smaller rms error than other methods.     

A Recurrent Self-Evolving Interval Type-2 Fuzzy Neural Network for Dynamic System Processing

This paper proposes a recurrent self-evolving interval type-2 fuzzy neural network (RSEIT2FNN) for dynamic system processing. An RSEIT2FNN incorporates type-2 fuzzy sets in a recurrent neural fuzzy system in order to increase the noise resistance of a system. The antecedent parts in each recurrent fuzzy rule in the RSEIT2FNN are interval type-2 fuzzy sets, and the consequent part is of the Takagi-Sugeno-Kang (TSK) type with interval weights. The antecedent part of RSEIT2FNN forms a local internal feedback loop by feeding the rule firing strength of each rule back to itself. The TSK-type consequent part is a linear model of exogenous inputs. The RSEIT2FNN initially contains no rules; all rules are learned online via structure and parameter learning. The structure learning uses online type-2 fuzzy clustering. For the parameter learning, the consequent part parameters are tuned by a rule-ordered Kalman filter algorithm to improve learning performance. The antecedent type-2 fuzzy sets and internal feedback loop weights are learned by a gradient descent algorithm. The RSEIT2FNN is applied to simulations of dynamic system identifications and chaotic signal prediction under both noise-free and noisy conditions. Comparisons with type-1 recurrent fuzzy neural networks validate the performance of the RSEIT2FNN.     

A novel prosodic-information synthesizer based on recurrent fuzzy neural network for the Chinese TTS system.

In this paper, a new technique for the Chinese text-to-speech (TTS) system is proposed. Our major effort focuses on the prosodic information generation. New methodologies for constructing fuzzy rules in a prosodic model simulating human's pronouncing rules are developed. The proposed Recurrent Fuzzy Neural Network (RFNN) is a multilayer recurrent neural network (RNN) which integrates a Self-cOnstructing Neural Fuzzy Inference Network (SONFIN) into a recurrent connectionist structure. The RFNN can be functionally divided into two parts. The first part adopts the SONFIN as a prosodic model to explore the relationship between high-level linguistic features and prosodic information based on fuzzy inference rules. As compared to conventional neural networks, the SONFIN can always construct itself with an economic network size in high learning speed. The second part employs a five-layer network to generate all prosodic parameters by directly using the prosodic fuzzy rules inferred from the first part as well as other important features of syllables. The TTS system combined with the proposed method can behave not only sandhi rules but also the other prosodic phenomena existing in the traditional TTS systems. Moreover, the proposed scheme can even find out some new rules about prosodic phrase structure. The performance of the proposed RFNN-based prosodic model is verified by imbedding it into a Chinese TTS system with a Chinese monosyllable database based on the time-domain pitch synchronous overlap add (TD-PSOLA) method. Our experimental results show that the proposed RFNN can generate proper prosodic parameters including pitch means, pitch shapes, maximum energy levels, syllable duration, and pause duration. Some synthetic sounds are online available for demonstration.     

Nonlinear Systems Identification via Two Types of Recurrent Fuzzy CMAC

Normal fuzzy CMAC neural network performs well for nonlinear systems identification because of its fast learning speed and local generalization capability for approximating nonlinear functions. However, it requires huge memory and the dimension increases exponentially with the number of inputs. It is difficult to model dynamic systems with static fuzzy CMACs. In this paper, we use two types of recurrent techniques for fuzzy CMAC to overcome the above problems. The new CMAC neural networks are named recurrent fuzzy CMAC (RFCMAC) which add feedback connections in the inner layers (local feedback) or the output layer (global feedback). The corresponding learning algorithms have time-varying learning rates, the stabilities of the neural identifications are proven.     

模糊竞争神经网络在线辨识及在CSTR中的应用

基于改进的T-S模糊模型构造了一种自适应模糊竞争神经网络模型(FCNN),给出了网络的连接结构和学习算法。它依据模糊竞争学习算法确定系统的模糊空间和模糊规则数,得出每个样本对每条规则的适用程度,并利用卡尔曼滤波算法在线辨识FCNN的后件参数。将其应用于化工过程连续搅拌反应器(CSTR)的建模中,仿真结果表明,FCNN具有结构简洁、收敛速度快、辨识精度高等特点,可当作复杂系统建模的一种有效手段。     

A genetic-based neuro-fuzzy approach for modeling and control ofdynamical systems

Linguistic modeling of complex irregular systems constitutes the heart of many control and decision making systems, and fuzzy logic represents one of the most effective algorithms to build such linguistic models. In this paper, a linguistic (qualitative) modeling approach is proposed. The approach combines the merits of the fuzzy logic theory, neural networks, and genetic algorithms (GAs). The proposed model is presented in a fuzzy-neural network (FNN) form which can handle both quantitative (numerical) and qualitative (linguistic) knowledge. The learning algorithm of a FNN is composed of three phases. The first phase is used to find the initial membership functions of the fuzzy model. In the second phase, a new algorithm is developed and used to extract the linguistic-fuzzy rules. In the third phase, a multiresolutional dynamic genetic algorithm (MRD-GA) is proposed and used for optimized tuning of membership functions of the proposed model. Two well-known benchmarks are used to evaluate the performance of the proposed modeling approach, and compare it with other modeling approaches.     

Fuzzy wavelet neural network based on fuzzy clustering and gradient techniques for time series prediction

This paper presents the development of fuzzy wavelet neural network system for time series prediction that combines the advantages of fuzzy systems and wavelet neural network. The structure of fuzzy wavelet neural network (FWNN) is proposed, and its learning algorithm is derived. The proposed network is constructed on the base of a set of TSK fuzzy rules that includes a wavelet function in the consequent part of each rule. A fuzzy c-means clustering algorithm is implemented to generate the rules, that is the structure of FWNN prediction model, automatically, and the gradient-learning algorithm is used for parameter identification. The use of fuzzy c-means clustering algorithm with the gradient algorithm allows to improve convergence of learning algorithm. FWNN is used for modeling and prediction of complex time series and prediction of foreign-exchange rates. Exchange rates are dynamic process that changes every day and have high-order nonlinearity. The statistical data for the last 2 years are used for the development of FWNN prediction model. Effectiveness of the proposed system is evaluated with the results obtained from the simulation of FWNN-based systems and with the comparative simulation results of previous related models.     

Adaptive Fuzzy Neural Networks as identifiers of discrete-time nonlinear dynamic systems

An adaptive supervised learning scheme is proposed in this paper for training Fuzzy Neural Networks (FNN) to identify discrete-time nonlinear dynamical systems. The FNN constructs are neural-network-based connectionist models consisting of several layers that are used to implement the functions of a fuzzy logic system. The fuzzy rule base considered here consists of Takagi-Sugeno IF-THEN rules, where the rule outputs are realized as linear polynomials of the input components. The FNN connectionist model is functionally partitioned into three separate parts, namely, the premise part, which provides the truth values of the rule preconditional statements, the consequent part providing the rule outputs, and the defuzzification part computing the final output of the FNN construct. The proposed learning scheme is a two-stage training algorithm that performs both structure and parameter learning, simultaneously. First, the structure learning task determines the proper fuzzy input partitions and the respective precondition matching, and is carried out by means of the rule base adaptation mechanism. The rule base adaptation mechanism is a self-organizing procedure which progressively generates the proper fuzzy rule base, during training, according to the operating conditions. Having completed the structure learning stage, the parameter learning is applied using the back-propagation algorithm, with the objective to adjust the premise/consequent parameters of the FNN so that the desired input/output representation is captured to an acceptable degree of accuracy. The structure/parameter training algorithm exhibits good learning and generalization capabilities as demonstrated via a series of simulation studies. Comparisons with conventional multilayer neural networks indicate the effectiveness of the proposed scheme.     

Dynamic non-Singleton fuzzy logic systems for nonlinear modeling

We investigate dynamic versions of fuzzy logic systems (FLSs) and, specifically, their non-Singleton generalizations (NSFLSs), and derive a dynamic learning algorithm to train the system parameters. The history-sensitive output of the dynamic systems gives them a significant advantage over static systems in modeling processes of unknown order. This is illustrated through an example in nonlinear dynamic system identification. Since dynamic NSFLS's can be considered to belong to the family of general nonlinear autoregressive moving average (NARMA) models, they are capable of parsimoniously modeling NARMA processes. We study the performance of both dynamic and static FLSs in the predictive modeling of a NARMA process     

一种基于模糊分类的模糊神经网络辨识方法

针对非线性辨识问题,基于改进的T-S模型,提出一种自适应模糊神经网络模型(AFNN)。首先,基于模糊竞争学习算法确定系统的模糊空间和模糊规则数,并得出每个样本对每条规则的适用程度。其次,利用卡尔曼滤波算法在线辨识AFNN的后件参数。AFNN具有结构简洁,逼近能力强,能够显著提高辨识精度,并且辨识的模糊模型简单有效。最后,将该AFNN用于非线性系统的模糊辨识,仿真结果验证了该方法的有效性。     

非线性系统的回归网络辨识

针对未知非线性系统的辨识问题,本文提出了一种新型的回归网络模型,证明了该网络模型在一定条件下能够逼近非线性系统的输入输出关系,提出了训练网络前向连接和反向连接权值的动态反向传播算法,伪真结果验证该方法的有效性。     

A Self-Evolving Interval Type-2 Fuzzy Neural Network With Online Structure and Parameter Learning

This paper proposes a self-evolving interval type-2 fuzzy neural network (SEIT2FNN) with online structure and parameter learning. The antecedent parts in each fuzzy rule of the SEIT2FNN are interval type-2 fuzzy sets and the fuzzy rules are of the Takagi–Sugeno–Kang (TSK) type. The initial rule base in the SEIT2FNN is empty, and the online clustering method is proposed to generate fuzzy rules that flexibly partition the input space. To avoid generating highly overlapping fuzzy sets in each input variable, an efficient fuzzy set reduction method is also proposed. This method independently determines whether a corresponding fuzzy set should be generated in each input variable when a new fuzzy rule is generated. For parameter learning, the consequent part parameters are tuned by the rule-ordered Kalman filter algorithm for high-accuracy learning performance. Detailed learning equations on applying the rule-ordered Kalman filter algorithm to the SEIT2FNN consequent part learning, with rules being generated online, are derived. The antecedent part parameters are learned by gradient descent algorithms. The SEIT2FNN is applied to simulations on nonlinear plant modeling, adaptive noise cancellation, and chaotic signal prediction. Comparisons with other type-1 and type-2 fuzzy systems in these examples verify the performance of the SEIT2FNN.      

A Wiener-type recurrent neural network and its control strategy for nonlinear dynamic applications

This paper presents a Wiener-type recurrent neural network with a systematic identification algorithm and a control strategy for the identification and control of unknown dynamic nonlinear systems. The proposed Wiener-type recurrent network resembles the conventional Wiener model that consists of a dynamic linear subsystem cascaded with a static nonlinear subsystem. The novelties of our network include: (1) the two subsystems are integrated into a single network whose output is expressed by a nonlinear transformation of a linear state-space equation; (2) the characteristics of the trained network can be analyzed by its associated state-space equation using the well-developed theory of linear systems; and (3) the size of the network structure is determined by the number of state variables (or the system order) of the unknown systems to be identified. To effectively identify a given unknown system from its input–output data, we have developed a systematic identification algorithm that consists of an order determination procedure, a parameterization procedure, and an online learning procedure. The false nearest neighbors algorithm was adopted to acquire a minimal embedding dimension from the input–output data as the system order, and then the eigensystem realization algorithm (ERA) was used to initialize a best-fit state-space representation according to the acquired system order. To improve the overall identification performance, we have derived an online parameter learning algorithm based on an ordered derivatives and momentum terms. Subsequently, a simple feedback linear controller was designed to control the unknown dynamic nonlinear systems without much complexity. Computer simulations and comparisons with some existing recurrent networks have conducted to confirm the effectiveness and superiority of the proposed Wiener-type network, identification algorithm and control strategy.