Function Approximation with Complex Neuro-Fuzzy System Using Complex Fuzzy Sets �C A New Approach

A new neuro-fuzzy computing paradigm using complex fuzzy sets is proposed in this paper. The novel computing paradigm is applied to the problem of function approximation to test its nonlinear mapping ability. A complex fuzzy set (CFS) is an extension of traditional type-1 fuzzy set whose membership is within the unit real-valued interval. For a CFS, the membership is extended to complex-valued state within the unit disc of the complex plane. For self-adaption of the proposed complex neuro-fuzzy system (CNFS), the Particle Swarm Optimization (PSO) algorithm and Recursive Least Squares Estimator (RLSE) algorithm are used in a hybrid way to adjust the free parameters of the CNFS. With the novel PSO-RLSE hybrid learning method, the CNFS parameters can be converged efficiently and quickly. By the PSO-RLSE method for the CNFS, fast learning convergence is observed and great performance in accuracy is shown. In the experimental results, the CNFS shows much better performance than its traditional neuro-fuzzy counterpart and other compared approaches. Excellent performance by the proposed approach has been shown.     

A fully complex-valued radial basis function classifier for real-valued classification problems

In this paper, we investigate the decision making ability of a fully complex-valued radial basis function (FC-RBF) network in solving real-valued classification problems. The FC-RBF classifier is a single hidden layer fully complex-valued neural network with a nonlinear input layer, a nonlinear hidden layer, and a linear output layer. The neurons in the input layer of the classifier employ the phase encoded transformation to map the input features from the Real domain to the Complex domain. The neurons in the hidden layer employ a fully complex-valued Gaussian-like activation function of the type of hyperbolic secant (sech). The classification ability of the classifier is first studied analytically and it is shown that the decision boundaries of the FC-RBF classifier are orthogonal to each other. Then, the performance of the FC-RBF classifier is studied experimentally using a set of real-valued benchmark problems and also a real-world problem. The study clearly indicates the superior classification ability of the FC-RBF classifier.     

Fast learning Circular Complex-valued Extreme Learning Machine (CC-ELM) for real-valued classification problems

In this paper, we present a fast learning fully complex-valued extreme learning machine classifier, referred to as ‘Circular Complex-valued Extreme Learning Machine (CC-ELM)’ for handling real-valued classification problems. CC-ELM is a single hidden layer network with non-linear input and hidden layers and a linear output layer. A circular transformation with a translational/rotational bias term that performs a one-to-one transformation of real-valued features to the complex plane is used as an activation function for the input neurons. The neurons in the hidden layer employ a fully complex-valued Gaussian-like (‘sech’) activation function. The input parameters of CC-ELM are chosen randomly and the output weights are computed analytically. This paper also presents an analytical proof to show that the decision boundaries of a single complex-valued neuron at the hidden and output layers of CC-ELM consist of two hyper-surfaces that intersect orthogonally. These orthogonal boundaries and the input circular transformation help CC-ELM to perform real-valued classification tasks efficiently.Performance of CC-ELM is evaluated using a set of benchmark real-valued classification problems from the University of California, Irvine machine learning repository. Finally, the performance of CC-ELM is compared with existing methods on two practical problems, viz., the acoustic emission signal classification problem and a mammogram classification problem. These study results show that CC-ELM performs better than other existing (both) real-valued and complex-valued classifiers, especially when the data sets are highly unbalanced.     

The Intermittent Control Synchronization of Complex-Valued Memristive Recurrent Neural Networks with Time-Delays

In this paper, the intermittent control synchronization of complex-valued memristive recurrent neural networks with time-delays is investigated. As a generalization on the real-valued memristive recurrent neural networks, complex-valued memristive recurrent neural networks own more complicated properties. In complex-valued domain, bounded and analytic complex-valued activation functions do not exist. Some assumptions about activation functions in real-valued domain cannot be applied directly to complex-valued fields. By appropriate transformation, complex-valued memristive recurrent neural networks can be divided into real parts and imaginary parts, which can avoid discussing the bounded and analytic. In the framework of differential inclusion theory and Lyapunov method, sufficient criteria of intermittent control synchronization are established. Finally, a simulation is given to verify the validity and feasibility of the sufficient conditions.

     

Boundedness and Convergence of Split-Complex Back-Propagation Algorithm with Momentum and Penalty

This paper investigates the split-complex back-propagation algorithm with momentum and penalty for training complex-valued neural networks. Here the momentum are used to accelerate the convergence of the algorithm and the penalty are used to control the magnitude of the network weights. The sufficient conditions for the learning rate, the momentum factor, the penalty coefficient, and the activation functions are proposed to establish the theoretical results of the algorithm. We theoretically prove the boundedness of the network weights during the training process, which is usually used as a precondition for convergence analysis in literatures. The monotonicity of the error function and the convergence of the algorithm are also guaranteed.     

Complex neutrosophic set

Complex fuzzy sets and complex intuitionistic fuzzy sets cannot handle imprecise, indeterminate, inconsistent, and incomplete information of periodic nature. To overcome this difficulty, we introduce complex neutrosophic set. A complex neutrosophic set is a neutrosophic set whose complex-valued truth membership function, complex-valued indeterminacy membership function, and complex-valued falsehood membership functions are the combination of real-valued truth amplitude term in association with phase term, real-valued indeterminate amplitude term with phase term, and real-valued false amplitude term with phase term, respectively. Complex neutrosophic set is an extension of the neutrosophic set. Further set theoretic operations such as complement, union, intersection, complex neutrosophic product, Cartesian product, distance measure, and δ-equalities of complex neutrosophic sets are studied here. A possible application of complex neutrosophic set is presented in this paper. Drawbacks and failure of the current methods are shown, and we also give a comparison of complex neutrosophic set to all such methods in this paper. We also showed in this paper the dominancy of complex neutrosophic set to all current methods through the graph.

     

Solving for time-varying and static cube roots in real and complex domains via discrete-time ZD models

Different from conventional gradient-based neural dynamics, a special type of neural dynamics has been proposed by Zhang et al. for online solution of time-varying and/or static (or termed, time-invariant) problems. The design of Zhang dynamics (ZD) is based on the elimination of an indefinite error function, instead of the elimination of a square-based positive (or at least lower-bounded) energy function usually associated with gradient dynamics (GD). In this paper, we generalize, propose and investigate the continuous-time ZD model and its discrete-time models in two situations (i.e., the time-derivative of the coefficient being known or unknown) for time-varying cube root finding, including the complex-valued continuous-time ZD model for finding cube roots in complex domain. In addition, to find the static scalar-valued cube root, a simplified continuous-time ZD model and its discrete-time model are generated. By focusing on such a static problem solving, Newton-Raphson iteration is found to be a special case of the discrete-time ZD model by utilizing the linear activation function and fixing the step-size value to be 1. Computer-simulation and testing results demonstrate the efficacy of the proposed ZD models (including real-valued ZD models and complex-valued ZD models) for time-varying and static cube root finding, as well as the link and new explanation to Newton-Raphson iteration.     

A class of complex ICA algorithms based on the kurtosis cost function.

In this paper, we introduce a novel way of performing real-valued optimization in the complex domain. This framework enables a direct complex optimization technique when the cost function satisfies the Brandwood's independent analyticity condition. In particular, this technique has been used to derive three algorithms, namely, kurtosis maximization using gradient update (KM-G), kurtosis maximization using fixed-point update (KM-F), and kurtosis maximization using Newton update (KM-N), to perform the complex independent component analysis (ICA) based on the maximization of the complex kurtosis cost function. The derivation and related analysis of the three algorithms are performed in the complex domain without using any complex-real mapping for differentiation and optimization. A general complex Newton rule is also derived for developing the KM-N algorithm. The real conjugate gradient algorithm is extended to the complex domain similar to the derivation of complex Newton rule. The simulation results indicate that the fixed-point version (KM-F) and gradient version (KM-G) are superior to other similar algorithms when the sources include both circular and noncircular distributions and the dimension is relatively high.     

A modified gradient-based neuro-fuzzy learning algorithm and its convergence

Neuro-fuzzy approach is known to provide an adaptive method to generate or tune fuzzy rules for fuzzy systems. In this paper, a modified gradient-based neuro-fuzzy learning algorithm is proposed for zero-order Takagi-Sugeno inference systems. This modified algorithm, compared with conventional gradient-based neuro-fuzzy learning algorithm, reduces the cost of calculating the gradient of the error function and improves the learning efficiency. Some weak and strong convergence results for this algorithm are proved, indicating that the gradient of the error function goes to zero and the fuzzy parameter sequence goes to a fixed value, respectively. A constant learning rate is used. Some conditions for the constant learning rate to guarantee the convergence are specified. Numerical examples are provided to support the theoretical findings.     

State Estimation of Quaternion-Valued Neural Networks with Leakage Time Delay and Mixed Two Additive Time-Varying Delays

In this paper, the state estimation of quaternion-valued neural networks (QVNNs) with leakage time delay, both discrete and distributed two additive time-varying delays is studied. By considering the QVNNs as a whole, instead of decomposing it into two complex-valued neural networks or four real-valued neural networks. Via constructing suitable Lyapunov–Krasovskii functionals, combining free weight matrix, reciprocally convex approach, and matrix inequalities, the sufficient criteria for time delays are given in the form of quaternion-valued linear matrix inequalities and complex-valued linear matrix inequalities. Some observable output measurements are used to estimate the state of neurons, which ensures the global asymptotic stability of the error-state system. Finally, the effectiveness of theoretical analysis is illustrated by a numerical simulation.

     

An IV-QR Algorithm for Neuro-Fuzzy Multivariable Online Identification

In this paper, a new algorithm for neuro-fuzzy identification of multivariable discrete-time nonlinear dynamic systems, more specifically applied to consequent parameters estimation of the neuro-fuzzy inference system, is proposed based on a decomposed form as a set of coupled multiple input and single output (MISO) Takagi-Sugeno (TS) neuro-fuzzy networks. An on-line scheme is formulated for modeling a nonlinear autoregressive with exogenous input (NARX) recurrent neuro-fuzzy structure from input-output samples of a multivariable nonlinear dynamic system in a noisy environment. The adaptive weighted instrumental variable (WIV) algorithm by QR factorization based on the numerically robust orthogonal Householder transformation is developed to modify the consequent parameters of the TS multivariable neuro-fuzzy network     

Speeding up the scaled conjugate gradient algorithm and its application in neuro-fuzzy classifier training

The aim of this study is to speed up the scaled conjugate gradient (SCG) algorithm by shortening the training time per iteration. The SCG algorithm, which is a supervised learning algorithm for network-based methods, is generally used to solve large-scale problems. It is well known that SCG computes the second-order information from the two first-order gradients of the parameters by using all the training datasets. In this case, the computation cost of the SCG algorithm per iteration is more expensive for large-scale problems. In this study, one of the first-order gradients is estimated from the previously calculated gradients without using the training dataset. To estimate this gradient, a least square error estimator is applied. The estimation complexity of the gradient is much smaller than the computation complexity of the gradient for large-scale problems, because the gradient estimation is independent of the size of dataset. The proposed algorithm is applied to the neuro-fuzzy classifier and the neural network training. The theoretical basis for the algorithm is provided, and its performance is illustrated by its application to several examples in which it is compared with several training algorithms and well-known datasets. The empirical results indicate that the proposed algorithm is quicker per iteration time than the SCG. The algorithm decreases the training time by 20–50% compared to SCG; moreover, the convergence rate of the proposed algorithm is similar to SCG.     

A Data-Reusing Nonlinear Gradient Descent Algorithm for a Class of Complex-Valued Neural Adaptive Filters

A complex-valued data-reusing nonlinear gradient descent (CDRNGD) learning algorithm for a class of complex-valued nonlinear neural adaptive filters is introduced and the affinity between the family of data-reusing algorithms and the class of normalised gradient descent algorithms is examined. Error bounds on the class of complex data-reusing algorithms are established and indicate the stability of such algorithms. Experiments on nonlinear inputs show the class of complex data-reusing algorithms outperforming the standard complex nonlinear gradient descent algorithms and converging to the normalised complex non-linear gradient descent algorithm without experiencing the stability problems commonly encountered with normalised gradient descent algorithms. This revised version was published online in June 2006 with corrections to the Cover Date.     

Properties of a general quaternion-valued gradient operator and its applications to signal processing

The gradients of a quaternion-valued function are often required for quaternionic signal processing algorithms. The HR gradient operator provides a viable framework and has found a number of applications. However, the applications so far have been limited to mainly real-valued quaternion functions and linear quaternionvalued functions. To generalize the operator to nonlinear quaternion functions, we define a restricted version of the HR operator, which comes in two versions, the left and the right ones. We then present a detailed analysis of the properties of the operators, including several different product rules and chain rules. Using the new rules, we derive explicit expressions for the derivatives of a class of regular nonlinear quaternion-valued functions, and prove that the restricted HR gradients are consistent with the gradients in the real domain. As an application, the derivation of the least mean square algorithm and a nonlinear adaptive algorithm is provided. Simulation results based on vector sensor arrays are presented as an example to demonstrate the effectiveness of the quaternion-valued signal model and the derived signal processing algorithm.     

Synchronisation control for a class of complex-valued fractional-order memristor-based delayed neural networks

In this paper, the problem of drive-response synchronisation of complex-valued fractional-order memristor-based delayed neural networks is discussed via linear feedback control method. By separating complex-valued system into two equivalent real-valued systems, and using the comparison theorem, algebraic criteria are given to ascertain the synchronisation of the considered system with single delay. Meanwhile, for the case of model with multiple delays, the corresponding sufficient conditions are also presented. Because complex-valued system can reduce to real-valued ones when the imaginary part is ignored, the proposed results of this paper generalise existing works on relevant real-valued system. Finally, the effectiveness of the obtained theoretical results is testified by two numerical examples.     

Improved Complex-valued Radial Basis Function (ICRBF) neural networks on multiple crack identification

This paper introduces a new class of neural networks in complex space called Complex-valued Radial Basis Function (CRBF) neural networks and also an improved version of CRBF called Improved Complex-valued Radial Basis Function (ICRBF) neural networks. They are used for multiple crack identification in a cantilever beam in the frequency domain. The novelty of the paper is that, these complex-valued neural networks are first applied on inverse problems (damage identification) which come under the category of function approximation. The conventional CRBF network was used in the first stage of ICRBF network and in the second stage a reduced search space moving technique was employed for accurate crack identification. The effectiveness of proposed ICRBF neural network was studied first on a single crack identification problem and then applied to a more challenging problem of multiple crack identification in a cantilever beam with zero noise as well as 5% noise polluted signals. The results proved that, the proposed ICRBF and real-valued Improved RBF (IRBF) neural networks have identified the single and multiple cracks with less than 1% absolute mean percentage error as compared to conventional CRBF and RBF neural networks, mainly because of their second stage reduced search space moving technique. It appears that IRBF neural network is a good compromise considering all factors like accuracy, simplicity and computational effort.     

A novel diagnosis system for Parkinson’s disease using complex-valued artificial neural network with k-means clustering feature weighting method

Parkinson’s disease (PD) is a degenerative, central nervous system disorder. The diagnosis of PD is difficult, as there is no standard diagnostic test and a particular system that gives accurate results. Therefore, automated diagnostic systems are required to assist the neurologist. In this study, we have developed a new hybrid diagnostic system for addressing the PD diagnosis problem. The main novelty of this paper lies in the proposed approach that involves a combination of the k-means clustering-based feature weighting (KMCFW) method and a complex-valued artificial neural network (CVANN). A Parkinson dataset comprising the features obtained from speech and sound samples were used for the diagnosis of PD. PD attributes are weighted through the use of the KMCFW method. New features obtained are converted into a complex number format. These feature values are presented as an input to the CVANN. The efficiency and effectiveness of the proposed system have been rigorously evaluated against the PD dataset in terms of five different evaluation methods. Experimental results have demonstrated that the proposed hybrid system, entitled KMCFW–CVANN, significantly outperforms the other methods detailed in the literature and achieves the highest classification results reported so far, with a classification accuracy of 99.52 %. Therefore, the proposed system appears to be promising in terms of a more accurate diagnosis of PD. Also, the application confirms the conclusion that the reliability of the classification ability of a complex-valued algorithm with regard to a real-valued dataset is high.

     

Derivation of a parameter stabilizing training criterion for adaptive neuro-fuzzy inference systems in motion control

This paper presents a novel training algorithm for adaptive neuro-fuzzy inference systems. The algorithm combines the error back-propagation algorithm with the variable structure systems approach. Expressing the parameter update rule as a dynamic system in continuous time and applying sliding mode control (SMC) methodology to the dynamic model of the gradient based training procedure results in the parameter stabilizing part of training algorithm. The proposed combination therefore exhibits a degree of robustness to the unmodelled multivariable internal dynamics of the gradient-based training algorithm. With conventional training procedures, the excitation of this dynamics during a training cycle can lead to instability, which may be difficult to alleviate owing to the multidimensionality of the solution space and the ambiguities concerning the environmental conditions. This paper shows that a neuro-fuzzy model can be trained such that the adjustable parameter values are forced to settle down (parameter stabilization) while minimizing an appropriate cost function (cost optimization), which is based on state tracking performance. In the application example, trajectory control of a two degrees of freedom direct drive robotic manipulator is considered. As the controller, an adaptive neuro-fuzzy inference mechanism is used and, in the parameter tuning, the proposed algorithm is utilized.     

A novel signal diagnosis technique using pseudo complex-valued autoregressive technique

In this paper, a new method of biomedical signal classification using complex- valued pseudo autoregressive (CAR) modeling approach has been proposed. The CAR coefficients were computed from the synaptic weights and coefficients of a split weight and activation function of a feedforward multilayer complex valued neural network. The performance of the proposed technique has been evaluated using PIMA Indian diabetes dataset with different complex-valued data normalization techniques and four different values of learning rate. An accuracy value of 81.28% has been obtained using this proposed technique.     

Convolutive Blind Source Separation in the Frequency Domain Based on Sparse Representation

Convolutive blind source separation (CBSS) that exploits the sparsity of source signals in the frequency domain is addressed in this paper. We assume the sources follow complex Laplacian-like distribution for complex random variable, in which the real part and imaginary part of complex-valued source signals are not necessarily independent. Based on the maximum a posteriori (MAP) criterion, we propose a novel natural gradient method for complex sparse representation. Moreover, a new CBSS method is further developed based on complex sparse representation. The developed CBSS algorithm works in the frequency domain. Here, we assume that the source signals are sufficiently sparse in the frequency domain. If the sources are sufficiently sparse in the frequency domain and the filter length of mixing channels is relatively small and can be estimated, we can even achieve underdetermined CBSS. We illustrate the validity and performance of the proposed learning algorithm by several simulation examples.