Adaptive control of a CSTR with a neural network model

An adaptive control algorithm with a neural network model, previously proposed in the literature for the control of mechanical manipulators, is applied to a CSTR (Continuous Stirred Tank Reactor). The neural network model uses either radial Gaussian or “Mexican hat” wavelets as basis functions. This work shows that the addition of linear functions to the networks significantly improves the error convergence when the CSTR is operated for long periods of time in a neighborhood of one operating point, a common scenario in chemical process control. Then, a quantitative comparative study based on output errors and control efforts is conducted where adaptive controllers using wavelets or Gaussian basis functions and PID controllers (IMC tuning with fixed parameters and self tuning PID) are compared. From this comparative study, the practicality and advantages of the adaptive controllers over fixed or adaptive PID control is assessed.     

Improvements in the predictive capability of neural networks

Neural networks can be used to develop effective models of nonlinear systems. Their main advantage being that they can model the vast majority of nonlinear systems to any arbitrary degree of accuracy. The ability of a neural network to predict the behavior of a nonlinear system accurately ought to be improved if there was some mechanism that allows the incorporation of first-principles model information into their training. This study proposes to use information obtained from a first-principle model to impart a sense of “direction” to the neural network model estimate. This is accomplished by modifying the objective function so as to include an additional term that is the difference between the time derivative of the outputs, as predicted by the neural network, and that of the outputs of the first-principles model during the training phase. The performance of a feedforward neural network model that uses this modified objective function is demonstrated on a chaotic process and compared to the conventional feedforward network trained on the usual objective function.     

Investigation of engine fault diagnosis using discrete wavelet transform and neural network

An investigation of a fault diagnostic technique for internal combustion engines using discrete wavelet transform (DWT) and neural network is presented in this paper. Generally, sound emission signal serves as a promising alternative to the condition monitoring and fault diagnosis in rotating machinery when the vibration signal is not available. Most of the conventional fault diagnosis techniques using sound emission and vibration signals are based on analyzing the signal amplitude in the time or frequency domain. Meanwhile, the continuous wavelet transform (CWT) technique was developed for obtaining both time-domain and frequency-domain information. Unfortunately, the CWT technique is often operated over a longer computing time. In the present study, a DWT technique which is combined with a feature selection of energy spectrum and fault classification using neural network for analyzing fault signal is proposed for improving the shortcomings without losing its original property. The features of the sound emission signal at different resolution levels are extracted by multi-resolution analysis and Parseval’s theorem [Gaing, Z. L. (2004). Wavelet-based neural network for power disturbance recognition and classification. IEEE Transactions on Power Delivery 19, 1560–1568]. The algorithm is obtained from previous work by Daubechies [Daubechies, I. (1988). Orthonormal bases of compactly supported wavelets. Communication on Pure and Applied Mathematics 41, 909–996.], the“db4”, “db8” and “db20” wavelet functions are adopted to perform the proposed DWT technique. Then, these features are used for fault recognition using a neural network. The experimental results indicated that the proposed system using the sound emission signal is effective and can be used for fault diagnosis of various engine operating conditions.     

Nonlinear black-box modeling in system identification: a unified overview

A nonlinear black-box structure for a dynamical system is a model structure that is prepared to describe virtually any nonlinear dynamics. There has been considerable recent interest in this area, with structures based on neural networks, radial basis networks, wavelet networks and hinging hyperplanes, as well as wavelet-transform-based methods and models based on fuzzy sets and fuzzy rules. This paper describes all these approaches in a common framework, from a user's perspective. It focuses on what are the common features in the different approaches, the choices that have to be made and what considerations are relevant for a successful system-identification application of these techniques. It is pointed out that the nonlinear structures can be seen as a concatenation of a mapping form observed data to a regression vector and a nonlinear mapping from the regressor space to the output space. These mappings are discussed separately. The latter mapping is usually formed as a basis function expansion. The basis functions are typically formed from one simple scalar function, which is modified in terms of scale and location. The expansion from the scalar argument to the regressor space is achieved by a radial- or a ridge-type approach. Basic techniques for estimating the parameters in the structures are criterion minimization, as well as two-step procedures, where first the relevant basis functions are determined, using data, and then a linear least-squares step to determine the coordinates of the function approximation. A particular problem is to deal with the large number of potentially necessary parameters. This is handled by making the number of ‘used’ parameters considerably less than the number of ‘offered’ parameters, by regularization, shrinking, pruning or regressor selection.     

Convex incremental extreme learning machine

Unlike the conventional neural network theories and implementations, Huang et al. [Universal approximation using incremental constructive feedforward networks with random hidden nodes, IEEE Transactions on Neural Networks 17(4) (2006) 879–892] have recently proposed a new theory to show that single-hidden-layer feedforward networks (SLFNs) with randomly generated additive or radial basis function (RBF) hidden nodes (according to any continuous sampling distribution) can work as universal approximators and the resulting incremental extreme learning machine (I-ELM) outperforms many popular learning algorithms. I-ELM randomly generates the hidden nodes and analytically calculates the output weights of SLFNs, however, I-ELM does not recalculate the output weights of all the existing nodes when a new node is added. This paper shows that while retaining the same simplicity, the convergence rate of I-ELM can be further improved by recalculating the output weights of the existing nodes based on a convex optimization method when a new hidden node is randomly added. Furthermore, we show that given a type of piecewise continuous computational hidden nodes (possibly not neural alike nodes), if SLFNs can work as universal approximators with adjustable hidden node parameters, from a function approximation point of view the hidden node parameters of such “generalized” SLFNs (including sigmoid networks, RBF networks, trigonometric networks, threshold networks, fuzzy inference systems, fully complex neural networks, high-order networks, ridge polynomial networks, wavelet networks, etc.) can actually be randomly generated according to any continuous sampling distribution. In theory, the parameters of these SLFNs can be analytically determined by ELM instead of being tuned.     

Effectiveness of neural network types for prediction of business failure

The study examines the effectiveness of different neural networks in predicting bankruptcy filing. Two approaches for training neural networks, Back-Propagation and Optimal Estimation Theory, are considered. Within the back-propagation training method, four different models (Back-Propagation, Functional Link Back-Propagation With Sines, Pruned Back-Propagation, and Cumulative Predictive Back-Propagation) are tested. The neural networks are compared against traditional bankruptcy prediction techniques such as discriminant analysis, logit, and probit. The results show that the level of Type I and Type II errors varies greatly across techniques. The Optimal Estimation Theory neural network has the lowest level of Type I error and the highest level of Type II error while the traditional statistical techniques have the reverse relationship (i.e., high Type I error and low Type II error). The back-propagation neural networks have intermediate levels of Type I and Type II error. We demonstrate that the performance of the neural networks tested is sensitive to the choice of variables selected and that the networks cannot be relied upon to “sift through” variables and focus on the most important variables (network performance based on the combined set of Ohlson and Altman data was frequently worse than their performance with one of the subsets). It is also important to note that the results are quite sensitive to sampling error. The significant variations across replications for some of the models indicate the sensitivity of the models to variations in the data.     

A novel hybrid classification model of artificial neural networks and multiple linear regression models

The classification problem of assigning several observations into different disjoint groups plays an important role in business decision making and many other areas. Developing more accurate and widely applicable classification models has significant implications in these areas. It is the reason that despite of the numerous classification models available, the research for improving the effectiveness of these models has never stopped. Combining several models or using hybrid models has become a common practice in order to overcome the deficiencies of single models and can be an effective way of improving upon their predictive performance, especially when the models in combination are quite different. In this paper, a novel hybridization of artificial neural networks (ANNs) is proposed using multiple linear regression models in order to yield more general and more accurate model than traditional artificial neural networks for solving classification problems. Empirical results indicate that the proposed hybrid model exhibits effectively improved classification accuracy in comparison with traditional artificial neural networks and also some other classification models such as linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), K-nearest neighbor (KNN), and support vector machines (SVMs) using benchmark and real-world application data sets. These data sets vary in the number of classes (two versus multiple) and the source of the data (synthetic versus real-world). Therefore, it can be applied as an appropriate alternate approach for solving classification problems, specifically when higher forecasting accuracy is needed.     

Fault diagnostic of induction motors for equipment reliability and health maintenance based upon Fourier and wavelet analysis

The motor is the workhorse of industry. The issues of preventive and condition-based maintenance, on-line monitoring, system fault detection, diagnosis, and prognosis are of increasing importance. This paper introduces fault detection for induction motors. Stator currents are measured by current meters and stored by time domain. The time domain is not suitable for representing current signals, so the frequency domain is applied to display signals. The Fourier transform is employed to convert signals. After signal conversion, signal features must be extracted by signal processing such as wavelet and spectrum analysis. Features are entered in a pattern classification model such as a neural network model, a polynomial neural network, or a fuzzy inference model. This paper describes fault detection results that use Fourier and wavelet analysis. This combined approach is very useful and powerful for detection signal features.This work was presented in part at the 9th International Symposium on Artificial Life and Robotics, Oita, Japan, January 28–30, 2004This work has been supported by “Research Center for Future Logistics Information Technology” hosted by the Ministry of Education in Korea.     

Linear electrostatic actuators: Gap maintenance via fluid bearings

This paper addresses a novel actuator for manufacturing applications, the “electrostatic artificial muscle.” Artificial muscle is composed of a dense array of small linear actuators. Its promise lies in the prospect of high performance (e.g. higher force-to-weight ratio and peak acceleration than a comparable magnetic motor), clean, quiet operation, and design versatility (especially the elimination of transmissions in many applications). The characteristics of artificial muscle are particularly appealing for applications in robotics and high-speed automation.A model of a linear electrostatic induction motor is presented to illustrate the potential for high performance as well as the difficulty of “gap maintenance.” Gap maintenance refers to the demanding task of preserving a uniform, narrow gap between “slider” and stator in the presence of destabilizing electrostatic forces. A novel approach to gap maintenance, the use of dielectric fluid bearings, is presented. Analysis of a simple, 2-D motor model shows that gap maintenance and motor efficiency may be characterized by two nondimensional parameters: a levitation number, and a gap aspect ratio. It is shown that achieving both low-speed levitation and high efficiency requires long, narrow gaps (high aspect ratio). The results of this analysis are extended to a more complex model featuring an unconstrained, rigid slider. An experimental study of fluid bearings is also presented.     

Wavelet networks for nonlinear system modeling

This study presents a nonlinear systems and function learning by using wavelet network. Wavelet networks are as neural network for training and structural approach. But, training algorithms of wavelet networks is required a smaller number of iterations when the compared with neural networks. Gaussian-based mother wavelet function is used as an activation function. Wavelet networks have three main parameters; dilation, translation, and connection parameters (weights). Initial values of these parameters are randomly selected. They are optimized during training (learning) phase. Because of random selection of all initial values, it may not be suitable for process modeling. Because wavelet functions are rapidly vanishing functions. For this reason heuristic procedure has been used. In this study serial-parallel identification model has been applied to system modeling. This structure does not utilize feedback. Real system outputs have been exercised for prediction of the future system outputs. So that stability and approximation of the network is guaranteed. Gradient methods have been applied for parameters updating with momentum term. Quadratic cost function is used for error minimization. Three example problems have been examined in the simulation. They are static nonlinear functions and discrete dynamic nonlinear system.     

Dynamic filter weights neural network model integrated with differential evolution for day-ahead price forecasting in energy market

In this paper a new dynamic model for forecasting electricity prices from 1 to 24 h in advance is proposed. The model is a dynamic filter weight Adaline using a sliding mode weight adaptation technique. The filter weights for this neuron constitute of first order dynamic filter with adjustable parameters. Sliding mode invariance conditions determine a least square characterization of the adaptive weights average dynamics whose stability features may be studied using standard time varying linear system results. The approach is found to exhibit robustness characteristics and first convergence properties. Comparison of results with a local linear wavelet neural network model is also presented in this paper. The hourly electricity prices of California and Spanish energy markets are taken as experimental data and the Mean Absolute Percentage Error (MAPE) and Root Mean Square Error (RMSE) are computed to find out the forecasting performance of both the models. In both the cases the MAPE and RMSE are found to be within the tolerable limits. As dynamic filter weight neural network gives better results in comparison to local linear wavelet neural network, the former has been further integrated with differential evolution algorithm to enhance the performance.     

Protean graphs with a variety of ranking schemes

We introduce a new class of random graph models for complex real-world networks, based on the protean graph model by Łuczak and Prałat. Our generalized protean graph models have two distinguishing features. First, they are not growth models, but instead are based on the assumption that a “steady state” of large but finite size has been reached. Second, the models assume that the vertices are ranked according to a given ranking scheme, and the rank of a vertex determines the probability that that vertex receives a link in a given time step. Precisely, the link probability is proportional to the rank raised to the power −α, where the attachment strength α is a tunable parameter. We show that the model leads to a power law degree distribution with exponent 1+1/α for ranking schemes based on a given prestige label, or on the degree of a vertex. We also study a scheme where each vertex receives an initial rank chosen randomly according to a biased distribution. In this case, the degree distribution depends on the distribution of the initial rank. For one particular choice of parameters we obtain a power law with an exponent that depends both on α and on a parameter determining the initial rank distribution.     

Comparative analysis of different approaches to target differentiation and localization with sonar

This study compares the performances of different methods for the differentiation and localization of commonly encountered features in indoor environments. Differentiation of such features is of interest for intelligent systems in a variety of applications such as system control based on acoustic signal detection and identification, map building, navigation, obstacle avoidance, and target tracking. Different representations of amplitude and time-of-flight measurement patterns experimentally acquired from a real sonar system are processed. The approaches compared in this study include the target differentiation algorithm, Dempster-Shafer evidential reasoning, different kinds of voting schemes, statistical pattern recognition techniques (k-nearest neighbor classifier, kernel estimator, parameterized density estimator, linear discriminant analysis, and fuzzy c-means clustering algorithm), and artificial neural networks. The neural networks are trained with different input signal representations obtained using pre-processing techniques such as discrete ordinary and fractional Fourier, Hartley and wavelet transforms, and Kohonen's self-organizing feature map. The use of neural networks trained with the back-propagation algorithm, usually with fractional Fourier transform or wavelet pre-processing results in near perfect differentiation, around 85% correct range estimation and around 95% correct azimuth estimation, which would be satisfactory in a wide range of applications.     

Perceptron training algorithms designed using discrete-time control Liapunov functions

Perceptrons, proposed in the seminal paper McCulloch–Pitts of 1943, have remained of interest to neural network community because of their simplicity and usefulness in classifying linearly separable data and can be viewed as implementing iterative procedures for “solving” linear inequalities. Gradient descent and conjugate gradient methods, normally used for linear equalities, can be used to solve linear inequalities by simple modifications that have been proposed in the literature but not been analyzed completely. This paper applies a recently proposed control-inspired approach to the design of iterative steepest descent and conjugate gradient algorithms for perceptron training in batch mode, by regarding certain parameters of the training/algorithm as controls and then using a control Liapunov technique to choose appropriate values of these parameters.     

State-dependent parameter modelling and identification of stochastic non-linear sampled-data systems

State-dependent parameter representations of stochastic non-linear sampled-data systems are studied. Velocity-based linearization is used to construct state-dependent parameter models which have a nominally linear structure but whose parameters can be characterized as functions of past outputs and inputs. For stochastic systems state-dependent parameter ARMAX (quasi-ARMAX) representations are obtained. The models are identified from input–output data using feedforward neural networks to represent the model parameters as functions of past inputs and outputs. Simulated examples are presented to illustrate the usefulness of the proposed approach for the modelling and identification of non-linear stochastic sampled-data systems.     

Modeling of chromatographic separation process with Wiener-MLP representation

Dynamic input–output-models have been identified for columns of an industrial sequential ion-exclusive chromatographic separation unit. Models are aimed at describing motion and form transformation of the fronts of different substances in the columns so that changes in “limit cycles” dynamics and drifts to undesired disturbed states could be observed on-line with model based simulations. The model structure has been innovated on the basis of classical Wiener representation, in which nonlinear dynamic system is described with a combination of linear Laguerre dynamics and static nonlinear mapping. The static mapping is realized here with MLP-type neural network. A separate delay model is needed for describing the movement of the front. The delay time adapts on variations of the process flow rate. Form transformation of the front is described with a dispersion model, which is smoother type Wiener-MLP model. Forward and backward Laguerre presentations are calculated with Laguerre filters. These Laguerre presentations are mapped to the output with a neural network. Dynamics of “salt” and two important compounds have been modeled on the basis of analyzed samples, which were taken in a factory experiment during normal production. A priori information about the process dynamics can be included in the dispersion model by choosing a suitable Laguerre parameter, but otherwise representativeness of the identification data determines validity of the model.     

Geographical classification of Spanish bottled mineral waters by means of iterative models based on linear discriminant analysis and artificial neural networks

The composition of Spanish natural mineral waters has been determined by means of inductively coupled plasma-mass spectrometry, inductively coupled plasma-atomic emission spectrometry, ionic chromatography and other routine techniques. Methods were applied to samples of bottled water from springs situated in five different mountain systems such as Cordillera Costero-Catalana, Macizo Galaico, Sistemas Béticos, Sistema Central and Sistema Ibérico. Pattern recognition techniques have been applied to differentiate the origin of samples. Data were initially studied by using nonparametric multiple comparison techniques and principal component analysis to highlight data trends. Classification models based on linear discriminant analysis and multilayer perceptron artificial neural networks have been built and validated by means of a stratified jackknifing methodology. An iterative approach has been used to build an artificial neural network model based on the variables selected by linear discriminant analysis. The prediction ability of the constructed model was 94 %.

     

Sensor fusion

Sensor fusion is a method of integrating signals from multiple sources. It allows extracting information from several different sources to integrate them into single signal or information. In many cases sources of information are sensors or other devices that allow for perception or measurement of changing environment. Information received from multiple-sensors is processed using “sensor fusion” or “data fusion” algorithms. These algorithms can be classified into three different groups. First, fusion based on probabilistic models, second, fusion based on least-squares techniques and third, intelligent fusion. The probabilistic model methods are Bayesian reasoning, evidence theory, robust statistics, recursive operators. The least-squares techniques are Kalman filtering, optimal theory, regularization and uncertainty ellipsoids. The intelligent fusion methods are fuzzy logic, neural networks and genetic algorithms. This paper will present three different methods of intelligent information fusion for different engineering applications. Chapter 2 is based on Sasiadek and Wang (2001) paper and presents an application of adaptive Kalman filtering to the problem of information fusion for guidance, navigation, and control. Chapter 3 is based on Sasiadek and Hartana (2000) and Chapter 4 on Sasiadek and Khe (2001) paper.     

Neighborhood rough set and SVM based hybrid credit scoring classifier

The credit scoring model development has become a very important issue, as the credit industry is highly competitive. Therefore, considerable credit scoring models have been widely studied in the areas of statistics to improve the accuracy of credit scoring during the past few years. This study constructs a hybrid SVM-based credit scoring models to evaluate the applicant’s credit score according to the applicant’s input features: (1) using neighborhood rough set to select input features; (2) using grid search to optimize RBF kernel parameters; (3) using the hybrid optimal input features and model parameters to solve the credit scoring problem with 10-fold cross validation; (4) comparing the accuracy of the proposed method with other methods. Experiment results demonstrate that the neighborhood rough set and SVM based hybrid classifier has the best credit scoring capability compared with other hybrid classifiers. It also outperforms linear discriminant analysis, logistic regression and neural networks.     

Identification of structural systems by neural networks

A method based on the use of neural networks is developed for the identification of systems encountered in the field of structural dynamics. The methodology is applied to the identification of linear and nonlinear dynamic systems such as the damped Duffing oscillator and the Van der Pol equation. The “generalization” ability of the neural networks is used to predict the response of the identified systems under deterministic and stochastic excitations. It is shown that neural networks provide high fidelity models of unknown structural dynamic systems, which are used in applications such as structural control, health monitoring of structures, earthquake engineering, etc.