A comparison of genetic programming and artificial neural networks in metamodeling of discrete-event simulation models

Genetic programming (GP) and artificial neural networks (ANNs) can be used in the development of surrogate models of complex systems. The purpose of this paper is to provide a comparative analysis of GP and ANNs for metamodeling of discrete-event simulation (DES) models. Three stochastic industrial systems are empirically studied: an automated material handling system (AMHS) in semiconductor manufacturing, an (s,S) inventory model and a serial production line. The results of the study show that GP provides greater accuracy in validation tests, demonstrating a better generalization capability than ANN. However, GP when compared to ANN requires more computation in metamodel development. Even given this increased computational requirement, the results presented indicate that GP is very competitive in metamodeling of DES models.     

Artificial neural network models for the prediction of surface roughness in electrical discharge machining

In the present paper Artificial Neural Networks (ANNs) models are proposed for the prediction of surface roughness in Electrical Discharge Machining (EDM). For this purpose two well-known programs, namely Matlab^® with associated toolboxes, as well as Netlab^®, were emplo- yed. Training of the models was performed with data from an extensive series of EDM experiments on steel grades; the proposed models use the pulse current, the pulse duration, and the processed material as input parameters. The reported results indicate that the proposed ANNs models can satisfactorily predict the surface roughness in EDM. Moreover, they can be considered as valuable tools for the process planning for EDMachining.     

Verification and validation of RANS maneuvering simulation of Esso Osaka: effects of drift and rudder angle on forces and moments

The present work covers rigorous verification and validation of a Reynolds averaged Navier–Stokes (RANS) code applied to a maneuvering problem covering the “static rudder” and “pure drift” conditions. The objectives are: (1) to apply the RANS technology together with the Chimera grid technique to compute the hydrodynamic forces acting on the bare hull and the appended hull of the tanker Esso Osaka during simple maneuvers; (2) to provide detailed information about the levels of verification and validation for the integral quantities; (3) to develop a procedure for generation of the systematically refined Chimera grids, which are used for the verification; and (4) to provide information about the trends in the forces and moments when the rudder and drift angles are varied. The flow problem is solved by the general-purpose RANS code CFDSHIP-IOWA, which is run in steady mode. The effect of the free surface is neglected and the two-equation k–ω model, models the turbulence. The verification and validation are performed by means of one of the latest approaches. It takes both the numerical and experimental uncertainties and errors into account, when the method is validated. The verification and validation of the forces and moments show that fair levels of verification and validation are established for most of the considered cases. A brief summary of the levels validation says that the bare hull results are validated at levels from 4.2% to 9.3%. For the appended hull the levels of validation for the rudder forces and the overall forces and moments range from 3.4% to 28.0% and from 6.3% to 37.2% for the “static rudder” and “pure drift” conditions, respectively. Further, it appears that even though validation is not achieved for all the cases, the method is generally capable of capturing the overall behavior of the integral quantities when the rudder and drift angles are varied.     

The relative utility of regression and artificial neural networks models for rapidly predicting the capacity of water supply reservoirs

Rapid prediction tools for reservoir over-year and within-year capacities that dispense with the sequential analysis of time-series runoff data are developed using multiple linear regression and multi-layer perceptron, artificial neural networks (MLP-ANNs). Linear regression was used to model the total (i.e. within-year + over-year) capacity using the over-year capacity as one of the inputs, while the ANNs were used to simultaneously model directly the over-year and total capacities. The inputs used for the ANNs were basic runoff and systems variables such as the coefficient of variation (Cv) of annual and monthly runoff, minimum monthly runoff, the demand ratio and reservoir reliability. The results showed that all the models performed well during their development and when they were tested with independent data sets. Both models offer faster prediction tools for reservoir capacity at gauged sites when compared with behaviour simulation. Additionally, when the predictor variables can be evaluated at un-gauged sites using e.g. catchment characteristics, they make capacity estimation at such un-gauged sites a feasible proposition.     

Neural networks for the prediction and forecasting of water resources variables: a review of modelling issues and applications

Artificial Neural Networks (ANNs) are being used increasingly to predict and forecast water resources variables. In this paper, the steps that should be followed in the development of such models are outlined. These include the choice of performance criteria, the division and pre-processing of the available data, the determination of appropriate model inputs and network architecture, optimisation of the connection weights (training) and model validation. The options available to modellers at each of these steps are discussed and the issues that should be considered are highlighted. A review of 43 papers dealing with the use of neural network models for the prediction and forecasting of water resources variables is undertaken in terms of the modelling process adopted. In all but two of the papers reviewed, feedforward networks are used. The vast majority of these networks are trained using the backpropagation algorithm. Issues in relation to the optimal division of the available data, data pre-processing and the choice of appropriate model inputs are seldom considered. In addition, the process of choosing appropriate stopping criteria and optimising network geometry and internal network parameters is generally described poorly or carried out inadequately. All of the above factors can result in non-optimal model performance and an inability to draw meaningful comparisons between different models. Future research efforts should be directed towards the development of guidelines which assist with the development of ANN models and the choice of when ANNs should be used in preference to alternative approaches, the assessment of methods for extracting the knowledge that is contained in the connection weights of trained ANNs and the incorporation of uncertainty into ANN models.     

Protocol for developing ANN models and its application to the assessment of the quality of the ANN model development process in drinking water quality modelling

The application of Artificial Neural Networks (ANNs) in the field of environmental and water resources modelling has become increasingly popular since early 1990s. Despite the recognition of the need for a consistent approach to the development of ANN models and the importance of providing adequate details of the model development process, there is no systematic protocol for the development and documentation of ANN models. In order to address this shortcoming, such a protocol is introduced in this paper. In addition, the protocol is used to critically review the quality of the ANN model development and reporting processes employed in 81 journal papers since 2000 in which ANNs have been used for drinking water quality modelling. The results show that model architecture selection is the best implemented step, while greater focus should be given to input selection considering input independence and model validation considering replicative and structural validity.     

Exactly satisfying initial conditions neural network models for numerical treatment of first Painlevé equation

In this paper, novel computing approach using three different models of feed-forward artificial neural networks (ANNs) are presented for the solution of initial value problem (IVP) based on first Painlevé equation. These mathematical models of ANNs are developed in an unsupervised manner with capability to satisfy the initial conditions exactly using log-sigmoid, radial basis and tan-sigmoid transfer functions in hidden layers to approximate the solution of the problem. The training of design parameters in each model is performed with sequential quadratic programming technique. The accuracy, convergence and effectiveness of the proposed schemes are evaluated on the basis of the results of statistical analyses through sufficient large number of independent runs with different number of neurons in each model as well. The comparisons of these results of proposed schemes with standard numerical and analytical solutions validate the correctness of the design models.     

Backoff hierarchical class n-gram language models: effectiveness to model unseen events in speech recognition

In this paper, we introduce the backoff hierarchical class n-gram language models to better estimate the likelihood of unseen n-gram events. This multi-level class hierarchy language modeling approach generalizes the well-known backoff n-gram language modeling technique. It uses a class hierarchy to define word contexts. Each node in the hierarchy is a class that contains all the words of its descendant nodes. The closer a node to the root, the more general the class (and context) is. We investigate the effectiveness of the approach to model unseen events in speech recognition. Our results illustrate that the proposed technique outperforms backoff n-gram language models. We also study the effect of the vocabulary size and the depth of the class hierarchy on the performance of the approach. Results are presented on Wall Street Journal (WSJ) corpus using two vocabulary set: 5000 words and 20,000 words. Experiments with 5000 word vocabulary, which contain a small numbers of unseen events in the test set, show up to 10% improvement of the unseen event perplexity when using the hierarchical class n-gram language models. With a vocabulary of 20,000 words, characterized by a larger number of unseen events, the perplexity of unseen events decreases by 26%, while the word error rate (WER) decreases by 12% when using the hierarchical approach. Our results suggest that the largest gains in performance are obtained when the test set contains a large number of unseen events.     

Damage detection in Timoshenko beam structures by multilayer perceptron and radial basis function networks

This study investigates the efficiency of artificial neural networks (ANNs) in health monitoring of pristine and damaged beam-like structures. Beam modeling is based on Timoshenko theory. Two commonly used network models, multilayer perceptron (MLP) and radial basis neural network (RBNN), are used. Beam material and geometrical properties, beam end conditions and dynamically obtained data are used as input to the neural networks. The combinations of these parameters yield umpteenth input data. Therefore, to examine the effectiveness of ANNs, the frequency of intact beams is first tried to be determined by the network models, given the material and geometrical characteristics of beam elements and support conditions. The methodology to compute the vibrational data utilized in training the networks is provided. Showing the robustness of network models, the second stage of the study is carried out. At this stage, the crack parameters (e.g. the location and severity of crack) are estimated by the ANNs using the beam properties, beam end conditions and vibrational data, which consist of natural frequencies and mode shape rotation values. Despite the multiplexed input data, no data reduction schemes or multistage computations are executed in training and validation of neural network models. As a result of analysis runs, the optimal MLP and RBNN models are determined. Comparison of these models shows that the optimal RBNN algorithm performs better. The effectiveness of optimal ANN models in the presence of noise is also presented. As a conclusion, the trained network can be used as a diagnosis method in structural health monitoring of beam-like structures.     

A study of overfitting in optimization of a manufacturing quality control procedure

Quality control of the commutator manufacturing process can be automated by means of a machine learning model that can predict the quality of commutators as they are being manufactured. Such a model can be constructed by combining machine vision, machine learning and evolutionary optimization techniques. In this procedure, optimization is used to minimize the model error, which is estimated using single cross-validation. This work exposes the overfitting that emerges in such optimization. Overfitting is shown for three machine learning methods with different sensitivity to it (trees, additionally pruned trees and random forests) and assessed in two ways (repeated cross-validation and validation on a set of unseen instances). Results on two distinct quality control problems show that optimization amplifies overfitting, i.e., the single cross-validation error estimate for the optimized models is overly optimistic. Nevertheless, minimization of the error estimate by single cross-validation in general results in minimization of the other error estimates as well, showing that optimization is indeed beneficial in this context.     

Smart meter monitoring and data mining techniques for predicting refrigeration system performance

A major challenge in many countries is providing sufficient energy for human beings and for supporting economic activities while minimizing social and environmental harm. This study predicted coefficient of performance (COP) for refrigeration equipment under varying amounts of refrigerant (R404A) with the aids of data mining (DM) techniques. The performance of artificial neural networks (ANNs), support vector machines (SVMs), classification and regression tree (CART), multiple regression (MR), generalized linear regression (GLR), and chi-squared automatic interaction detector (CHAID) were applied within DM process. After obtaining the COP value, abnormal equipment conditions can be evaluated for refrigerant leakage. Analytical results from cross-fold validation method are compared to determine the best models. The study shows that DM techniques can be used for accurately and efficiently predicting COP. In the liquid leakage phase, ANNs provide the best performance. In the vapor leakage phase, the best model is the GLR model. Experimental results confirm that systematic analyses of model construction processes are effective for evaluating and optimizing refrigeration equipment performance.     

VCR: Virtual community recommender using the technology acceptance model and the user’s needs type

A recommender system is a kind of automated and sophisticated decision support system that is needed to provide a personalized solution in a brief form without going through a complicated search process. There have been a substantial number of studies to make recommender systems more accurate and efficient, however, most of them have a common critical limitation – these systems are used as virtual salespeople, rather than as marketing tools. A crucial reason for this phenomenon is that the models suggested by prior studies only focus on a user’s behavioral outcomes without consideration of the embedded procedure. In this study, we propose a novel recommender system based on user’s behavioral model. Our proposed system, labeled VCR—virtual community recommender, recommends optimal virtual communities for an active user by case-based reasoning (CBR) using behavioral factors suggested in the technology acceptance model (TAM) and its extended models. In addition, it refines its recommendation results by considering the user’s needs type at the point of usage. To test the usefulness of our recommendation model, we conducted two-step validation–empirical validation for the collected data set, and practical validation to investigate the actual satisfaction level of users. Experimental results showed that our model outperformed all comparative models from the perspective of user satisfaction.     

Comparing the performance of multispectral vegetation indices and machine-learning algorithms for remote estimation of chlorophyll content: a case study in the Sundarbans mangrove forest

Optical vegetation indices (VIs) have been used to retrieve and assess biophysical variables from satellite reflectance data. These indices, however, also are sensitive to a number of confounding factors, such as canopy geometry, soil optical properties, and solar position. This suggests that VIs should be used cautiously for biophysical parameter estimation. Among biophysical variables, chlorophyll content is of particular importance as an indicator of photosynthetic activity. The goal of this study is to investigate the performance of multispectral optical VIs for chlorophyll content estimation in the world’s largest mangrove forest, the Sundarbans, and to compare these with machine-learning algorithms (MLAs). To this end, we have investigated the performance of 15 multispectral VIs and six state-of-the-art MLAs that are widely used for adaptive data fitting. The MLAs are Artificial Neural Networks (ANNs), Genetic Algorithm (GA), Gaussian Processes for Machine Learning (GPML), Kernel Ridge Regression (KRR), Locally Weighted Polynomials (LWP), and Multivariate Adaptive Regression Splines (MARS). We use an in situ data set of reflectance and chlorophyll measurements to develop and validate our models. Each MLA was evaluated 500 times with random partitions of training and validation data. Results showed that the weight optimization and term selection used within GA produce the most reliable chlorophyll content estimation. However, green normalized difference VI (GNDVI) is a simple and computationally efficient VI that produces results that are nearly as accurate as GA in terms of model fit and performance. Results also show that all methods except ANNs and MARS produce a quasi-linear relationship between spectral reflectance and chlorophyll content. Statistical transformations of GNDVI and chlorophyll content have the capability of further reducing model error.     

Using a heuristic approach to derive a grey-box model through an artificial neural network knowledge extraction technique

Artificial neural networks (ANNs) are primarily used in academia for their ability to model complex nonlinear systems. Though ANNs have been used to solve practical problems in industry, they are not typically used in nonacademic environments because they are not very well understood, complicated to implement, or have the reputation of being a “black-box” model. Few mathematical models exist that outperform ANNs. If a highly accurate model can be constructed, the knowledge should be used to understand and explain relationships in a system. Output surfaces can be analyzed in order to gain additional knowledge about a system being modeled. This paper presents a systematic approach to derive a “grey-box” model from the knowledge obtained from the ANN. A database for an automobile’s gas mileage performance is used as a case study for the proposed methodology. The results show a greater ability to generalize system behavior than other benchmarked methods.     

A study of ANFIS-based multi-factor time series models for forecasting stock index

Despite the widespread use of time series models in stock index forecasts, some of these models have encountered problems: (1) the selection of input factors may depend on personal experience or opinion; and (2) most conventional time series models consider only one variable. Furthermore, traditional forecasting models suffer from the following drawbacks: (1) models may rely on restrictive assumptions (such as linear separability or normality) about the variables being analyzed; and (2) it is hard to define and select applicable input factors for artificial neural networks (ANNs) in particular, and the rules generated from ANNs are not easily understood. To address these issues, we propose a multi-factor time series model based on an adaptive network-based fuzzy inference system (ANFIS) for stock index forecasting. In the proposed model, stepwise regression was first applied for the objective selection of technical indicators and then combined with ANFIS to construct the forecasting model. We evaluated the performance of our proposed model against three other models, with transaction data from the Taiwan Stock Exchange Capitalization Weighted Stock Index (TAIEX) and the Hong Kong Hang Seng Index (HSI) stock markets from 1998 to 2006 as experimental data sets and the root mean square error (RMSE) as the evaluation criterion. The results show the superiority of the proposed combined model, which outperformed other models in terms of RMSE and profitability, with strategies for increasing long-term uses of stock index forecasts made on the TAIEX and the HSI.     

Artificial neural networks for predicting dorsal pressures on the foot surface while walking

In this work, artificial neural networks (ANNs) are proposed to predict the dorsal pressure over the foot surface exerted by the shoe upper while walking. A model that is based on the multilayer perceptron (MLP) is used since it can provide a single equation to model the exerted pressure for all the materials used as shoe uppers. Five different models are produced, one model for each one of the four subjects under study and an overall model for the four subjects. The inputs to the neural model include the characteristics of the material and the positions during a whole step of 14 pressure sensors placed on the foot surface. The goal is to find models with good generalization capabilities, (i.e., models that work appropriately not only for the cases used to train the model but also for new cases) in order to have a useful predictor in routine practice. New cases may involve either new materials for the same subject or even new subjects and new materials. To accomplish this goal, two thirds of the patterns are trained to obtain the model (training data set) and the remaining third is kept for validation purposes. The achieved accuracy was very satisfactory since correlation coefficients between the predicted output and the actual pressure in the validation data were higher than 0.95 for those models developed for individual subjects. For the much more challenging problem of an overall prediction for all the subjects, the correlation coefficient was close to 0.9 in the validation data set (i.e., with data not previously seen by the model).     

Neuro-fuzzy MIMO nonlinear control for ceramic roller kiln

Artificial neural networks (ANNs) and neuro-fuzzy systems (NFSs) have been widely used in modeling and control of many practical industrial nonlinear processes. However, most of them have concentrated on single-output systems only. In this paper, we present a comparative study using ANNs and co-active neuro-fuzzy inference system (CANFIS) in modeling a real, complicated multi-input–multi-output (MIMO) nonlinear temperature process of roller kiln used in ceramic tile manufacturing line. Using this study, we prove that CANFIS is better suited for modeling the temperature process in control phase. After that, a neural network (NN) controller has been developed to control the above mentioned temperature process due to a feedback control diagram. The designed controller performance is tested by a Visual C++ project and the resulting numerical data shows that this controller can work accurately and reliably when the roller kiln set-point temperature set changes.     

Application of integrated soft computing techniques for optimisation of hybrid CO2 laser–MIG welding process

In this paper, artificial neural networks (ANNs), genetic algorithm (GA), simulated annealing (SA) and Quasi Newton line search techniques have been combined to develop three integrated soft computing based models such as ANN–GA, ANN–SA and ANN–Quasi Newton for prediction modelling and optimisation of welding strength for hybrid CO₂ laser–MIG welded joints of aluminium alloy. Experimental dataset employed for the purpose has been generated through full factorial experimental design. Laser power, welding speeds and wires feed rate are considered as controllable input parameters. These soft computing models employ a trained ANN for calculation of objective function value and thereby eliminate the need of closed form objective function. Among 11 tested networks, the ANN with best prediction performance produces maximum percentage error of only 3.21%. During optimisation ANN–GA is found to show best performance with absolute percentage error of only 0.09% during experimental validation. Low value of percentage error indicates efficacy of models. Welding speed has been found as most influencing factor for welding strength.     

Forecasting financial time series volatility using Particle Swarm Optimization trained Quantile Regression Neural Network

Accurate forecasting of volatility from financial time series is paramount in financial decision making. This paper presents a novel, Particle Swarm Optimization (PSO)-trained Quantile Regression Neural Network namely PSOQRNN, to forecast volatility from financial time series. We compared the effectiveness of PSOQRNN with that of the traditional volatility forecasting models, i.e., Generalized Autoregressive Conditional Heteroskedasticity (GARCH) and three Artificial Neural Networks (ANNs) including Multi-Layer Perceptron (MLP), General Regression Neural Network (GRNN), Group Method of Data Handling (GMDH), Random Forest (RF) and two Quantile Regression (QR)-based hybrids including Quantile Regression Neural Network (QRNN) and Quantile Regression Random Forest (QRRF). The results indicate that the proposed PSOQRNN outperformed these models in terms of Mean Squared Error (MSE), on a majority of the eight financial time series including exchange rates of USD versus JPY, GBP, EUR and INR, Gold Price, Crude Oil Price, Standard and Poor 500 (S&P 500) Stock Index and NSE India Stock Index considered here. It was corroborated by the Diebold–Mariano test of statistical significance. It also performed well in terms of other important measures such as Directional Change Statistic (Dstat) and Theil's Inequality Coefficient. The superior performance of PSOQRNN can be attributed to the role played by PSO in obtaining the better solutions. Therefore, we conclude that the proposed PSOQRNN can be used as a viable alternative in forecasting volatility.     

Comparison of ANN approach with 2D and 3D hydrodynamic models for simulating estuary water stage

Accurately predicting tidal levels, including tidal and freshwater discharge effects, is important for human activities in estuaries. The traditional harmonic analysis method and numerical modeling are usually adopted to simulate and predict estuary water stages. This study applied artificial neural networks (ANNs) as an alternative modeling approach to simulate the water stage time-series of the Danshui River estuary in northern Taiwan. We compared this approach with vertical (laterally averaged) 2D and 3D hydrodynamic models. Five ANN models were constructed to simulate the water stage time-series at the Shizi Tou, Taipei Bridge, Rukuoyan, Xinhai Bridge, and Zhongzheng Bridge locations along the Danshui River estuary. ANN models can preserve nonlinear characteristics between input and output variables and are superior to physical-based hydrodynamic models during the training phase. The simulated results reveal that the vertical 2D and 3D hydrodynamic models could not capture the observed water stages during an input of high freshwater discharge from upstream boundaries, while the ANN could match the observed water stage. However, during the testing phase, the ANN approach was slightly inferior to the 2D and 3D models at the Xinhai Bridge, Zhongzheng Bridge, and Rukouyan locations. Our results show that the ANN was able to predict the water stage time-series with reasonable accuracy, suggesting that ANNs can be a valuable tool for estuarine management.