Models to predict the deformation modulus and the coefficient of subgrade reaction for earth filling structures

Determination of deformation modulus and coefficient of subgrade reaction of soils have major importance, whether the projects are in design, and construction or compaction assessment stage of earth filling structures. Plate load test is one of the frequently used method to directly determine the parameters but the method is both costly and time consuming. For this reason, this paper is concerned with the applications of artificial neural networks (ANN) and simple-multiple regression analysis to predict deformation modulus and coefficient of subgrade reaction of compacted soils from compaction parameters (such as maximum dry density (MDD) and optimum moisture content (OMC), field dry density (FDD), and field moisture content (FMC)). Regression analysis and artificial neural network estimation indicated that there are acceptable correlations between deformation modulus and coefficient of subgrade reaction and these parameters. Artificial neural networks model exhibits higher performance than traditional statistical model for predicting deformation modulus and coefficient of subgrade reaction.     

Developing an advanced ANN-based approach to estimate compaction characteristics of highway subgrade

The current methodology of assessing the compaction characteristic of subgrade fillers is time-consuming. The spot-sampling method and the following laboratory tests bring bias to engineering practice. This study proposes an advanced approach to determine the maximum dry density, optimum moisture content, and compactness of subgrade based on ANN algorithms. A large number of compaction tests are performed to assess the compaction quality of various types of subgrade fillers. The particle gradation is specifically considered. Wave propagations of the compacted specimen are assessed based on the ultrasonic test, and the anisotropic feature is demonstrated. The dynamic elastic parameters are derived to further reveal the interaction mechanism. A dataset is proposed by combining these laboratory data and the literature data. The PSO-BP-NN model is developed to automatically predict the compaction parameters of different filling materials. A positive correlation is demonstrated between the elastic modulus and the total density, and Poisson’s ratio illustrates an inverse trend. The compaction energy is influential to the maximum dry density while the moisture content has the greatest impact on the compactness. This study aims to develop an accurate model to replace the extensive Proctor test and efficiently evaluate the compaction quality of subgrade for highway constructions.     

Image interpolation using MLP neural network with phase compensation of wavelet coefficients

When interpolating images in the wavelet domain, the main problem is how to estimate the finest detail coefficients. Wavelet coefficients across scales have an interscale dependency, and the dependency varies according to the local energy of the coefficients. This implies the possible existence of functional mappings from one scale to another scale. If we can estimate the mapping parameters from the observed coefficients, then it is possible to predict the finest detail coefficients. In this article, we use the multilayer perceptron (MLP) neural networks to learn a mapping from the coarser scale to the finer scale. When exploiting the MLP neural networks, phase uncertainty, a well-known drawback of wavelet transforms, makes it difficult for the networks to learn the interscale mapping. We solve this location ambiguity by using a phase-shifting filter. After the single-level phase compensation, a wavelet coefficient vector is assigned to one of the energy-dependent classes. Each class has its corresponding network. In the simulation results, we show that the proposed scheme outperforms the previous wavelet-domain interpolation method as well as the conventional spatial domain methods.     

神经网络在软件失效预测中的性能比较和研究

在软件开发的早期预测有失效倾向的软件模块,能够极大地提高软件的质量.软件失效预测中的一个普遍问题是数据中噪声的存在.神经网络具有鲁棒性而且对噪声有很强的抑制能力.不同结构的神经网络在训练算法和应用领域都有差异.该文主要就软件失效预测这个应用领域叙述几种适用的网络,并比较这几种网络在训练结果和性能上的差异.上述方法在SDH通信软件的失效预测中得到了成功的应用.试验结果显示虽然MLP、PNN、LVQ网络都能解决这类模式分类问题,但是只有MLP网络训练结果比较稳定,在不同的数据集上训练出的网络都有很好的预测效果.     

Application of modified MLP input weights’ matrices: an analysis of sectorial investment distribution in the emerging markets

This paper aims to explore the modified multilayer perceptron (MLP) input weights’ (IW) matrices relating them with the weights of the constituent input determinants. Non-traditional MLP topologies were designed, optimized and compared with other neural networks (NN) and multidimensional linear regression methods and statistically tested. The chosen NN topology directly related the MLP IW matrices with the relative contribution of each input variable. The contribution (weights) of each input variable was estimated in a non-linear manner, which is a novel approach in the investment research domain. This approach was applied to an investigation of sectorial investment distribution in emerging investment markets. To our knowledge, there is no experiment in the field that would focus on the NN mechanisms of sectorial indices (SI) weights estimation in such an experimental setting. In summary, we found apparent correlations between multivariate linear and other NN estimates (like Garson’s, Tchaban’s and SNA methods) having some new results not revealed in the previous research.     

Extraction of Logical Rules from Neural Networks

Three neural-based methods for extraction of logical rules from data are presented. These methods facilitate conversion of graded response neural networks into networks performing logical functions. MLP2LN method tries to convert a standard MLP into a network performing logical operations (LN). C-MLP2LN is a constructive algorithm creating such MLP networks. Logical interpretation is assured by adding constraints to the cost function, forcing the weights to ±1 or 0. Skeletal networks emerge ensuring that a minimal number of logical rules are found. In both methods rules covering many training examples are generated before more specific rules covering exceptions. The third method, FSM2LN, is based on the probability density estimation. Several examples of performance of these methods are presented.     

Neural computing models for prediction of permeability coefficient of coarse-grained soils

Correlations are very significant from the earliest days; in some cases, it is essential as it is difficult to measure the amount directly, and in other cases it is desirable to ascertain the results with other tests through correlations. Soft computing techniques are now being used as alternate statistical tool, and new techniques such as artificial neural networks, fuzzy inference systems, genetic algorithms, and their hybrids were employed for developing the predictive models to estimate the needed parameters, in the recent years. Determination of permeability coefficient (k) of soils is very important for the definition of hydraulic conductivity and is difficult, expensive, time-consuming, and involves destructive tests. In this paper, use of some soft computing techniques such as ANNs (MLP, RBF, etc.) and ANFIS (adaptive neuro-fuzzy inference system) for prediction of permeability of coarse-grained soils was described and compared. As a result of this paper, it was obtained that the all constructed soft computing models exhibited high performance for predicting k. In order to predict the permeability coefficient, ANN models having three inputs, one output were applied successfully and exhibited reliable predictions. However, all four different algorithms of ANN have almost the same prediction capability, and accuracy of MLP was relatively higher than RBF models. The ANFIS model for prediction of permeability coefficient revealed the most reliable prediction when compared with the ANN models, and the use of soft computing techniques will provide new approaches and methodologies in prediction of some parameters in soil mechanics.     

Sensitivity analysis of multilayer perceptron with differentiableactivation functions

In a neural network, many different sets of connection weights can approximately realize an input-output mapping. The sensitivity of the neural network varies depending on the set of weights. For the selection of weights with lower sensitivity or for estimating output perturbations in the implementation, it is important to measure the sensitivity for the weights. A sensitivity depending on the weight set in a single-output multilayer perceptron (MLP) with differentiable activation functions is proposed. Formulas are derived to compute the sensitivity arising from additive/multiplicative weight perturbations or input perturbations for a specific input pattern. The concept of sensitivity is extended so that it can be applied to any input patterns. A few sensitivity measures for the multiple output MLP are suggested. For the verification of the validity of the proposed sensitivities, computer simulations have been performed, resulting in good agreement between theoretical and simulation outcomes for small weight perturbations.     

Centroid based Multilayer Perceptron Networks

In this study we investigate a hybrid neural network architecture for modelling purposes. The proposed network is based on the multilayer perceptron (MLP) network. However, in addition to the usual hidden layers the first hidden layer is selected to be a centroid layer. Each unit in this new layer incorporates a centroid that is located somewhere in the input space. The output of these units is the Euclidean distance between the centroid and the input. The centroid layer clearly resembles the hidden layer of the radial basis function (RBF) networks. Therefore the centroid based multilayer perceptron (CMLP) networks can be regarded as a hybrid of MLP and RBF networks. The presented benchmark experiments show that the proposed hybrid architecture is able to combine the good properties of MLP and RBF networks resulting fast and efficient learning, and compact network structure.     

Speeding up MLP Execution through Difference Forward Propagation

At present the Multi-layer Perceptron model (MLP) is, without doubt, the most diffused neural network for applications. So, it is important, from an engineering point of view, to design and test methods to improve MLP efficiency at run time. This paper describes a simple but effective method to cut down execution time for MLP networks dealing with a sequential input. This case is very common, including all kinds of temporal processing, like speech, video, and in general signals varying in time. The suggested technique requires neither specialized hardware nor big quantities of additional memory. The method is based on the ubiquitous idea of difference transmission, widely used in signal coding. For each neuron, the activation value at a certain moment is compared with the corresponding activation value computed at the previous net forward computation: if no relevant change occurred the neuron does not perform any computation, otherwise it propagates to the connected neurons the difference of its two activations multiplied by its outgoing weights. The method requires the introduction of a quantization of the unit activation function that causes an error which is analyzed empirically. In particular, the effectiveness of the method is verified on two speech recognition tasks with two different neural networks architectures. The results show a drastic reduction of the execution time on both the neural architectures and no significant changes in recognition quality.     

Modeling of permeability and compaction characteristics of soils using evolutionary polynomial regression

This paper presents a new approach, based on evolutionary polynomial regression (EPR), for prediction of permeability (K), maximum dry density (MDD), and optimum moisture content (OMC) as functions of some physical properties of soil. EPR is a data-driven method based on evolutionary computing aimed to search for polynomial structures representing a system. In this technique, a combination of the genetic algorithm (GA) and the least-squares method is used to find feasible structures and the appropriate parameters of those structures. EPR models are developed based on results from a series of classification, compaction, and permeability tests from the literature. The tests included standard Proctor tests, constant head permeability tests, and falling head permeability tests conducted on soils made of four components, bentonite, limestone dust, sand, and gravel, mixed in different proportions. The results of the EPR model predictions are compared with those of a neural network model, a correlation equation from the literature, and the experimental data. Comparison of the results shows that the proposed models are highly accurate and robust in predicting permeability and compaction characteristics of soils. Results from sensitivity analysis indicate that the models trained from experimental data have been able to capture many physical relationships between soil parameters. The proposed models are also able to represent the degree to which individual contributing parameters affect the maximum dry density, optimum moisture content, and permeability.     

Handwritten characters recognition based on nature-inspired computing and neuro-evolution

The enormous services obtainable by bank and postal systems are not 100 % guaranteed due to variability of handwriting styles. Various methods based on neural networks have been suggested to address this issue. Unfortunately, they often fall into local optima that arises from the use of old learning methods. Global optimization methods provided new directions for neural networks evolution that may be useful in recognition. This paper develops efficient algorithms that compute globally optimal solutions by exploiting the benefits of both swarm intelligence and neuro-evolution in a way to improve the overall performance of a character recognition system. Various adaptations implied to both MLP and RBF networks have been suggested namely: particle swarm optimization (PSO) and the bees algorithm (BA) for characters classification, MLP training or RBF design by co-evolution and effective combinations of MLPs, RBFs or SVMs as an attempt to overcome the drawbacks of old recognition methods. Results proved that networks combination proposals ensure the highest improvement compared to either standard MLP and RBF networks, the co-evolutionary alternatives or other classifiers combination based on common combination rules namely majority voting, the fusion rules of min, max, sum, average, product and Bayes, Decision template and the Behavior Knowledge Space (BKS).     

APPLICATION OF ARTIFICIAL NEURAL NETWORKS FOR PREDICTING pH IN SEAWATER ALONG GAZA BEACH

Coastal water issues are gaining worldwide attention because of their impact on health and other environmental problems. This article is concerned with the comparison between artificial neural networks and statistical methods to predict the degree of acidity (pH) in the coastal waters along the Gaza beach. Multilayer perceptron (MLP) and radial basis function (RBF) neural networks are trained and developed with reference to three parameters (water temperature, wind velocity, and turbidity) to predict the level of pH in the seawater. Both networks were developed using the combination of the data collected from nine sites over a period of 4 years, including 294 samples for training and 90 samples for testing the performance of models. The results show that the MLP and RBF models have good ability to predict the pH level. Each network's performance was tested with different sets of data, and the results show satisfactory performance. Results of the developed networks were compared with the statistical regression method and found that the predictions of neural networks are better than the conventional methods. Predictions result show that artificial neural networks approach have good ability for the modeling of pH level in the coastal waters along Gaza beach. It is hoped that neural networks will prove to be a promising alternative to traditional methods used and can contribute in the improvement of the quality of seawater.     

Bio-inspired and gradient-based algorithms to train MLPs: The influence of diversity

This paper has three main goals: (i) to employ two classes of algorithms: bio-inspired and gradient-based to train multi-layer perceptron (MLP) neural networks for pattern classification; (ii) to combine the trained neural networks into ensembles of classifiers; and (iii) to investigate the influence of diversity in the classification performance of individual and ensembles of classifiers. The optimization version of an artificial immune network, named opt-aiNet, particle swarm optimization (PSO) and an evolutionary algorithm (EA) are used as bio-inspired methods to train MLP networks. Besides, the standard backpropagation with momentum (BPM), a quasi-Newton method called DFP and a modified scaled-conjugate gradient (SCGM) are the gradient-based algorithms used to train MLP networks in this work. Comparisons among all the training methods are presented in terms of classification accuracy and diversity of the solutions found. The results obtained suggest that most bio-inspired algorithms deteriorate the diversity of solutions during the search, while immune-based methods, like opt-aiNet, and multiple initializations of standard gradient-based algorithms provide diverse solutions that result in good classification accuracy for the ensembles.     

前向MLP网络的FPGA实现

分别采用CORDIC(Coordinate rotation digital computer)算法和分布式算法实现多层感知器网络的传输函数计算和输入与权重乘积和计算,通过模块复用的方法构造了一个用于函数逼近的、无需乘法器的神经网络,并在NoisⅡ开发平台上测试了该网络的性能.该网络每17个时钟周期输出一个数据,占用FPGA的7781个LE(Logic element)和8976 bit存储器,具有良好的扩展性.     

Evolutionary training of hardware realizable multilayer perceptrons

The use of multilayer perceptrons (MLP) with threshold functions (binary step function activations) greatly reduces the complexity of the hardware implementation of neural networks, provides tolerance to noise and improves the interpretation of the internal representations. In certain case, such as in learning stationary tasks, it may be sufficient to find appropriate weights for an MLP with threshold activation functions by software simulation and, then, transfer the weight values to the hardware implementation. Efficient training of these networks is a subject of considerable ongoing research. Methods available in the literature mainly focus on two-state (threshold) nodes and try to train the networks by approximating the gradient of the error function and modifying appropriately the gradient descent, or by progressively altering the shape of the activation functions. In this paper, we propose an evolution-motivated approach, which is eminently suitable for networks with threshold functions and compare its performance with four other methods. The proposed evolutionary strategy does not need gradient related information, it is applicable to a situation where threshold activations are used from the beginning of the training, as in “on-chip” training, and is able to train networks with integer weights.     

A non-symbolic implementation of abdominal pain estimation in childhood

The abdominal pain is a very common disease in childhood, which lurks complications. Pediatric surgeons have to estimate at least 15 clinical and laboratory factors in order to make a diagnosis and decide about performing a surgical operation of the abdomen. Artificial Neural Networks (ANNs) are particular implementations of Artificial Intelligence (AI) systems and they are used in a wide area of application fields. This study examines the implementation of ANN architectures, using Multi-Layer Perceptron (MLP) neural networks and Probabilistic Neural Networks (PNN) architectures, in order to specify the appropriate ANN structure for abdominal pain estimation in childhood. The architecture with the best performance is a fully interconnected MLP neural network with an input layer of 15 nodes, one hidden layer of 5 neurons and an output layer, with error back-propagation algorithm being used as the learning scheme. In the output layer, the estimation of appendicitis’ stage is reached automatically. The proposed ANN achieved a percentage of 88.5% of correct classification on testing set cases. Further analysis of obtained results, exhibited the ability of ANN for distinguishing the necessity of a case for operative treatment of abdominal pain based on diagnostic features, attaining a percentage of 100% of successful prognosis over the cases of testing set. The aim of proposed MLP neural network is to assist surgeons in appendicitis prediction, avoiding an unnecessary operative treatment.     

基于条件生成对抗网络的侧信道攻击技术研究

近年来,深度学习技术广泛应用于侧信道攻击(side channel attack,SCA)领域.针对在基于深度学习的侧信道攻击中训练集数量不足的问题,提出了一种用于侧信道攻击的功耗轨迹扩充技术,使用条件生成对抗网络(conditional generate against network,CGAN)实现对原始功耗轨迹的...     

Combined projection and kernel basis functions for classification in evolutionary neural networks

This paper proposes a hybrid neural network model using a possible combination of different transfer projection functions (sigmoidal unit, SU, product unit, PU) and kernel functions (radial basis function, RBF) in the hidden layer of a feed-forward neural network. An evolutionary algorithm is adapted to this model and applied for learning the architecture, weights and node typology. Three different combined basis function models are proposed with all the different pairs that can be obtained with SU, PU and RBF nodes: product–sigmoidal unit (PSU) neural networks, product–radial basis function (PRBF) neural networks, and sigmoidal–radial basis function (SRBF) neural networks; and these are compared to the corresponding pure models: product unit neural network (PUNN), multilayer perceptron (MLP) and the RBF neural network. The proposals are tested using ten benchmark classification problems from well known machine learning problems. Combined functions using projection and kernel functions are found to be better than pure basis functions for the task of classification in several datasets.     

Using results of eye movement signal analysis in the neural network recognition of otoneurological patients

We extracted a collection of eye movement signals employed for almost two decades in clinical otoneurological tests at a balance laboratory. During those years we designed and programmed signal analysis methods to analyse their features in detail and to compute medically important attributes. In the present study, using such attributes and their results computed we classified test cases into groups of healthy subjects and patients with multilayer perceptron neural networks. Classification succeeded in total accuracies from 60% to 90% depending on the type of eye movements, which were saccades, nystagmus, sinusoidal movements and vestibulo-ocular reflex stimulated in two different ways; these are the chief eye movement tests applied in otoneurology.