ANN构造性设计中基于GA优选神经元激活函数类型

构造性设计是ANN设计的发展方向之一。全面的高质量的ANN学习应包括神经元激活函数类型的自动优化。该文在构造性设计的框架内讨论了如何实现典型前馈网络的包括神经元激活函数类型在内的全面学习。首先,提出了典型前馈网络的一种构造性设计方法的原理和算法框架,把整个网络的设计分解成了一个个单个神经元的设计问题;然后提出了基于GA的能实现激活函数类型优选的单个神经元的设计方法。大量函数拟合的仿真实验显示:与其它几种激活函数类型不优选的常见ANN设计方法相比,该文提出的方法更有效,能用较小的网络结构获得较好的泛化性能。     

Generation of optimal artificial neural networks using a pattern search algorithm: application to approximation of chemical systems

A pattern search optimization method is applied to the generation of optimal artificial neural networks (ANNs). Optimization is performed using a mixed variable extension to the generalized pattern search method. This method offers the advantage that categorical variables, such as neural transfer functions and nodal connectivities, can be used as parameters in optimization. When used together with a surrogate, the resulting algorithm is highly efficient for expensive objective functions. Results demonstrate the effectiveness of this method in optimizing an ANN for the number of neurons, the type of transfer function, and the connectivity among neurons. The optimization method is applied to a chemistry approximation of practical relevance. In this application, temperature and a chemical source term are approximated as functions of two independent parameters using optimal ANNs. Comparison of the performance of optimal ANNs with conventional tabulation methods demonstrates equivalent accuracy by considerable savings in memory storage. The architecture of the optimal ANN for the approximation of the chemical source term consists of a fully connected feedforward network having four nonlinear hidden layers and 117 synaptic weights. An equivalent representation of the chemical source term using tabulation techniques would require a 500 x 500 grid point discretization of the parameter space.     

Predicting neutron diffusion eigenvalues with a query-basedadaptive neural architecture

A query-based approach for adaptively retraining and restructuring a two-hidden-layer artificial neural network (ANN) has been developed for the speedy prediction of the fundamental mode eigenvalue of the neutron diffusion equation, a standard nuclear reactor core design calculation which normally requires the iterative solution of a large-scale system of nonlinear partial differential equations (PDEs). The approach developed focuses primarily upon the adaptive selection of training and cross-validation data and on artificial neural-network (ANN) architecture adjustments, with the objective of improving the accuracy and generalization properties of ANN-based neutron diffusion eigenvalue predictions. For illustration, the performance of a "bare bones" feedforward multilayer perceptron (MLP) is upgraded through a variety of techniques; namely, nonrandom initial training set selection, adjoint function input weighting, teacher-student membership and equivalence queries for generation of appropriate training data, and a dynamic node architecture (DNA) implementation. The global methodology is flexible in that it ran "wrap around" any specific training algorithm selected for the static calculations (i.e., training iterations with a fixed training set and architecture). Finally, the improvements obtained are carefully contrasted against past works reported in the literature.     

Adaptive load shedding for an industrial petroleum cogeneration system

This paper presents the design of adaptive load-shedding strategy by executing the artificial neural network (ANN) and transient stability analysis for an Industrial cogeneration facility. ? To prepare the training data set for ANN, the transient stability analysis has been performed to solve the minimum load shedding for various operation scenarios without causing tripping problem of cogeneration units. Various training algorithms have been adopted and incorporated into the back-propagation learning algorithm for the feed-forward neural networks. ? By selecting the total power generation, total load demand and frequency decay rate as the input neurons of the ANN, the minimum amount of load shedding is determined to maintain the stability of power system. ? To demonstrate the effectiveness of the ANN minimum load-shedding scheme, the traditional method and the present load shedding schemes of the selected cogeneration system are also applied for comparison and verification of the proposed methodology.     

Nearly-orthogonal sampling and neural network metamodel driven conceptual design of multistage space launch vehicle

This paper presents a new design methodology for efficient conceptual design of complex systems involving multidisciplinary and computationally intensive analysis with large number of design variables. A nearly-orthogonal sampling of design space is proposed with good space filling properties to extract maximum useful information about system behavior using much lower number of trial designs. This sampled data is then used as training data for artificial neural network, which will act as a metamodel to represent the time consuming disciplinary analyses. A stage-wise interconnection of separate neural networks is also proposed for trajectory metamodel to offset dimensionality curse of neural networks. Genetic Algorithm performs global optimization by utilizing this metamodel and subsequently sequential quadratic programming performs the local optimization utilizing exact analyses. The performance of proposed methodology is investigated in this paper for the conceptual design optimization of multistage solid fueled space launch vehicle. The results show excellent approximation of highly non-linear functions using proposed sampling and drastic reduction in overall design optimization time, due to greatly reduced number of exact disciplinary analyses.     

Metaheuristics for the feedforward artificial neural network (ANN) architecture optimization problem

This article deals with evolutionary artificial neural network (ANN) and aims to propose a systematic and automated way to find out a proper network architecture. To this, we adapt four metaheuristics to resolve the problem posed by the pursuit of optimum feedforward ANN architecture and introduced a new criteria to measure the ANN performance based on combination of training and generalization error. Also, it is proposed a new method for estimating the computational complexity of the ANN architecture based on the number of neurons and epochs needed to train the network. We implemented this approach in software and tested it for the problem of identification and estimation of pollution sources and for three separate benchmark data sets from UCI repository. The results show the proposed computational approach gives better performance than a human specialist, while offering many advantages over similar approaches found in the literature.     

Parallel neural net training on the KSR1

In modern day pattern recognition, neural nets are used extensively. General use of a feedforward neural net consists of a training phase followed by a classification phase. Classification of an unknown test vector is very fast and only consists of the propagation of the test vector through the neural net. Training involves an optimization procedure and is very time-consuming since a feasible local minimum is sought in high-dimensional weight space. In this paper we present an analysis of a parallel implementation of the backpropagation training algorithm using conjugate gradient optimization for a three-layered, feedforward neural network, on the KSR1 parallel shared-memory machine. We implement two parallel neural net training versions on the KSR1, one using native code, the other using P4, a library of macros and functions. A speedup model is presented which we use to clarify our experimental results. We identify the general requirements which render the parallel implementation useful, compared to the sequential execution of the same neural net training procedure. We determine the usefulness of a library of functions (such as P4) developed to ease the task of the programmer. Using experimental results we further identify the limits in processor utilization for our parallel training algorithm.     

Smooth function approximation using neural networks

An algebraic approach for representing multidimensional nonlinear functions by feedforward neural networks is presented. In this paper, the approach is implemented for the approximation of smooth batch data containing the function's input, output, and possibly, gradient information. The training set is associated to the network adjustable parameters by nonlinear weight equations. The cascade structure of these equations reveals that they can be treated as sets of linear systems. Hence, the training process and the network approximation properties can be investigated via linear algebra. Four algorithms are developed to achieve exact or approximate matching of input-output and/or gradient-based training sets. Their application to the design of forward and feedback neurocontrollers shows that algebraic training is characterized by faster execution speeds and better generalization properties than contemporary optimization techniques.     

基于改进型Hopfield神经网络的盲多用户检测算法

基于Hopfield神经网络没有学习规则,不需要训练,也不会自学习,靠Lyapunov函数的设计过程来调节权值的特点,将广义罚函数与Hopfield神经网络的能量函数结合,基于最小平均输出能量准则,构造出更合适的新目标函数,分析讨论了一种实现DS/CDMA盲多用户检测的改进型Hopfield神经网络方法。仿真结果表明,该算法在误码率、抗远近效应方面都有明显的改善。     

Application of a radial basis function artificial neural network to seismic data inversion

We investigate here the performance and the application of a radial basis function artificial neural network (RBF-ANN) type, in the inversion of seismic data. The proposed structure has the advantage of being easily trained by means of a back-propagation algorithm without getting stuck in local minima. The effects of network architectures, i.e. the number of neurons in the hidden layer, the rate of convergence and prediction accuracy of ANN models are examined. The optimum network parameters and performance were decided as a function of testing error convergence with respect to the network training error. An adequate cross-validation test is run to ensure the performance of the network on new data sets. The application of such a network to synthetic and real data shows that the inverted acoustic impedance section was efficient.     

An integrated approach for optimum design of bridge decks using genetic algorithms and artificial neural networks

The objective of this paper is to develop an integrated approach using artificial neural networks (ANN) and genetic algorithms (GA) for cost optimization of bridge deck configurations. In the present work, ANN is used to predict the structural design responses which are used further in evaluation of fitness and constraint violation in GA process. A multilayer back-propagation neural network is trained with the results obtained using grillage analysis program for different bridge deck configurations and the correlation between sectional parameters and design responses has been established. Subsequently, GA is employed for arriving at optimum configuration of the bridge deck system by minimizing the total cost. By integrating ANN with GA, the computational time required for obtaining optimal solution could be reduced substantially. The efficacy of this approach is demonstrated by carrying out studies on cost optimization of T-girder bridge deck system for different spans. The method presented in this paper, would greatly reduce the computational effort required to find the optimum solution and guarantees bridge engineers to arrive at the near-optimal solution that could not be easily obtained using general modeling programs or by trial-and-error.     

On Non-Uniform Rational B-Splines Surface Neural Networks

This article presents a simulation study for validation of an adaptation methodology for learning weights of a Hopfield neural network configured as a static optimizer. The quadratic Liapunov function associated with the Hopfield network dynamics is leveraged to map the set of constraints associated with a static optimization problem. This approach leads to a set of constraint-specific penalty or weighting coefficients whose values need to be defined. The methodology leverages a learning-based approach to define values of constraint weighting coefficients through adaptation. These values are in turn used to compute values of network weights, effectively eliminating the guesswork in defining weight values for a given static optimization problem, which has been a long-standing challenge in artificial neural networks. The simulation study is performed using the Traveling Salesman problem from the domain of combinatorial optimization. Simulation results indicate that the adaptation procedure is able to guide the Hopfield network towards solutions of the problem starting with random values for weights and constraint weighting coefficients. At the conclusion of the adaptation phase, the Hopfield network acquires weight values which readily position the network to search for local minimum solutions. The demonstrated successful application of the adaptation procedure eliminates the need to guess or predetermine the values for weights of the Hopfield network.     

基于Matlab的SCM822H齿轮钢性能预测

采用Matlab的人工神经网络工具箱建立BP人工神经元网络,预测SCM822H齿轮钢的性能.选择10×12×3网络结构及基于Levenberg-Marquardt 优化算法和改进的误差函数的训练函数trainbr,BP网络对SCM822H齿轮钢的性能进行快速训练的同时,使网络的泛性得到提高.最后对网络性能进行回归分析,证明了网络设计的合理性.使用训练好的网络对SCM822H齿轮钢力学性能及淬透性进行预测,预测结果表明,网络具有较高的预测精度,可在实际生产和科学研究中进行应用.     

Prediction of quality using ANN based on Teaching‐Learning Optimization in component‐based software systems

The primary objective of our research work is to enhance the prediction of the quality of a component‐based software system and to develop an artificial neural network (ANN) model for the system reliability optimization problem. In this paper, we introduced the ANN‐supported Teaching‐Learning Optimization by transforming constraints to objective functions. Artificial neural network techniques are found to be powerful in the modeling software package quality metrics compared with the ancient statistical techniques. Therefore, by using the neural network, the quality characteristics of software components of the proposed work are predicted. A nonlinear differentiable transfer function of ANN used in the proposed approach is hyperbolic tangent sigmoid. A new efficient optimization methodology referred to as the Teaching‐Learning–based Optimization is proposed in this paper to optimize reliability and different cost functions. The weight values of the network are then adjusted consistent with a proposed optimization rule, therefore minimizing the network error. The proposed work is implemented in MATLAB by using the Neural Network Toolbox. The proposed work provides improved performance in terms of sensitivity, precision, specificity, negative predictive value, fall‐out or false positive rate, false discovery rate, accuracy, Matthews correlation coefficient, and rate of convergence.     

Evolving neural network using real coded genetic algorithm for permeability estimation of the reservoir

In this work we investigate how artificial neural network (ANN) evolution with genetic algorithm (GA) improves the reliability and predictability of artificial neural network. This strategy is applied to predict permeability of Mansuri Bangestan reservoir located in Ahwaz, Iran utilizing available geophysical well log data. Our methodology utilizes a hybrid genetic algorithm–neural network strategy (GA–ANN). The proposed algorithm combines the local searching ability of the gradient–based back-propagation (BP) strategy with the global searching ability of genetic algorithms. Genetic algorithms are used to decide the initial weights of the gradient decent methods so that all the initial weights can be searched intelligently. The genetic operators and parameters are carefully designed and set avoiding premature convergence and permutation problems. For an evaluation purpose, the performance and generalization capabilities of GA–ANN are compared with those of models developed with the common technique of BP. The results demonstrate that carefully designed genetic algorithm-based neural network outperforms the gradient descent-based neural network.     

Customer churn prediction by hybrid neural networks

As churn management is a major task for companies to retain valuable customers, the ability to predict customer churn is necessary. In literature, neural networks have shown their applicability to churn prediction. On the other hand, hybrid data mining techniques by combining two or more techniques have been proved to provide better performances than many single techniques over a number of different domain problems. This paper considers two hybrid models by combining two different neural network techniques for churn prediction, which are back-propagation artificial neural networks (ANN) and self-organizing maps (SOM). The hybrid models are ANN combined with ANN (ANN + ANN) and SOM combined with ANN (SOM + ANN). In particular, the first technique of the two hybrid models performs the data reduction task by filtering out unrepresentative training data. Then, the outputs as representative data are used to create the prediction model based on the second technique. To evaluate the performance of these models, three different kinds of testing sets are considered. They are the general testing set and two fuzzy testing sets based on the filtered out data by the first technique of the two hybrid models, i.e. ANN and SOM, respectively. The experimental results show that the two hybrid models outperform the single neural network baseline model in terms of prediction accuracy and Types I and II errors over the three kinds of testing sets. In addition, the ANN + ANN hybrid model significantly performs better than the SOM + ANN hybrid model and the ANN baseline model.     

Developing an expert system for predicting alluvial channel geometry using ANN

Cross section geometry of stable alluvial channels usually is estimated by simple inaccurate empirical equations, because of the complexity of the phenomena and unknown physical processes of regime channels. So, the main purpose of this study is to evaluate the potential of simulating regime channel treatments using artificial neural networks (ANNs). The process of training and testing of this new model is done using a set of available published filed data (371 data numbers). Several statistical and graphical criterions are used to check the accuracy of the model in comparison with previous empirical equations. The multilayer perceptron (MLP) artificial neural network was used to construct the simulation model based on the training data using back-propagation algorithm. The results show a considerably better performance of the ANN model over the available empirical or rational equations. The constructed ANN models can almost perfectly simulate the width, depth and slope of alluvial regime channels, which clearly describes the dominant geometrical parameters of alluvial rivers. The results demonstrate that the ANN can precisely simulate the regime channel geometry, while the empirical, regression or rational equations can’t do this. The presented methodology in this paper is a new approach in establishing alluvial regime channel relations and predicting cross section geometry of alluvial rivers also it can be used to design stable irrigation and water conveyance channels.     

Design of ANN Based Non-Linear Network Using Interconnection of Parallel Processor

Suspicious mass traffic constantly evolves, making network behaviour tracing and structure more complex. Neural networks yield promising results by considering a sufficient number of processing elements with strong interconnections between them. They offer efficient computational Hopfield neural networks models and optimization constraints used by undergoing a good amount of parallelism to yield optimal results. Artificial neural network (ANN) offers optimal solutions in classifying and clustering the various reels of data, and the results obtained purely depend on identifying a problem. In this research work, the design of optimized applications is presented in an organized manner. In addition, this research work examines theoretical approaches to achieving optimized results using ANN. It mainly focuses on designing rules. The optimizing design approach of neural networks analyzes the internal process of the neural networks. Practices in developing the network are based on the interconnections among the hidden nodes and their learning parameters. The methodology is proven best for nonlinear resource allocation problems with a suitable design and complex issues. The ANN proposed here considers more or less 46k nodes hidden inside 49 million connections employed on full-fledged parallel processors. The proposed ANN offered optimal results in real-world application problems, and the results were obtained using MATLAB.     

A multilayer neural network with piecewise-linear structure andback-propagation learning

A multilayer neural network which is given a two-layer piecewise-linear structure for every cascaded section is proposed. The neural networks have nonlinear elements that are neither sigmoidal nor of a signum type. Each nonlinear element is an absolute value operator. It is almost everywhere differentiable, which makes back-propagation feasible in a digital setting. Both the feedforward signal propagation and the backward coefficient update rules belong to the class of regular iterative algorithms. This form of neural network specializes in functional approximation and is anticipated to have applications in control, communications, and pattern recognition.     

基于连续Hopfield网络的物流路径优化研究与仿真

路径优化问题一直是智能控制领域中一个重要的研究对象.针对连续Hopfield神经网络和离散Hopfield神经网络的优缺点,设计了一种基于连续Hopfield网络的物流路径规划方案.首先对网络的结构进行了阐述,同时引入了能量函数的概念,对网络的稳定性进行了证明.根据实际问题的描述,将路径行程映射为换位矩阵,将路径优化的目标函数映射为网络的能量函数,设计出目标函数的动态方程,方程的最小值就为路径规划的最优值.最终通过软件仿真,求得最优解,证明了网络的可行性.