Comparison of multiple linear regression and artificial neural network in developing the objective functions of the orthopaedic screws

Optimizing the orthopaedic screws can greatly improve their biomechanical performances. However, a methodical design optimization approach requires a long time to search the best design. Thus, the surrogate objective functions of the orthopaedic screws should be accurately developed. To our knowledge, there is no study to evaluate the strengths and limitations of the surrogate methods in developing the objective functions of the orthopaedic screws. Three-dimensional finite element models for both the tibial locking screws and the spinal pedicle screws were constructed and analyzed. Then, the learning data were prepared according to the arrangement of the Taguchi orthogonal array, and the verification data were selected with use of a randomized selection. Finally, the surrogate objective functions were developed by using either the multiple linear regression or the artificial neural network. The applicability and accuracy of those surrogate methods were evaluated and discussed. The multiple linear regression method could successfully construct the objective function of the tibial locking screws, but it failed to develop the objective function of the spinal pedicle screws. The artificial neural network method showed a greater capacity of prediction in developing the objective functions for the tibial locking screws and the spinal pedicle screws than the multiple linear regression method. The artificial neural network method may be a useful option for developing the objective functions of the orthopaedic screws with a greater structural complexity. The surrogate objective functions of the orthopaedic screws could effectively decrease the time and effort required for the design optimization process.     

支持向量机与RBF神经网络回归性能比较研究

支持向量机与RBF神经网络相比各有优缺点,通过对支持向量机与RBF神经网络的研究,从理论上分析了这两种学习机在回归预测原理上的异同,通过仿真实验对比了两者在测试集上的逼近能力及泛化能力。仿真结果表明,对于小样本集,支持向量机的逼近能力及泛化能力要优于RBF神经网络。对实际应用中回归模型的选择问题提出了建议。     

A novel hybrid classification model of artificial neural networks and multiple linear regression models

The classification problem of assigning several observations into different disjoint groups plays an important role in business decision making and many other areas. Developing more accurate and widely applicable classification models has significant implications in these areas. It is the reason that despite of the numerous classification models available, the research for improving the effectiveness of these models has never stopped. Combining several models or using hybrid models has become a common practice in order to overcome the deficiencies of single models and can be an effective way of improving upon their predictive performance, especially when the models in combination are quite different. In this paper, a novel hybridization of artificial neural networks (ANNs) is proposed using multiple linear regression models in order to yield more general and more accurate model than traditional artificial neural networks for solving classification problems. Empirical results indicate that the proposed hybrid model exhibits effectively improved classification accuracy in comparison with traditional artificial neural networks and also some other classification models such as linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), K-nearest neighbor (KNN), and support vector machines (SVMs) using benchmark and real-world application data sets. These data sets vary in the number of classes (two versus multiple) and the source of the data (synthetic versus real-world). Therefore, it can be applied as an appropriate alternate approach for solving classification problems, specifically when higher forecasting accuracy is needed.     

Logistic regression using covariates obtained by product-unit neural network models

We propose a logistic regression method based on the hybridation of a linear model and product-unit neural network models for binary classification. In a first step we use an evolutionary algorithm to determine the basic structure of the product-unit model and afterwards we apply logistic regression in the new space of the derived features. This hybrid model has been applied to seven benchmark data sets and a new microbiological problem. The hybrid model outperforms the linear part and the nonlinear part obtaining a good compromise between them and they perform well compared to several other learning classification techniques. We obtain a binary classifier with very promising results in terms of classification accuracy and the complexity of the classifier.     

基于广义回归神经网络的流量矩阵估计*

研究大尺度IP骨干网络流量矩阵估计,通过使用广义回归神经网络来捕捉流量矩阵特征,将流量矩阵估计描述成马氏距离下的最优化过程,能成功克服流量矩阵估计的病态特性,获得精确的估计值。仿真结果表明,该估计算法具有更高的估计精度和显著的性能改善。     

Artificial neural network application for modeling the rail rolling process

Rail rolling process is one of the most complicated hot rolling processes. Evaluating the effects of parametric values on this complex process is only possible through modeling. In this study, the production parameters of different types of rails in the rail rolling processes were modeled with an artificial neural network (ANN), and it was aimed to obtain optimum parameter values for a different type of rail. For this purpose, the data from the Rail and Profile Rolling Mill in Kardemir Iron & Steel Works Co. (Karabük, Turkey) were used. BD1, BD2, and Tandem are three main parts of the rolling mill, and in order to obtain the force values of the 49 kg/m rail in each pass for the BD1 and BD2 sections, the force and torque values for the Tandem section, parameter values of 60, 54, 46, and 33 kg/m type rails were used. Comparing the results obtained from the ANN model and the actual field data demonstrated that force and torque values were obtained with acceptable error rates. The results of the present study demonstrated that ANN is an effective and reliable method to acquire data required for producing a new rail, and concerning the rail production process, it provides a productive way for accurate and fast decision making.     

Comparison of artificial neural network and regression models for estimating software development effort

Estimating the amount of effort required for developing an information system is an important project management concern. In recent years, a number of studies have used neural networks in various stages of software development. This study compares the prediction performance of multilayer perceptron and radial basis function neural networks to that of regression analysis. The results of the study indicate that when a combined third generation and fourth generation languages data set were used, the neural network produced improved performance over conventional regression analysis in terms of mean absolute percentage error.     

BP网络和多元线性回归在产量预测中的应用

采用改进的BP神经网络算法和多元线性回归模型分别建立目标函数,并以油田产量预测为例计算验证。通过比较分析,BP网络模型克服了多元线性回归模型的局限性,检验误差为0.016 2,同时表明神经网络的非线性映射能力能够更好地反应多个自变量和因变量之间的复杂关系,具有较好的精确性和可行性。     

A comparative study of neural network and Box-Jenkins ARIMA modeling in time series prediction

This paper aims to investigate suitable time series models for repairable system failure analysis. A comparative study of the Box-Jenkins autoregressive integrated moving average (ARIMA) models and the artificial neural network models in predicting failures are carried out. The neural network architectures evaluated are the multi-layer feed-forward network and the recurrent network. Simulation results on a set of compressor failures showed that in modeling the stochastic nature of reliability data, both the ARIMA and the recurrent neural network (RNN) models outperform the feed-forward model; in terms of lower predictive errors and higher percentage of correct reversal detection. However, both models perform better with short term forecasting. The effect of varying the damped feedback weights in the recurrent net is also investigated and it was found that RNN at the optimal weighting factor gives satisfactory performances compared to the ARIMA model.     

A comparison of experimental designs in the development of a neural network simulation metamodel

In this case study, we investigate the effects of experimental design on the development of artificial neural networks as simulation metamodels. A simple, deterministic combat model developed within the paradigm of system dynamics provides the underlying simulation. The neural network metamodels are developed using six different experimental design approaches. These include a traditional full factorial design, a random sampling design, a central composite design, a modified Latin Hypercube design and designs supplemented with domain knowledge. The results from this case study show how much impact the experimental design chosen for the neural network training set can have on the predictive accuracy achieved by the metamodel. We compare the networks in terms of various performance measures. The neural network developed from the modified Latin Hypercube design supplemented with domain knowledge produces the best performance, outperforming networks developed from other designs of the same size.     

Learning experiments with genetic optimization of a generalized regression neural network

This paper reports a study unifying optimization by genetic algorithm with a generalized regression neural network. Experiments compare hill-climbing optimization with that of a genetic algorithm, both in conjunction with a generalized regression neural network. Controlled data with nine independent variables are used in combination with conjunctive and compensatory decision forms, having zero percent and 10 percent noise levels. Results consistently favor the GRNN unified with the genetic algorithm.     

Comparison of TSCS regression and neural network models for panel data forecasting: debt policy

Empirical studies of variations in debt ratios across firms have analyzed important determinants of capital structure using statistical models. Researchers, however, rarely employ nonlinear models to examine the determinants and make little effort to identify a superior prediction model among competing ones. This paper reviews the time-series cross-sectional (TSCS) regression and the predictive abilities of neural network (NN) utilizing panel data concerning debt ratio of high-tech industries in Taiwan. We built models with these two methods using the same set of measurements as determinants of debt ratio and compared the forecasting performance of five models, namely, three TSCS regression models and two NN models. Models built with neural network obtained the lowest mean square error and mean absolute error. These results reveal that the relationships between debt ratio and determinants are nonlinear and that NNs are more competent in modeling and forecasting the test panel data. We conclude that NN models can be used to solve panel data analysis and forecasting problems.     

Meta-modeling using generalized regression neural network and particle swarm optimization

Swarm intelligence (SI) and evolutionary computation (EC) algorithms are often used to solve various optimization problems. SI and EC algorithms generally require a large number of fitness function evaluations (i.e., higher computational requirements) to obtain quality solutions. This requirement becomes more challenging when optimization problems are associated with computationally expensive analyses and/or simulation tasks. To tackle this issue, meta-modeling has shown successful results in improving computational efficiency by approximating the fitness or constraint functions of these complex optimization problems. Meta-modeling approaches typically use polynomial regression, kriging, radial basis function network, and support vector machines. Less attention has been given to the generalized regression neural network approach, and yet, it offers several advantages. Specifically, the model construction process does not require iterations. Its only one parameter is known to be less sensitive and usually requires less effort in selecting an optimal parameter. We use generalized regression neural network in this paper to construct meta-models and to approximate the fitness function in particle swarm optimization. To assess the performance and quality of these solutions, the proposed meta-modeling approach is tested on ten benchmark functions. The results are promising in terms of the solution quality and computational efficiency, especially when compared against the results of particle swarm optimization without meta-modeling and several other meta-modeling methods in previously published literature.     

用于电磁兼容预测的自适应泛化回归神经网络

为了更好地对电磁兼容进行预测,提出一种自适应泛化回归神经网络（AGRNN）,与传统泛化回归神经网络（GRNN）区别在于：将光滑因子设为最小数据距离的1/2,将偏置设为光滑因子的倒数。对简单一维数据的测试表明,无论数据如何分布,AGRNN的拟合曲线均较GRNN更加接近样本点、且更平滑。以平行线间电磁耦合干扰为具体算例,证明AGRNN对训练数据与测试数据的预测优于改进BP算法,且网络不需要训练。     

神经网络结构搜索方法综述

如今,深度学习广泛地应用于生活、工作中的各个方面,给我们带来了极大的便利.在此背景下,需要设计针对不同任务的神经网络结构,满足不同的需求.但是,人工设计神经网络结构需要专业的知识,进行大量的实验.因此,神经网络结构搜索算法的研究显得极为重要.神经网络结构搜索(NAS)是自动深度学习(AutoDL)过程中的一个基本步骤,对深度学习的发展与应用有着重要的影响.早期,一些神经网络结构搜索算法虽然搜索到了性能优越的神经网络结构,但是需要大量的计算资源且搜索效率低下.因此,研究人员探索了多种设计神经网络结构的算法,也提出了许多减少计算资源、提高搜索效率的方法.本文首先简要介绍了神经网络结构的搜索空间,其次对神经网络结构搜索算法进行了全面的分类汇总、分析,主要包括随机搜索算法、进化算法、强化学习、基于梯度下降的方法、基于顺序模型的优化算法,再其次探索并总结了提高神经网络结构搜索效率的方法,最后探讨了目前神经网络结构搜索工作中存在的问题以及未来的研究方向.     

Optimization of optical lens-controlled scanning electron microscopic resolution using generalized regression neural network and genetic algorithm

A scanning electron microscope (SEM) is a sophisticated equipment employed for fine imaging of a variety of surfaces. In this study, prediction models of SEM were constructed by using a generalized regression neural network (GRNN) and genetic algorithm (GA). The SEM components examined include condenser lens 1 and 2 and objective lens (coarse and fine) referred to as CL1, CL2, OL-Coarse, and OL-Fine. For a systematic modeling of SEM resolution (R), a face-centered Box–Wilson experiment was conducted. Two sets of data were collected with or without the adjustment of magnification. Root-mean-squared prediction error of optimized GRNN models are GA 0.481 and 1.96×10^-12 for non-adjusted and adjusted data, respectively. The optimized models demonstrated a much improved prediction over statistical regression models. The optimized models were used to optimize parameters particularly under best tuned SEM environment. For the variations in CL2 and OL-Coarse, the highest R could be achieved at all conditions except a larger CL2 either at smaller or larger OL-Coarse. For the variations in CL1 and CL2, the highest R was obtained at all conditions but larger CL2 and smaller CL1.     

Predicting lightning outages of transmission lines using generalized regression neural network

Lightning is the major cause of transmission line outages, which can result in large area blackouts of power systems. One effective method to prevent catastrophic consequences is to predict lightning outages before they occur. The abundance of recorded lightning and lightning outage data in power system makes it possible to predict lightning outages of transmission lines. This paper proposes an artificially intelligent algorithm using general regression neural networks (GRNN) to predict lightning outages of transmission lines. First, the data that can be obtained from the operation and management system of a power company are analyzed, and the features that can be used as input parameters of GRNN are extracted. The prediction model based on GRNN is then built to perform lightning outage prediction. Finally, the effectiveness of the proposed method is validated by comparing it with (Back Propagation) BP and (Radial Basis Function) RBF neural networks using actual lightning data and lightning outage data. The simulation results show that the proposed method provides much better prediction performance.     

基于广义回归神经网络的传感器非线性误差校正

介绍了径向基函数网络的函数逼近原理和方法,提出了一种基于广义回归神经网络(GRNN)的传感器非线性误差校正方法。通过Matlab的Network Toolbox(神经网络工具箱),GRNN训练程序实现了输出特性曲线逼近。仿真分析表明:GRNN能够很好地满足传感器非线性拟合的要求,网络结构简单,收敛速度快。     

Unsupervised constrained neural network modeling of boundary value corneal model for eye surgery

In this article, a numerical computing technique is developed for solving the nonlinear second order corneal shape model (CSM) using feed-forward artificial neural networks, optimized with particle swarm optimization (PSO) and active-set algorithms (ASA). The design parameter is approved initially with PSO known as global search, while for further prompt local refinements ASA is used. The performance of the design structure is scrutinized by solving a number of variants of CSM. The typical Adams numerical results are used for comparison of the proposed results, which establish the worth of the scheme in terms of convergence and accuracy. For more satisfaction, the present results are also compared with radial basis function (RBF) results. Moreover, statistical analysis based on mean absolute deviation, Theil’s inequality coefficient and Nash Sutcliffe efficiency is presented     

卷积神经网络金相组织自动识别

为了降低人工分辨金相组织图像类别的误差率,提高分辨效率,采用卷积神经网络模型对金相组织图像进行自动辨识。对制备金相样块所得铁素体与马氏体两种金相组织图像进行分析,提出符合金相组织图像分布特征的预处理方案。通过采用图像尺寸归一化、灰度值归一化以及高斯平滑处理等方法,对原始金相组织图像进行预处理,建立金相组织图像数据集。针对建立的铁素体和马氏体金相组织图像数据集,提出了适合金相组织图像辨识的改进模型,分别记为LeNet-MetStr模型、AlexNet-MetStr模型和VGGNet-MetStr模型。对3种改进卷积神经网络进行模型训练及分析,结果表明VGGNet-MetStr模型对2种金相组织图像自动辨识具有更高的准确度。