Three-dimensional medical image analysis of the heart by the revised GMDH-type neural network self-selecting optimum neural network architecture

In this study, a revised group method of data handling (GMDH)-type neural network algorithm which self-selects the optimum neural network architecture is applied to 3-dimensional medical image analysis of the heart. The GMDH-type neural network can automatically organize the neural network architecture by using the heuristic self-organization method, which is the basic theory of the GMDH algorism. The heuristic self-organization method is a kind of evolutionary computation method. In this revised GMDH-type neural network algorithm, the optimum neural network architecture was automatically organized using the polynomial and sigmoid function neurons. Furthermore, the structural parameters, such as the number of layers, the number of neurons in the hidden layers, and the useful input variables, are selected automatically in order to minimize the prediction error criterion, defined as the prediction sum of squares (PSS).     

Medical image diagnosis of lung cancer by a revised GMDH-type neural network using various kinds of neuron

A revised group method of data handling (GMDH)-type neural network algorithm using various kinds of neuron is applied to the medical image diagnosis of lung cancer. The optimum neural network architecture for medical image diagnosis is automatically organized using a revised GMDH-type neural network algorithm, and the regions of lung cancer are recognized and extracted accurately. In this revised GMDH-type neural network algorithm, polynomial-type and radial basis function (RBF)-type neurons are used for organizing the neural network architecture in order to fit the complexity of the nonlinear system.     

Deep multi-layered GMDH-type neural network using revised heuristic self-organization and its application to medical image diagnosis of liver cancer

In this study, the deep multi-layered group method of data handling (GMDH)-type neural network algorithm using revised heuristic self-organization method is proposed and applied to medical image diagnosis of liver cancer. The deep GMDH-type neural network can automatically organize the deep neural network architecture which has many hidden layers. The structural parameters such as the number of hidden layers, the number of neurons in hidden layers and useful input variables are automatically selected to minimize prediction error criterion defined as Akaike’s information criterion (AIC) or prediction sum of squares (PSS). The architecture of the deep neural network is automatically organized using the revised heuristic self-organization method which is a type of the evolutionary computation. This new neural network algorithm is applied to the medical image diagnosis of the liver cancer and the recognition results are compared with the conventional 3-layered sigmoid function neural network.     

Revised GMDH-type neural network algorithm self-selecting optimum neural network architecture

In this study, the revised group method of data handling (GMDH)-type neural network (NN) algorithm self-selecting the optimum neural network architecture is applied to the identification of a nonlinear system. In this algorithm, the optimum neural network architecture is automatically organized using two kinds of neuron architecture, such as the polynomial- and sigmoid function-type neurons. Many combinations of the input variables, in which the high order effects of the input variables are contained, are generated using the polynomial-type neurons, and useful combinations are selected using the prediction sum of squares (PSS) criterion. These calculations are iterated, and the multilayered architecture is organized. Furthermore, the structural parameters, such as the number of layers, the number of neurons in the hidden layers, and the useful input variables, are automatically selected in order to minimize the prediction error criterion defined as PSS.     

Medical image analysis of abdominal X-ray CT images by deep multi-layered GMDH-type neural network

In this study, a deep multi-layered group method of data handling (GMDH)-type neural network is applied to the medical image analysis of the abdominal X-ray computed tomography (CT) images. The deep neural network architecture which has many hidden layers are automatically organized using the deep multi-layered GMDH-type neural network algorithm so as to minimize the prediction error criterion defined as Akaike’s information criterion (AIC) or prediction sum of squares (PSS). The characteristics of the medical images are very complex and therefore the deep neural network architecture is very useful for the medical image diagnosis and medical image recognition. In this study, it is shown that this deep multi-layered GMDH-type neural network is useful for the medical image analysis of abdominal X-ray CT images.     

Feedback GMDH-type neural network algorithm and its application to medical image analysis of cancer of the liver

A revised group method of data handling (GMDH)-type neural network algorithm for medical image recognition is proposed, and is applied to medical image analysis of cancer of the liver. The revised GMDH-type neural network algorithm has a feedback loop and can identify the characteristics of the medical images accurately using feedback-loop calculations. In this algorithm, the polynomial type and the radial basis function (RBF)-type neurons are used for organizing the neural network architecture. The optimum neural network architecture fitting the complexity of the medical images is automatically organized so as to minimize the prediction error criterion, defined as the prediction sum of squares (PSS).     

Medical image diagnosis of kidney regions by deep feedback GMDH-type neural network using principal component-regression analysis

The deep feedback Group Method of Data Handling (GMDH)-type neural network is applied to the medical image recognition of kidney regions. In this algorithm, the principal component-regression analysis is used for the learning calculation of the neural network, and the accurate and stable predicted values are obtained. The neural network architecture is automatically organized so as to fit the complexity of the medical images using the prediction error criterion defined as Akaike’s Information Criterion (AIC) or Prediction Sum of Squares (PSS). The recognition results show that the deep feedback GMDH-type neural network algorithm is useful for the medical image recognition of kidney regions, because the optimum neural network architecture is automatically organized.     

Medical image diagnosis of liver cancer by feedback GMDH-type neural network using knowledge base

A revised group method of data handling (GMDH)-type neural network algorithm using knowledge base for medical image diagnosis, is proposed and applied to medical image diagnosis of liver cancer. In this algorithm, the knowledge base for medical image diagnosis is used for organizing the neural network architecture for medical image diagnosis. Furthermore, the revised GMDH-type neural network algorithm has a feedback loop and can identify the characteristics of the medical images accurately using feedback loop calculations. It is shown that the revised GMDH-type neural network is accurate and a useful method for the medical image diagnosis of the liver cancer.     

Deep feedback GMDH-type neural network and its application to medical image analysis of MRI brain images

The deep feedback group method of data handling (GMDH)-type neural network is applied to the medical image analysis of MRI brain images. In this algorithm, the complexity of the neural network is increased gradually using the feedback loop calculations. The deep neural network architecture is automatically organized so as to fit the complexity of the medical images using the prediction error criterion defined as Akaike’s information criterion (AIC) or prediction sum of squares (PSS). The recognition results show that the deep feedback GMDH-type neural network algorithm is useful for the medical image analysis of MRI brain images, because the optimum neural network architectures fitting the complexity of the medical images are automatically organized so as to minimize the prediction error criterion defined as AIC or PSS.     

Three-dimensional medical image recognition of cancer of the liver by a revised radial basis function (RBF) neural network algorithm

In this study, we propose a revised radial basis function (RBF) neural network algorithm and apply this algorithm to computer-aided diagnosis (CAD) of the liver. First, the revised RBF neural network algorithm is applied to recognition of the liver regions, and the recognition results are compared with those obtained using the conventional RBF neural network and the conventional multilayered neural network trained using the back-propagation algorithm. It is shown that the revised RBF neural network is accurate, and is a useful method because the parameters are automatically determined. Then, the revised RBF neural network is applied to CAD of the liver cancer called hepatocellular carcinoma (HCC).     

Multi-objective evolutionary optimization of polynomial neural networks for modelling and prediction of explosive cutting process

In this paper, evolutionary algorithms (EAs) are deployed for multi-objective Pareto optimal design of group method of data handling (GMDH)-type neural networks which have been used for modelling an explosive cutting process using some input–output experimental data. In this way, multi-objective EAs (non-dominated sorting genetic algorithm, NSGA-II) with a new diversity-preserving mechanism are used for Pareto optimization of such GMDH-type neural networks. The important conflicting objectives of GMDH-type neural networks that are considered in this work are, namely, training error (TE), prediction error (PE), and number of neurons (N) of such neural networks. Different pairs of theses objective functions are selected for 2-objective optimization processes. Therefore, optimal Pareto fronts of such models are obtained in each case which exhibit the trade-off between the corresponding pair of conflicting objectives and, thus, provide different non-dominated optimal choices of GMDH-type neural networks models for explosive cutting process. Moreover, all the three objectives are considered in a 3-objective optimization process, which consequently leads to some more non-dominated choices of GMDH-type models representing the trade-offs among the training error, prediction error, and number of neurons (complexity of network), simultaneously. The overlay graphs of these Pareto fronts also reveal that the 3-objective results include those of the 2-objective results and, thus, provide more optimal choices for the multi-objective design of GMDH-type neural networks in terms of minimum training error, minimum prediction error, and minimum complexity.     

一种基于混合粒子群优化算法的深度卷积神经网络架构搜索方法

神经架构搜索（neural architecture search,NAS）技术自动寻找神经网络中各层的最佳组合和连接方式,以及各种超参数的最佳分布。该方法从搜索空间生成若干不同的卷积神经网络（CNN）,使用混合粒子群优化（hybrid particle swarm optimization,HPSO）算法,将一定数目的神经网络个体视做一个群体,将每个网络个体在评价指标下的表现值视做适应度,在给定的世代数范围内,每个神经网络个体都学习自身的历史最佳适应度个体,和整个群体的最佳适应度个体,迭代改善自身的网络架构。实验结果表明,算法运行中出现的最优网络架构,在图像分类任务的多个基准数据集上,与手工设计的神经网络和以遗传算法为基础的NAS算法相比,在网络参数数量和准确率的平衡上取得了有竞争力的结果。     

Neural network training with optimal bounded ellipsoid algorithm

Compared to normal learning algorithms, for example backpropagation, the optimal bounded ellipsoid (OBE) algorithm has some better properties, such as faster convergence, since it has a similar structure as Kalman filter. OBE has some advantages over Kalman filter training, the noise is not required to be Guassian. In this paper OBE algorithm is applied in training the weights of the feedforward neural network for nonlinear system identification. Both hidden layers and output layers can be updated. From a dynamic system point of view, such training can be useful for all neural network applications requiring real-time updating of the weights. Two simulations give the effectiveness of the suggested algorithm.     

A classification technique based on radial basis function neural networks

In this paper, a new classification method is proposed based on the radial basis function (RBF) neural network architecture. The method is particularly useful for manufacturing processes, in cases where on-line sensors for classifying the product quality are not available. More specifically, the fuzzy means algorithm is employed on a set of training data, where the input data refer to variables that are measured on-line and the output data correspond to quality variables that are classified by human experts. The produced neural network model acts as an artificial sensor that is able to classify the product quality in real time. The proposed method is illustrated through an application to real data collected from a paper machine. The method produces successful results and outperforms a number of classifiers, which are based on the feedforward neural network (FNN) architecture.     

基于改进遗传算法的深度神经网络优化研究

深度前馈神经网络在分类和回归问题上得到了很好的应用,但网络性能极大程度上受到其结构和超参数影响.为了获得高性能的神经网络,首先对遗传算法的选择策略进行改进,之后利用该改进遗传算法,采用二进制编码与实数编码的混合编码策略对深度前馈神经网络层数、每层节点量以及学习率和权重进行优化.改进的选择策略,在最优保存策略的基础上从父代和子代合并的2n个个体中,以一定的概率选择部分适应值较差个体作为新父代,以增加种群多样性,避免陷入局部最优.同时引入dropout方法减少网络过拟合训练数据.使用Ring、Breast cancer、Twonorm、Heart、Blood、Ionosphere、Monk共7个数据集进行数值实验,并与其他相关文献中的算法比较,仿真结果表明,改进的遗传算法能搜索到较高性能的神经网络.     

用于股指预测的自适应训练及删剪算法

提出了利用基于自适应训练及删剪算法的抽头延迟神经网络模型对股指这一非线性时间序列进行预测。首先采用基于递归最小方差的自适应学习算法对网络模型进行学习训练，由于该算法的学习步长能够自行调整，初始参数少，所以收敛速度很快；再利用删剪算法对学习后的网络结构进行删剪，优化网络的拓扑结构，降低网络的计算复杂度，提高网络的泛化能力；然后对优化后的网络进行再学习，使优化后的网络具有最佳参数；最后利用优化后的网络对未来的股指（测试样本）进行预测。仿真实验表明，与删剪前的网络结构相比，优化后的网络结构不但降低了计算复杂度而且提高了预测精度，运算复杂度降低到原来的0.0556，预测均方误差达到8.7961e-5。     

Fuzzy logic in the gravitational search algorithm enhanced using fuzzy logic with dynamic alpha parameter value adaptation for the optimization of modular neural networks in echocardiogram recognition

In this paper the main goal is to find the optimal architecture of modular neural networks, which means finding out the optimal number of modules, layers and nodes of the neural network. The fuzzy gravitational search algorithm with dynamic parameter adaptation is used for optimizing the modular neural network in a particular pattern recognition application. The proposed method is applied to medical images in echocardiogram recognition. One of the most common methods for detection and analysis of diseases in the human body, by physicians and specialists, is the use of medical images. Simulation results of the proposed approach in echocardiogram recognition show the advantages of using the fuzzy gravitational search in the optimization of modular neural networks. In this case the proposed approach provides a very good 99.49% echocardiogram recognition rate.     

Design of hybrid differential evolution and group method of data handling networks for modeling and prediction

This paper proposes a hybrid modeling approach based on two familiar non-linear methods of mathematical modeling; the group method of data handling (GMDH) and differential evolution (DE) population-based algorithm. The proposed method constructs a GMDH self-organizing network model of a population of promising DE solutions. The new hybrid implementation is then applied to modeling tool wear in milling operations and also applied to two representative time series prediction problems of exchange rates of three international currencies and the well-studied Box-Jenkins gas furnace process data. The results of the proposed DE–GMDH approach are compared with the results obtained by the standard GMDH algorithm and its variants. Results presented show that the proposed DE–GMDH algorithm appears to perform better than the standard GMDH algorithm and the polynomial neural network (PNN) model for the tool wear problem. For the exchange rate problem, the results of the proposed DE–GMDH algorithm are competitive with all other approaches except in one case. For the Box-Jenkins gas furnace data, the experimental results clearly demonstrates that the proposed DE–GMDH-type network outperforms the existing models both in terms of better approximation capabilities as well as generalization abilities. Consequently, this self-organizing modeling approach may be useful in modeling advanced manufacturing systems where it is necessary to model tool wear during machining operations, and in time series applications such as in prediction of time series exchange rate and industrial gas furnace problems.     

一种快速逼近的神经网络构造性算法

选取合适的网络结构是前馈网络应用中首先遇到一个十分重要而又困难的问题。文章针对前馈网络的特点,提出了一种网络构造算法。该算法的主要特点是把二层网络的训练问题转化为求解线性方程组的最小二乘解,直接求得使误差最小的权值矩阵,并通过划分样本子集来创建子网,把子网输出映射到输出层,形成三层网络结构。由于避免了传统BP算法的梯度搜索过程,该算法能快速收敛。仿真结果表明,所提出的算法可快速、有效地逼近连续函数。     

基于改进遗传算法的RBF神经网络结构优化研究

针对RBF神经网络隐含层节点数过多导致网络结构复杂的问题,提出了一种基于改进遗传算法(IGA)的RBF神经网络优化算法。利用IGA优化基于正交最小二乘法的RBF神经网络结构,通过对隐含层输出矩阵的列向量进行全局寻优,从而设计出结构更优的基于IGA的RBF神经网络(IGA-RBF)。将IGA-RBF神经网络的学习算法应用于电子元器件贮存环境温湿度预测模型,与基于正交最小二乘法的RBF神经网络进行比较的结果表明:IGA-RBF神经网络设计出来的网络训练步数减少了44步,隐含层节点数减少了34个,且预测模型得到的温湿度误差较小,拟合精度大于0.95,具有更高的预测精度。