梯度算法下RBF网的参数变化动态

分析神经网络学习过程中各参数的变化动态,对理解网络的动力学行为,改进网络的结构和性能等具有积极意义.本文讨论了用梯度算法优化误差平方和损失函数时RBF网隐节点参数的变化动态,即算法收敛后各隐节点参数的可能取值.主要结论包括:如果算法收敛后损失函数不为零,则各隐节点将位于样本输入的加权聚类中心;如果损失函数为零,则网络中的冗余隐节点将出现萎缩、衰减、外移或重合现象.进一步的试验发现,对结构过大的RBF网,冗余隐节点的萎缩、外移、衰减和重合是频繁出现的现象.     

一种有效的增量顺序学习模糊神经网络

利用误差下降率定义输入数据对系统输出的敏感性,并以此作为规则产生标准,提出一种有效增量顺序学习模糊神经网络。将修剪策略引入规则产生过程,因此该算法产生的模糊神经网络不需要进行修剪。通过仿真实验,本算法在达到与其他算法相当性能的情况下,能够获得更高的准确率和更简单的结构。     

Risk-sensitive loss functions for sparse multi-category classification problems

In this paper, we propose two risk-sensitive loss functions to solve the multi-category classification problems where the number of training samples is small and/or there is a high imbalance in the number of samples per class. Such problems are common in the bio-informatics/medical diagnosis areas. The most commonly used loss functions in the literature do not perform well in these problems as they minimize only the approximation error and neglect the estimation error due to imbalance in the training set. The proposed risk-sensitive loss functions minimize both the approximation and estimation error. We present an error analysis for the risk-sensitive loss functions along with other well known loss functions. Using a neural architecture, classifiers incorporating these risk-sensitive loss functions have been developed and their performance evaluated for two real world multi-class classification problems, viz., a satellite image classification problem and a micro-array gene expression based cancer classification problem. To study the effectiveness of the proposed loss functions, we have deliberately imbalanced the training samples in the satellite image problem and compared the performance of our neural classifiers with those developed using other well-known loss functions. The results indicate the superior performance of the neural classifier using the proposed loss functions both in terms of the overall and per class classification accuracy. Performance comparisons have also been carried out on a number of benchmark problems where the data is normal i.e., not sparse or imbalanced. Results indicate similar or better performance of the proposed loss functions compared to the well-known loss functions.     

基于径向基函数网络的浮游植物活体三维荧光光谱分类

将小波变换与神经网络相结合,对浮游植物活体的三维荧光光谱进行分类．首先利用小波变换对数据进行压缩,然后利用径向基函数（Radial Basis Function,RBF）神经网络对光谱曲线进行逼近,从而进行物种的识别,平均识别率高达95．8％．结果表明,该方法较传统的统计方法更方便、准确率更高．     

A gradient-based sequential radial basis function neural network modeling method

Radial basis function neural network (RBFNN) is widely used in nonlinear function approximation. One of the key issues in RBFNN modeling is to improve the approximation ability with samples as few as possible, so as to limit the network’s complexity. To solve this problem, a gradient-based sequential RBFNN modeling method is proposed. This method can utilize the gradient information of the present model to expand the sample set and refine the model sequentially, so as to improve the approximation accuracy effectively. Two mathematical examples and one practical problem are tested to verify the efficiency of this method. This article was originally presented in the fifth International Symposium on Neural Networks.     

基于径向基函数神经网络的网络流量识别模型

提出了一种基于径向基函数神经网络的网络流量识别方法。根据实际网络中的流量数据,建立了一个基于RBF神经网络的流量识别模型。先介绍了RBF神经网络的结构设计及学习算法,针对RBF神经网络在隐节点过多的情况下算法过于复杂的缺点,采用了优化的算法计算隐含层节点。仿真实验证明,该模型具有较好的准确率、低复杂度、高识别效果和良好的自适应性。     

A fully complex-valued radial basis function classifier for real-valued classification problems

In this paper, we investigate the decision making ability of a fully complex-valued radial basis function (FC-RBF) network in solving real-valued classification problems. The FC-RBF classifier is a single hidden layer fully complex-valued neural network with a nonlinear input layer, a nonlinear hidden layer, and a linear output layer. The neurons in the input layer of the classifier employ the phase encoded transformation to map the input features from the Real domain to the Complex domain. The neurons in the hidden layer employ a fully complex-valued Gaussian-like activation function of the type of hyperbolic secant (sech). The classification ability of the classifier is first studied analytically and it is shown that the decision boundaries of the FC-RBF classifier are orthogonal to each other. Then, the performance of the FC-RBF classifier is studied experimentally using a set of real-valued benchmark problems and also a real-world problem. The study clearly indicates the superior classification ability of the FC-RBF classifier.     

径向基函数神经网络的一种两级学习方法

建立RBF(radial basis function)神经网络模型关键在于确定网络隐中心向量、基宽度参数和隐节点数.为设计结构简单,且具有良好泛化性能径向基网络结构,本文提出了一种RBF网络的两级学习新设计方法.该方法在下级由正则化正交最小二乘法与D-最优试验设计结合算法自动构建结构节俭的RBF网络模型;在上级通过粒子群优化算法优选结合算法中影响网络泛化性能的3个学习参数,即基宽度参数、正则化系数和D-最优代价系数的最佳参数组合.仿真实例表明了该方法的有效性.     

Cooperative learning for radial basis function networks using particle swarm optimization

This paper presents a new evolutionary cooperative learning scheme, able to solve function approximation and classification problems with improved accuracy and generalization capabilities. The proposed method optimizes the construction of radial basis function (RBF) networks, based on a cooperative particle swarm optimization (CPSO) framework. It allows for using variable-width basis functions, which increase the flexibility of the produced models, while performing full network optimization by concurrently determining the rest of the RBF parameters, namely center locations, synaptic weights and network size. To avoid the excessive number of design variables, which hinders the optimization task, a compact representation scheme is introduced, using two distinct swarms. The first swarm applies the non-symmetric fuzzy means algorithm to calculate the network structure and RBF kernel center coordinates, while the second encodes the basis function widths by introducing a modified neighbor coverage heuristic. The two swarms work together in a cooperative way, by exchanging information towards discovering improved RBF network configurations, whereas a suitably tailored reset operation is incorporated to help avoid stagnation. The superiority of the proposed scheme is illustrated through implementation in a wide range of benchmark problems, and comparison with alternative approaches.     

Optimizing widths with PSO for center selection of Gaussian radial basis function networks

The radial basis function （RBF） centers play different roles in determining the classification capa- bility of a Gaussian radial basis function neural network （GRBFNN） and should hold different width values. However, it is very hard and time-consuming to optimize the centers and widths at the same time. In this paper, we introduce a new insight into this problem. We explore the impact of the definition of widths on the selection of the centers, propose an optimization algorithm of the RBF widths in order to select proper centers from the center candidate pool, and improve the classification performance of the GRBFNN. The design of the objective function of the optimization algorithm is based on the local mapping capability of each Gaussian RBF. Further, in the design of the objective function, we also handle the imbalanced problem which may occur even when different local regions have the same number of examples. Finally, the recursive orthogonal least square （ROLS） and genetic algorithm （GA）, which are usually adopted to optimize the RBF centers, are separately used to select the centers from the center candidates with the initialized widths, in order to testify the validity of our proposed width initialization strategy on the selection of centers. Our experimental results show that, compared with the heuristic width setting method, the width optimization strategy makes the selected cen- ters more appropriate, and improves the classification performance of the GRBFNN. Moreover, the GRBFNN constructed by our method can attain better classification performance than the RBF LS-SVM, which is a state-of-the-art classifier.     

Using radial basis function networks and significance testing to select effective siRNA sequences

Although short interfering RNA (siRNA) has been widely used for studying gene functions in mammalian cells, its gene silencing efficacy varies markedly and there are only a few consistencies among the recently reported design rules/guidelines for selecting siRNA sequences effective for mammalian genes. We propose a method for selecting effective siRNA target sequences by using a radial basis function (RBF) network and statistical significance analysis for a large number of known effective and ineffective siRNAs. The siRNA classification is first carried out by using the RBF network and then the preferred and unpreferred nucleotides for effective siRNAs at individual positions are chosen by significance testing. The gene degradation measure is defined as a score based on the preferred and unpreferred nucleotides. The effectiveness for the proposed method was confirmed by evaluating effective and ineffective siRNAs for the recently reported genes (15 genes, 196 sequences) and comparing the scores thus obtained with those obtained using other scoring methods. Since the score is closely correlated with the degree of gene degradation, it can easily be used for selecting high-potential siRNA candidates. The evaluation results indicate that the proposed method may be applicable for many other genes. It will therefore be useful for selecting siRNA sequences in mammalian genes.     

A review of genetic algorithms applied to training radial basis function networks

The problems associated with training feedforward artificial neural networks (ANNs) such as the multilayer perceptron (MLP) network and radial basis function (RBF) network have been well documented. The solutions to these problems have inspired a considerable amount of research, one particular area being the application of evolutionary search algorithms such as the genetic algorithm (GA). To date, the vast majority of GA solutions have been aimed at the MLP network. This paper begins with a brief overview of feedforward ANNs and GAs followed by a review of the current state of research in applying evolutionary techniques to training RBF networks.     

Efficacy of utilizing a hybrid algorithmic method in enhancing the functionality of multi-instance multi-label radial basis function neural networks

The facts show that multi-instance multi-label (MIML) learning plays a pivotal role in Artificial Intelligence studies. Evidently, the MIML learning introduces a framework in which data is described by a bag of instances associated with a set of labels. In this framework, the modeling of the connection is the challenging problem for MIML. The RBF neural network can explain the complex relations between the instances and labels in the MIMLRBF. The parameters estimation of the RBF network is a difficult task. In this paper, the computational convergence and the modeling accuracy of the RBF network has been improved. The present study aimed to investigate the impact of a novel hybrid algorithm consisting of Gases Brownian Motion optimization (GBMO) algorithm and the gradient based fast converging parameter estimation method on multi-instance multi-label learning. In the current study, a hybrid algorithm was developed to estimate the RBF neural network parameters (the weights, widths and centers of the hidden units) simultaneously. The algorithm uses the robustness of the GBMO to search the parameter space and the efficiency of the gradient. For this purpose, two real-world MIML tasks and a Corel dataset were utilized within a two-step experimental design. In the first step, the GBMO algorithm was used to determine the widths and centers of the network nodes. In the second step, for each molecule with fixed inputs and number of hidden nodes, the parameters were optimized by a structured nonlinear parameter optimization method (SNPOM). The findings demonstrated the superior performance of the hybrid algorithmic method. Additionally, the results for training and testing the dataset revealed that the hybrid method enhances RBF network learning more efficiently in comparison with other conventional RBF approaches. The results obtain better modeling accuracy than some other algorithms.     

A Direct Link Minimal Resource Allocation Network for Adaptive Noise Cancellation

This paper presents a modification to the Minimal Resource Allocation Network (MRAN) of Yingwei et al. by introducing direct links from inputs to output and investigates its performance for noise cancellation problems. MRAN has the same structure as a Radial Basis Function network but uses a sequential learning algorithm that adds and prunes hidden neurons as input data is received sequentially so as to produce a parsimonious network. Earlier work by Sun Yonghong et al. has demonstrated the capability of MRAN to produce a compact network with excellent noise reduction properties. In this paper the capability of the direct link Minimal Resource Allocation Network (DMRAN) is evaluated by comparing it with MRAN on several nonlinear adaptive noise cancellation problems. The direct link MRAN uses the same learning algorithm as MRAN but with the introduction of direct links we are able to realise even smaller networks than MRAN with better noise reduction properties.     

基于资源分配网络的小数据集并行集成学习方法

机器学习领域中,如何在小规模的训练数据集上获得一个具有稳定的高计算精度的算法模型,一直以来都是一个棘手而富有挑战的问题。从算法模型出发,提出了一种基于扩展卡尔曼滤波器的资源分配网络并行集成学习方法。该集成系统由多个带有扩展卡尔曼滤波器的资源分配网络（RANEKF）组成,并且每个RANEKF子网的输入由原始数据集中的输入经过随机权值的修正得到。通过和其他神经网络构成的集成学习算法的实验对比,发现提出的方法在小训练集上拥有更高的计算精度和稳定性。     

Output value-based initialization for radial basis function neural networks

The use of Radial Basis Function Neural Networks (RBFNNs) to solve functional approximation problems has been addressed many times in the literature. When designing an RBFNN to approximate a function, the first step consists of the initialization of the centers of the RBFs. This initialization task is very important because the rest of the steps are based on the positions of the centers. Many clustering techniques have been applied for this purpose achieving good results although they were constrained to the clustering problem. The next step of the design of an RBFNN, which is also very important, is the initialization of the radii for each RBF. There are few heuristics that are used for this problem and none of them use the information provided by the output of the function, but only the centers or the input vectors positions are considered. In this paper, a new algorithm to initialize the centers and the radii of an RBFNN is proposed. This algorithm uses the perspective of activation grades for each neuron, placing the centers according to the output of the target function. The radii are initialized using the center’s positions and their activation grades so the calculation of the radii also uses the information provided by the output of the target function. As the experiments show, the performance of the new algorithm outperforms other algorithms previously used for this problem.     

Speech recognition using pulse-coupled neural networks with a radial basis function

We have developed a novel pulse-coupled neural network (PCNN) for speech recognition. One of the advantages of the PCNN is in its biologically based neural dynamic structure using feedback connections. To recall the memorized pattern, a radial basis function (RBF) is incorporated into the proposed PCNN. Simulation results show that the PCNN with a RBF can be useful for phoneme recognition. This work was presented in part at the 7th International Symposium on Artificial Life and Robotics, Oita, Japan, January 16–18, 2002     

A new hybrid methodology for cooperative-coevolutionary optimization of radial basis function networks

This paper presents a new multiobjective cooperative–coevolutive hybrid algorithm for the design of a Radial Basis Function Network (RBFN). This approach codifies a population of Radial Basis Functions (RBFs) (hidden neurons), which evolve by means of cooperation and competition to obtain a compact and accurate RBFN. To evaluate the significance of a given RBF in the whole network, three factors have been proposed: the basis function’s contribution to the network’s output, the error produced in the basis function radius, and the overlapping among RBFs. To achieve an RBFN composed of RBFs with proper values for these quality factors our algorithm follows a multiobjective approach in the selection process. In the design process, a Fuzzy Rule Based System (FRBS) is used to determine the possibility of applying operators to a certain RBF. As the time required by our evolutionary algorithm to converge is relatively small, it is possible to get a further improvement of the solution found by using a local minimization algorithm (for example, the Levenberg–Marquardt method). In this paper the results of applying our methodology to function approximation and time series prediction problems are also presented and compared with other alternatives proposed in the bibliography.     

基于改进型径向基函数网络的功放非线性建模

针对功率放大器(PA)的非线性建模,提出了改进型径向基函数神经网络(RBFNN)模型。首先,在该模型的输入端加入延迟交叉项和输出反馈项,利用正交最小二乘法提取模型的权值以及隐含层的中心;然后,采用15MHz带宽的宽带码分多址（WCDMA）三载波信号对Doherty功放进行测试,其归一化均方误差(NMSE)可以达到-45dB;最后,通过逆F类功放对模型的普遍适用性进行验证。仿真结果表明,该模型能够更加真实地拟合功率放大器的特性。     

基于RBF神经网络曲线重构的算法研究

提出一种基于径向基(RBF)函数神经网络的曲线重构学习方法,即由描述物体轮廓特征的样本点作为RBF神经网络的学习样本,利用RBF神经网络强大的函教逼近能力对样本点进行学习和训练,从而仿真出包含这些样本点的原始曲线,同时对于曲线一些样本点缺少的情况下,仍然能构通过调整参数训练得到这些样本点的原始拟和曲线.实验表明,基于径向基(RBF)函数的神经网络具有很强的物体边界描述能力和缺损修复能力.