Risk-sensitive loss functions for sparse multi-category classification problems

In this paper, we propose two risk-sensitive loss functions to solve the multi-category classification problems where the number of training samples is small and/or there is a high imbalance in the number of samples per class. Such problems are common in the bio-informatics/medical diagnosis areas. The most commonly used loss functions in the literature do not perform well in these problems as they minimize only the approximation error and neglect the estimation error due to imbalance in the training set. The proposed risk-sensitive loss functions minimize both the approximation and estimation error. We present an error analysis for the risk-sensitive loss functions along with other well known loss functions. Using a neural architecture, classifiers incorporating these risk-sensitive loss functions have been developed and their performance evaluated for two real world multi-class classification problems, viz., a satellite image classification problem and a micro-array gene expression based cancer classification problem. To study the effectiveness of the proposed loss functions, we have deliberately imbalanced the training samples in the satellite image problem and compared the performance of our neural classifiers with those developed using other well-known loss functions. The results indicate the superior performance of the neural classifier using the proposed loss functions both in terms of the overall and per class classification accuracy. Performance comparisons have also been carried out on a number of benchmark problems where the data is normal i.e., not sparse or imbalanced. Results indicate similar or better performance of the proposed loss functions compared to the well-known loss functions.     

基于径向基函数网络的浮游植物活体三维荧光光谱分类

将小波变换与神经网络相结合,对浮游植物活体的三维荧光光谱进行分类．首先利用小波变换对数据进行压缩,然后利用径向基函数（Radial Basis Function,RBF）神经网络对光谱曲线进行逼近,从而进行物种的识别,平均识别率高达95．8％．结果表明,该方法较传统的统计方法更方便、准确率更高．     

A gradient-based sequential radial basis function neural network modeling method

Radial basis function neural network (RBFNN) is widely used in nonlinear function approximation. One of the key issues in RBFNN modeling is to improve the approximation ability with samples as few as possible, so as to limit the network’s complexity. To solve this problem, a gradient-based sequential RBFNN modeling method is proposed. This method can utilize the gradient information of the present model to expand the sample set and refine the model sequentially, so as to improve the approximation accuracy effectively. Two mathematical examples and one practical problem are tested to verify the efficiency of this method. This article was originally presented in the fifth International Symposium on Neural Networks.     

基于径向基函数神经网络的网络流量识别模型

提出了一种基于径向基函数神经网络的网络流量识别方法。根据实际网络中的流量数据,建立了一个基于RBF神经网络的流量识别模型。先介绍了RBF神经网络的结构设计及学习算法,针对RBF神经网络在隐节点过多的情况下算法过于复杂的缺点,采用了优化的算法计算隐含层节点。仿真实验证明,该模型具有较好的准确率、低复杂度、高识别效果和良好的自适应性。     

A fully complex-valued radial basis function classifier for real-valued classification problems

In this paper, we investigate the decision making ability of a fully complex-valued radial basis function (FC-RBF) network in solving real-valued classification problems. The FC-RBF classifier is a single hidden layer fully complex-valued neural network with a nonlinear input layer, a nonlinear hidden layer, and a linear output layer. The neurons in the input layer of the classifier employ the phase encoded transformation to map the input features from the Real domain to the Complex domain. The neurons in the hidden layer employ a fully complex-valued Gaussian-like activation function of the type of hyperbolic secant (sech). The classification ability of the classifier is first studied analytically and it is shown that the decision boundaries of the FC-RBF classifier are orthogonal to each other. Then, the performance of the FC-RBF classifier is studied experimentally using a set of real-valued benchmark problems and also a real-world problem. The study clearly indicates the superior classification ability of the FC-RBF classifier.     

Optimizing widths with PSO for center selection of Gaussian radial basis function networks

The radial basis function （RBF） centers play different roles in determining the classification capa- bility of a Gaussian radial basis function neural network （GRBFNN） and should hold different width values. However, it is very hard and time-consuming to optimize the centers and widths at the same time. In this paper, we introduce a new insight into this problem. We explore the impact of the definition of widths on the selection of the centers, propose an optimization algorithm of the RBF widths in order to select proper centers from the center candidate pool, and improve the classification performance of the GRBFNN. The design of the objective function of the optimization algorithm is based on the local mapping capability of each Gaussian RBF. Further, in the design of the objective function, we also handle the imbalanced problem which may occur even when different local regions have the same number of examples. Finally, the recursive orthogonal least square （ROLS） and genetic algorithm （GA）, which are usually adopted to optimize the RBF centers, are separately used to select the centers from the center candidates with the initialized widths, in order to testify the validity of our proposed width initialization strategy on the selection of centers. Our experimental results show that, compared with the heuristic width setting method, the width optimization strategy makes the selected cen- ters more appropriate, and improves the classification performance of the GRBFNN. Moreover, the GRBFNN constructed by our method can attain better classification performance than the RBF LS-SVM, which is a state-of-the-art classifier.     

Using radial basis function networks and significance testing to select effective siRNA sequences

Although short interfering RNA (siRNA) has been widely used for studying gene functions in mammalian cells, its gene silencing efficacy varies markedly and there are only a few consistencies among the recently reported design rules/guidelines for selecting siRNA sequences effective for mammalian genes. We propose a method for selecting effective siRNA target sequences by using a radial basis function (RBF) network and statistical significance analysis for a large number of known effective and ineffective siRNAs. The siRNA classification is first carried out by using the RBF network and then the preferred and unpreferred nucleotides for effective siRNAs at individual positions are chosen by significance testing. The gene degradation measure is defined as a score based on the preferred and unpreferred nucleotides. The effectiveness for the proposed method was confirmed by evaluating effective and ineffective siRNAs for the recently reported genes (15 genes, 196 sequences) and comparing the scores thus obtained with those obtained using other scoring methods. Since the score is closely correlated with the degree of gene degradation, it can easily be used for selecting high-potential siRNA candidates. The evaluation results indicate that the proposed method may be applicable for many other genes. It will therefore be useful for selecting siRNA sequences in mammalian genes.     

A review of genetic algorithms applied to training radial basis function networks

The problems associated with training feedforward artificial neural networks (ANNs) such as the multilayer perceptron (MLP) network and radial basis function (RBF) network have been well documented. The solutions to these problems have inspired a considerable amount of research, one particular area being the application of evolutionary search algorithms such as the genetic algorithm (GA). To date, the vast majority of GA solutions have been aimed at the MLP network. This paper begins with a brief overview of feedforward ANNs and GAs followed by a review of the current state of research in applying evolutionary techniques to training RBF networks.     

Efficacy of utilizing a hybrid algorithmic method in enhancing the functionality of multi-instance multi-label radial basis function neural networks

The facts show that multi-instance multi-label (MIML) learning plays a pivotal role in Artificial Intelligence studies. Evidently, the MIML learning introduces a framework in which data is described by a bag of instances associated with a set of labels. In this framework, the modeling of the connection is the challenging problem for MIML. The RBF neural network can explain the complex relations between the instances and labels in the MIMLRBF. The parameters estimation of the RBF network is a difficult task. In this paper, the computational convergence and the modeling accuracy of the RBF network has been improved. The present study aimed to investigate the impact of a novel hybrid algorithm consisting of Gases Brownian Motion optimization (GBMO) algorithm and the gradient based fast converging parameter estimation method on multi-instance multi-label learning. In the current study, a hybrid algorithm was developed to estimate the RBF neural network parameters (the weights, widths and centers of the hidden units) simultaneously. The algorithm uses the robustness of the GBMO to search the parameter space and the efficiency of the gradient. For this purpose, two real-world MIML tasks and a Corel dataset were utilized within a two-step experimental design. In the first step, the GBMO algorithm was used to determine the widths and centers of the network nodes. In the second step, for each molecule with fixed inputs and number of hidden nodes, the parameters were optimized by a structured nonlinear parameter optimization method (SNPOM). The findings demonstrated the superior performance of the hybrid algorithmic method. Additionally, the results for training and testing the dataset revealed that the hybrid method enhances RBF network learning more efficiently in comparison with other conventional RBF approaches. The results obtain better modeling accuracy than some other algorithms.     

Output value-based initialization for radial basis function neural networks

The use of Radial Basis Function Neural Networks (RBFNNs) to solve functional approximation problems has been addressed many times in the literature. When designing an RBFNN to approximate a function, the first step consists of the initialization of the centers of the RBFs. This initialization task is very important because the rest of the steps are based on the positions of the centers. Many clustering techniques have been applied for this purpose achieving good results although they were constrained to the clustering problem. The next step of the design of an RBFNN, which is also very important, is the initialization of the radii for each RBF. There are few heuristics that are used for this problem and none of them use the information provided by the output of the function, but only the centers or the input vectors positions are considered. In this paper, a new algorithm to initialize the centers and the radii of an RBFNN is proposed. This algorithm uses the perspective of activation grades for each neuron, placing the centers according to the output of the target function. The radii are initialized using the center’s positions and their activation grades so the calculation of the radii also uses the information provided by the output of the target function. As the experiments show, the performance of the new algorithm outperforms other algorithms previously used for this problem.     

Speech recognition using pulse-coupled neural networks with a radial basis function

We have developed a novel pulse-coupled neural network (PCNN) for speech recognition. One of the advantages of the PCNN is in its biologically based neural dynamic structure using feedback connections. To recall the memorized pattern, a radial basis function (RBF) is incorporated into the proposed PCNN. Simulation results show that the PCNN with a RBF can be useful for phoneme recognition. This work was presented in part at the 7th International Symposium on Artificial Life and Robotics, Oita, Japan, January 16–18, 2002     

A new hybrid methodology for cooperative-coevolutionary optimization of radial basis function networks

This paper presents a new multiobjective cooperative–coevolutive hybrid algorithm for the design of a Radial Basis Function Network (RBFN). This approach codifies a population of Radial Basis Functions (RBFs) (hidden neurons), which evolve by means of cooperation and competition to obtain a compact and accurate RBFN. To evaluate the significance of a given RBF in the whole network, three factors have been proposed: the basis function’s contribution to the network’s output, the error produced in the basis function radius, and the overlapping among RBFs. To achieve an RBFN composed of RBFs with proper values for these quality factors our algorithm follows a multiobjective approach in the selection process. In the design process, a Fuzzy Rule Based System (FRBS) is used to determine the possibility of applying operators to a certain RBF. As the time required by our evolutionary algorithm to converge is relatively small, it is possible to get a further improvement of the solution found by using a local minimization algorithm (for example, the Levenberg–Marquardt method). In this paper the results of applying our methodology to function approximation and time series prediction problems are also presented and compared with other alternatives proposed in the bibliography.     

基于RBF神经网络曲线重构的算法研究

提出一种基于径向基(RBF)函数神经网络的曲线重构学习方法,即由描述物体轮廓特征的样本点作为RBF神经网络的学习样本,利用RBF神经网络强大的函教逼近能力对样本点进行学习和训练,从而仿真出包含这些样本点的原始曲线,同时对于曲线一些样本点缺少的情况下,仍然能构通过调整参数训练得到这些样本点的原始拟和曲线.实验表明,基于径向基(RBF)函数的神经网络具有很强的物体边界描述能力和缺损修复能力.     

新型广义径向基函数神经网络结构研究

提出了一种新型的广义径向基函数(RBF)神经网络,并研究了该网络的学习方法.不同于传统三层结构的RBF网络,广义RBF网络增加了基函数输出加权层,并在输出层采用超曲面去逼近任意的非线性曲面.实例仿真结果表明,与传统的RBF网络相比,该网络具有良好的逼近性能,收敛速度快,可逼近任意多变量非线性函数.     

Probabilistic self-organizing map and radial basis function networks

We propose in this paper a new learning algorithm probabilistic self-organizing map (PRSOM) using a probabilistic formalism for topological maps. This algorithm approximates the density distribution of the input set with a mixture of normal distributions. The unsupervised learning is based on the dynamic clusters principle and optimizes the likelihood function. A supervised version of this algorithm based on radial basis functions (RBF) is proposed. In order to validate the theoretical approach, we achieve regression tasks on simulated and real data using the PRSOM algorithm. Moreover, our results are compared with normalized Gaussian basis functions (NGBF) algorithm.     

Error-driven active learning in growing radial basis function networks for early robot learning

In this paper, we describe a new error-driven active learning approach to self-growing radial basis function networks for early robot learning. There are several mappings that need to be set up for an autonomous robot system for sensorimotor coordination and transformation of sensory information from one modality to another, and these mappings are usually highly nonlinear. Traditional passive learning approaches usually cause both large mapping errors and nonuniform mapping error distribution compared to active learning. A hierarchical clustering technique is introduced to group large mapping errors and these error clusters drive the system to actively explore details of these clusters. Higher level local growing radial basis function subnetworks are used to approximate the residual errors from previous mapping levels. Plastic radial basis function networks construct the substrate of the learning system and a simplified node-decoupled extended Kalman filter algorithm is presented to train these radial basis function networks. Experimental results are given to compare the performance among active learning with hierarchical adaptive RBF networks, passive learning with adaptive RBF networks and hierarchical mixtures of experts, as well as their robustness under noise conditions.     

Consideration effect of uncertainty in power system reliability indices using radial basis function network and fuzzy logic theory

Reliability assessment of composite power systems is a critical and important part of power investigations especially in the market-driven environments. Therefore, the reliability indices as criteria for the comparison of the reliability of the power systems should be evaluated precisely and carefully. Because of the nonlinear behavior of the systems as the effect of different parameters like weather conditions, load pattern changes and some others, reliability indices always contain much uncertainty. In this paper a neuro-fuzzy based method is proposed to reduce the degree of the uncertainty in the reliability indices and therefore to evaluate the reliability of the composite power systems precisely. Fuzzy logic theory makes it possible to make use of the human experts knowledge in the reliability evaluations. Also by the use of RBFNN and its powerful characteristic to learn any nonlinear mapping between two states it would be possible to evaluate the reliability indices for every short time interval needed so that reliability evaluation in real time would be achievable and feasible.In this paper the RBFNN is trained by the training patterns that are achieved by the use of fuzzy logic theory, then the results are examined on a standard Reliability Test System (RTS-96).     

A forecasting system for car fuel consumption using a radial basis function neural network

A predictive system for car fuel consumption using a radial basis function (RBF) neural network is proposed in this paper. The proposed work consists of three parts: information acquisition, fuel consumption forecasting algorithm and performance evaluation. Although there are many factors affecting the fuel consumption of a car in a practical drive procedure, in the present system the relevant factors for fuel consumption are simply decided as make of car, engine style, weight of car, vehicle type and transmission system type which are used as input information for the neural network training and fuel consumption forecasting procedure. In fuel consumption forecasting, to verify the effect of the proposed RBF neural network predictive system, an artificial neural network with a back-propagation (BP) neural network is compared with an RBF neural network for car fuel consumption prediction. The prediction results demonstrated the proposed system using the neural network is effective and the performance is satisfactory in terms of fuel consumption prediction.     

Time series prediction using evolving radial basis function networks with new encoding scheme

This paper presents a new encoding scheme for training radial basis function (RBF) networks by genetic algorithms (GAs). In general, it is very difficult to select the proper input variables and the exact number of nodes before training an RBF network. In the proposed encoding scheme, both the architecture (numbers and selections of nodes and inputs) and the parameters (centres and widths) of the RBF networks are represented in one chromosome and evolved simultaneously by GAs so that the selection of nodes and inputs can be achieved automatically. The performance and effectiveness of the presented approach are evaluated using two benchmark time series prediction examples and one practical application example, and are then compared with other existing methods. It is shown by the simulation tests that the developed evolving RBF networks are able to predict the time series accurately with the automatically selected nodes and inputs.