PARTICLE SWARM OPTIMIZATION-BASED FEATURE SELECTION AND PARAMETER OPTIMIZATION FOR POWER SYSTEM DISTURBANCES CLASSIFICATION

In many data mining applications that address classification problems, feature and model selection are considered as key tasks. The appropriate input features of the classifier are selected from a given set of possible features, and the structure parameters of the classifier are adapted with respect to these features and a given dataset. This paper describes the particle swarm optimization algorithm (PSO) that performs feature and model selection simultaneously for the probabilistic neural network (PNN) classifier for power system disturbances. The probabilistic neural network is one of the successful classifiers used to solve many classification problems. However, the computational effort and storage requirement of the PNN method will prohibitively increase as the number of patterns used in the training set increases. An important issue that has not been given enough attention is the selection of a “spread parameter,” also called a “smoothing parameter,” in the PNN classifier. PSO is a powerful meta-heuristic technique in the artificial intelligence field; therefore, this study proposes a PSO-based approach, called PSO-PNN, to specify the beneficial features and the value of spread parameter to enhance the performance of PNN. The experimental results indicate that the proposed PSO-based approach significantly improves the classification accuracy with the discriminating input features for PNN.     

Gap-based estimation: choosing the smoothing parameters for probabilistic and general regression neural networks

Probabilistic neural networks (PNN) and general regression neural networks (GRNN) represent knowledge by simple but interpretable models that approximate the optimal classifier or predictor in the sense of expected value of the accuracy. These models require the specification of an important smoothing parameter, which is usually chosen by cross-validation or clustering. In this article, we demonstrate the problems with the cross-validation and clustering approaches to specify the smoothing parameter, discuss the relationship between this parameter and some of the data statistics, and attempt to develop a fast approach to determine the optimal value of this parameter. Finally, through experimentation, we show that our approach, referred to as a gap-based estimation approach, is superior in speed to the compared approaches, including support vector machine, and yields good and stable accuracy.     

Multi-goal Q-learning of cooperative teams

This paper studies a multi-goal Q-learning algorithm of cooperative teams. Member of the cooperative teams is simulated by an agent. In the virtual cooperative team, agents adapt its knowledge according to cooperative principles. The multi-goal Q-learning algorithm is approached to the multiple learning goals. In the virtual team, agents learn what knowledge to adopt and how much to learn (choosing learning radius). The learning radius is interpreted in Section 3.1. Five basic experiments are manipulated proving the validity of the multi-goal Q-learning algorithm. It is found that the learning algorithm causes agents to converge to optimal actions, based on agents’ continually updated cognitive maps of how actions influence learning goals. It is also proved that the learning algorithm is beneficial to the multiple goals. Furthermore, the paper analyzes how sensitive the learning performance is affected by the parameter values of the learning algorithm.     

OPTIMAL FEATURE SET AND PARAMETER SELECTION FOR POWER QUALITY DATA MINING

The significance of detection and classification of power quality (PQ) events that disturb the voltage and/or current waveforms in electrical power distribution networks is well known. Consequently, in spite of a large number of research reports in this area, research on the selection of useful features from the existing feature set and the parameter selection for specific classifiers has thus far not been explored. The choice of a smoothing parameter for a probabilistic neural network classifier (PNN) in the training process, together with feature selection, will significantly impact the classification accuracy. In this work, a thorough analysis is carried out, using two wrapper-based optimization techniques—the genetic algorithm and simulated annealing—for identifying the ensemble of celebrated features obtained using discrete wavelet transform together with the smoothing parameter selection of the PNN classifier. As a result of these analyses, the proper smoothing parameter together with a more useful feature set from among a wider set of features for the PNN classifier is obtained with improved classification accuracy. Furthermore, the results show that the performance of simulated annealing is better than the genetic algorithm for feature selection and parameter optimization in Power Quality Data Mining.     

Obstacle avoidance of redundant manipulators using neural networks based reinforcement learning

This paper proposes a new approach for solving the problem of obstacle avoidance during manipulation tasks performed by redundant manipulators. The developed solution is based on a double neural network that uses Q-learning reinforcement technique. Q-learning has been applied in robotics for attaining obstacle free navigation or computing path planning problems. Most studies solve inverse kinematics and obstacle avoidance problems using variations of the classical Jacobian matrix approach, or by minimizing redundancy resolution of manipulators operating in known environments. Researchers who tried to use neural networks for solving inverse kinematics often dealt with only one obstacle present in the working field. This paper focuses on calculating inverse kinematics and obstacle avoidance for complex unknown environments, with multiple obstacles in the working field. Q-learning is used together with neural networks in order to plan and execute arm movements at each time instant. The algorithm developed for general redundant kinematic link chains has been tested on the particular case of PowerCube manipulator. Before implementing the solution on the real robot, the simulation was integrated in an immersive virtual environment for better movement analysis and safer testing. The study results show that the proposed approach has a good average speed and a satisfying target reaching success rate.     

Soft computing system for bank performance prediction

This paper presents a soft computing based bank performance prediction system. It is an ensemble system whose constituent models are a multi-layered feed forward neural network trained with backpropagation (MLFF-BP), a probabilistic neural network (PNN) and a radial basis function neural network (RBFN), support vector machine (SVM), classification and regression trees (CART) and a fuzzy rule based classifier. Further, principal component analysis (PCA) based hybrid neural networks, viz. PCA-MLFF-BP, PCA-PNN and PCA-RBF are also included as constituents of the ensemble. Moreover, GRNN and PNN were trained with a genetic algorithm to optimize the smoothing factors. Two ensembles (i) simple majority voting based and (ii) weightage based are implemented. This system predicts the performance of a bank in the coming financial year based on its previous 2-years’ financial data. Ten-fold cross-validation is performed in the training sessions and results are validated with an independent production set. It is demonstrated that the ensemble is able to yield lower Type I and Type II errors compared to its constituent models. Further, the ensemble also outperformed an earlier study [P.G. Swicegood, Predicting poor bank profitability: a comparison of neural network, discriminant analysis and professional human judgement, Ph.D. Thesis, Department of Finance, Florida State University, 1998] that used multivariate discriminant analysis (MDA), MLFF-BP and human judgment.     

State-chain sequential feedback reinforcement learning for path planning of autonomous mobile robots

This paper deals with a new approach based on Q-learning for solving the problem of mobile robot path planning in complex unknown static environments.As a computational approach to learning through interaction with the environment,reinforcement learning algorithms have been widely used for intelligent robot control,especially in the field of autonomous mobile robots.However,the learning process is slow and cumbersome.For practical applications,rapid rates of convergence are required.Aiming at the problem of slow convergence and long learning time for Q-learning based mobile robot path planning,a state-chain sequential feedback Q-learning algorithm is proposed for quickly searching for the optimal path of mobile robots in complex unknown static environments.The state chain is built during the searching process.After one action is chosen and the reward is received,the Q-values of the state-action pairs on the previously built state chain are sequentially updated with one-step Q-learning.With the increasing number of Q-values updated after one action,the number of actual steps for convergence decreases and thus,the learning time decreases,where a step is a state transition.Extensive simulations validate the efficiency of the newly proposed approach for mobile robot path planning in complex environments.The results show that the new approach has a high convergence speed and that the robot can find the collision-free optimal path in complex unknown static environments with much shorter time,compared with the one-step Q-learning algorithm and the Q(λ)-learning algorithm.     

Online classifier adaptation for cost-sensitive learning

In this paper, we propose the problem of online cost-sensitive classifier adaptation and the first algorithm to solve it. We assume that we have a base classifier for a cost-sensitive classification problem, but it is trained with respect to a cost setting different to the desired one. Moreover, we also have some training data samples streaming to the algorithm one by one. The problem is to adapt the given base classifier to the desired cost setting using the steaming training samples online. To solve this problem, we propose to learn a new classifier by adding an adaptation function to the base classifier, and update the adaptation function parameter according to the streaming data samples. Given an input data sample and the cost of misclassifying it, we update the adaptation function parameter by minimizing cost-weighted hinge loss and respecting previous learned parameter simultaneously. The proposed algorithm is compared to both online and off-line cost-sensitive algorithms on two cost-sensitive classification problems, and the experiments show that it not only outperforms them on classification performances, but also requires significantly less running time.

     

Model-based Predictive Control of Hybrid Systems: A Probabilistic Neural-network Approach to Real-time Control

This paper proposes an approach for reducing the computational complexity of a model-predictive-control strategy for discrete-time hybrid systems with discrete inputs only. Existing solutions are based on dynamic programming and multi-parametric programming approaches, while the one proposed in this paper is based on a modified version of performance-driven reachability analyses. The algorithm abstracts the behaviour of the hybrid system by building a ’tree of evolution’. The nodes of the tree represent the reachable states of a process, and the branches correspond to input combinations leading to designated states. A cost-function value is associated with each node and based on this value the exploration of the tree is driven. For any initial state, an input sequence is thus obtained, driving the system optimally over a finite horizon. According to the model predictive strategy, only the first input is actually applied to the system. The number of possible discrete input combinations is finite and the feasible set of the states of the system may be partitioned according to the optimization results. In the proposed approach, the partitioning is performed offline and a probabilistic neural network (PNN) is then trained by the set of points at the borders of the state-space partitions. The trained PNN is used as a system-state-based control-law classifier. Thus, the online computational effort is minimized and the control can be implemented in real time.     

Autonomous power management in mobile devices using dynamic frequency scaling and reinforcement learning for energy minimization

Embedded systems execute applications that execute hardware differently depending on the computation task, generating time-varying workloads. Energy minimization can be reached by using the low-power central processing unit (CPU) frequency for each workload. We propose an autonomous and online approach, capable of reducing energy consumption from adaptation to workload variations even in an unknown environment. In this approach, we improved the AEWMA algorithm into a new algorithm called AEWMA-MSE, adding new functionality to detect workload changes and demonstrating why it is better to use statistical analysis for real user cases in a mobile environment. Also, a new power model for mobile devices based on k-NN algorithm for regression was proposed and validated proving to have a better trade-off between execution time and precision than neural networks and linear regression-based models. AEWMA-MSE and the proposed power model are integrated into a novel algorithm for energy management based on reinforcement learning that suitably selects the appropriate CPU frequency based on workload predictions to minimize energy consumption. The proposed approach is validated through simulation by using real smartphone data from an ARM Cortex A7 processor used in a commercial smartphone. Our proposal proved to have an improvement in the Q-learning cost function and can effectively minimize the average energy consumption by 21% and up to 29% when compared to the already existing approaches.     

基于HHT与PNN的输电线路故障分类

针对电力系统输电线路一系列的短路故障,把希尔伯特黄变换(HHT)完成对暂态信号特征量的提取与概率神经网络(PNN)作为诊断故障分类器相结合的方式,作为对输电线路故障分类的方法.利用HHT能够充分反映局部暂态信号的特点,对集合经验模态分解(EEMD)后的故障信号进行小波阈值降噪,运用HHT进行重构,得到三相电流以及零序电流4组特征能量函数值,并作为4组特征量输入到经遗传算法优化过平滑因子的PNN中进行训练,最终得到分类器.经Matlab仿真实验显示,该方法能够有效优化信号波形并提高故障分类精度.     

A <Emphasis Type="Italic">Q</Emphasis>-learning-based swarm optimization algorithm for economic dispatch problem

In this paper, we treat optimization problems as a kind of reinforcement learning problems regarding an optimization procedure for searching an optimal solution as a reinforcement learning procedure for finding the best policy to maximize the expected rewards. This viewpoint motivated us to propose a Q-learning-based swarm optimization (QSO) algorithm. The proposed QSO algorithm is a population-based optimization algorithm which integrates the essential properties of Q-learning and particle swarm optimization. The optimization procedure of the QSO algorithm proceeds as each individual imitates the behavior of the global best one in the swarm. The best individual is chosen based on its accumulated performance instead of its momentary performance at each evaluation. Two data sets including a set of benchmark functions and a real-world problem—the economic dispatch (ED) problem for power systems—were used to test the performance of the proposed QSO algorithm. The simulation results on the benchmark functions show that the proposed QSO algorithm is comparable to or even outperforms several existing optimization algorithms. As for the ED problem, the proposed QSO algorithm has found solutions better than all previously found solutions.     

Neural networks for the recognition and pose estimation of 3D objects from a single 2D perspective view

In this paper we present a neural network (NN) based system for recognition and pose estimation of 3D objects from a single 2D perspective view. We develop an appearance based neural approach for this task. First the object is represented in a feature vector derived by a principal component network. Then a NN classifier trained with Resilient backpropagation (Rprop) algorithm is applied to identify it. Next pose parameters are obtained by four NN estimators trained on the same feature vector. Performance on recognition and pose estimation for real images under occlusions are shown. Comparative studies with two other approaches are carried out.     

利用PNN算法改进初始码书的GLA算法

针对改进广义Lloyd算法（GLA）对初始码书的敏感性，提出用成对最近邻（PNN）算法训练码书作为GLA算法的初始码书，再由GLA算法产生最终码书。PNN算法得到的码书作为GLA算法的初始码书，其码字能在整个输入矢量空间中很好地散开，尽可能地占据输入概率密度较大区域，提高最后码书的质量。仿真实验中，利用正态分布的随机数训练码书，恢复一段正态分布的随机数，采用Mahalanobis失真测度评价恢复数据的失真度。仿真结果表明，改进算法降低了GLA算法对初始码书的敏感性，提高了最终训练码书的质量，降低了恢复数据的失真。     

Modeling durations of syllables using neural networks

In this paper, we propose a neural network model for predicting the durations of syllables. A four layer feedforward neural network trained with backpropagation algorithm is used for modeling the duration knowledge of syllables. Broadcast news data in three Indian languages Hindi, Telugu and Tamil is used for this study. The input to the neural network consists of a set of features extracted from the text. These features correspond to phonological, positional and contextual information. The relative importance of the positional and contextual features is examined separately. For improving the accuracy of prediction, further processing is done on the predicted values of the durations. We also propose a two-stage duration model for improving the accuracy of prediction. From the studies we find that 85% of the syllable durations could be predicted from the models within 25% of the actual duration. The performance of the duration models is evaluated using objective measures such as average prediction error (μ), standard deviation (σ) and correlation coefficient (γ).     

基于SVM的柴油机故障诊断方法研究

介绍了基于支持向量机(SVM)的故障诊断方法的原理和算法。利用小波包分解提取信号的特征参数,将特征送入故障分类器中训练。实验结果表明,当数据样本较少时,相比神经网络而言,基于SVM的故障分类器仍能正确分类多种故障。这种诊断方法具有算法简单、故障分类能力强的优点。     

基于改进词嵌入表示和卷积神经网络的银行智能派单系统的研究

银行智能派单系统的实现和功能完善,对银行提升客户满意度、提高突发事件处理效率、降低人工处理成本等非常重要。针对现有的基于Word2vec和TextCNN模型的银行智能派单系统进行了改进,针对特征词权重表达性弱,特征词类别及位置区分性弱等问题,提出基于改进TF-IDF加权的Word2vec词嵌入表示和卷积神经网络结合的银行智能派单系统:首先利用Word2vec模型得到输入事件单的词嵌入向量;再针对经典TF-IDF方法不具备类别区分性、位置区分性,也没有考虑极端频率特征词代表性的情况,提出改进型TF-IDF算法,计算每个特征词的权重,得到基于改进TF-IDF加权的Word2vec词嵌入表示;最后在卷积神经网络模型中进行训练,通过迭代训练最终得到分类器,利用分类器可对输入事件单信息自动进行系统类别的判断。实验结果表明改进词嵌入表示的银行智能派单系统分类模型的宏查准率、宏查全率、准确率以及宏F1值都得到进一步的提高。     

On-line multi-stage sorting algorithm for agriculture products

This paper presents an on-line multi-stage sorting algorithm capable of adapting to different populations. The sorting algorithm selects on-line the most appropriate classifier and feature subsets for the incoming population. The sorting algorithm includes two levels, a low level for population detection and a high level for classifier selection which incorporates feature selection. Population detection is achieved by an on-line unsupervised clustering algorithm that analyzes product variability. The classifier selection uses n fuzzy kNN classifiers, each trained with different feature combinations that function as input to a fuzzy rule-based decision system. Re-training of the n fuzzy kNN classifiers occurs when the rule based system cannot assign an existing classifier with high confidence level. Classification results for synthetic and real world databases are presented.     

Determining significance of input neurons for probabilistic neural network by sensitivity analysis procedure

In classical feedforward neural networks such as multilayer perceptron, radial basis function network, or counter‐propagation network, the neurons in the input layer correspond to features of the training patterns. The number of these features may be large, and their meaningfulness can be various. Therefore, the selection of appropriate input neurons should be regarded. The aim of this paper is to present a complete step‐by‐step algorithm for determining the significance of particular input neurons of the probabilistic neural network (PNN). It is based on the sensitivity analysis procedure applied to a trained PNN. The proposed algorithm is utilized in the task of reduction of the input layer of the considered network, which is achieved by removing appropriately indicated features from the data set. For comparison purposes, the PNN's input neuron significance is established by using the ReliefF and variable importance procedures that provide the relevance of the input features in the data set. The performance of the reduced PNN is verified against a full structure network in classification problems using real benchmark data sets from an available machine learning repository. The achieved results are also referred to the ones attained by entropy‐based algorithms. The prediction ability expressed in terms of misclassifications is obtained by means of a 10‐fold cross‐validation procedure. Received outcomes point out interesting properties of the proposed algorithm. It is shown that the efficiency determined by all tested reduction methods is comparable.     

基于标签语义相似的动态多标签文本分类算法

针对标签随着时间变化的动态多标签文本分类问题,提出了一种基于标签语义相似的动态多标签文本分类算法。该算法在训练阶段,首先按照标签固定训练得到一个基于卷积神经网络的多标签文本分类器,然后以该分类器的倒数第二层的输出为文本的特征向量。由于该特征向量是在有标签训练得到的,因而相对于基于字符串即文本内容而言,该特征向量含有标签语义信息。在测试阶段,将测试文档输入训练阶段的多标签文本分类器获取相应的特征向量,然后计算相似性,同时乘以时间衰减因子修正,使得时间越近的文本具有较高的相似性。最后,采用最近邻算法分类。实验结果表明,该算法在处理动态多标签文本分类问题上具有较优的性能。