Solving the stochastic support vector regression with probabilistic constraints by a high-performance neural network model

This paper offers a recurrent neural network to support vector machine (SVM) learning in stochastic support vector regression with probabilistic constraints. The SVM is first converted into an equivalent quadratic programming (QP) formulation in linear and nonlinear cases. An artificial neural network for SVM learning is then proposed. The presented neural network framework guarantees obtaining the optimal solution of the SVM problem. The existence and convergence of the trajectories of the network are studied. The Lyapunov stability for the considered neural network is also shown. The efficiency of the proposed method is shown by three illustrative examples.

     

A one-layer recurrent neural network with a discontinuous hard-limiting activation function for quadratic programming.

In this paper, a one-layer recurrent neural network with a discontinuous hard-limiting activation function is proposed for quadratic programming. This neural network is capable of solving a large class of quadratic programming problems. The state variables of the neural network are proven to be globally stable and the output variables are proven to be convergent to optimal solutions as long as the objective function is strictly convex on a set defined by the equality constraints. In addition, a sequential quadratic programming approach based on the proposed recurrent neural network is developed for general nonlinear programming. Simulation results on numerical examples and support vector machine (SVM) learning show the effectiveness and performance of the neural network.     

Unsupervised learning method for a support vector machine and its application to surface electromyogram recognition

The support vector machine (SVM) is known as one of the most influential and powerful tools for solving classification and regression problems, but the original SVM does not have an online learning technique. Therefore, many researchers have introduced online learning techniques to the SVM. In this article, we propose an unsupervised online learning method using a self-organized map for a SVM. Furthermore, the proposed method has a technique for the reconstruction of a SVM. We compare its performance with the original SVM, the supervised learning method for the SVM, and a neural network, and also test our proposed method on surface electromyogram recognition problems.     

基于支持向量机的机械故障智能分类研究

故障样本不足是制约故障诊断技术向智能化方向发展的主要原因之一，支持向量机（SVM)是一种基于统计学习理论(SLT)的机器学习算法，它能在训练样本很少的情况下达到很好的分类效果，从而为故障诊断技术向智能化发展提供了新的途径．本文介绍了支持向量机分类算法，以滚动轴承的故障分类为例，探讨了该算法在故障诊断领域中的应用，并与BP神经网络分类方法进行了对比研究，结果表明，SVM方法在少样本情况下的分类效果优于BP神经网络分类方法．     

基于SVM-MLP的乳腺癌预测

乳腺癌一直是影响女性健康最重要的问题之一,已经成为全球女性发病率最高的恶性肿瘤.近年来,利用机器学习和深度学习方法来诊断癌症已经成为发展较快的一个分支.通过使用逻辑回归模型(LR)、高斯核函数支持向量机(SVM)、前馈神经网络(MLP)对同一数据集进行预测,得出其中SVM迭代时间最短,前馈神经网络预测准确率最高.为了减...     

基于支持向量回归的唇动参数预测

支持向量机学习方法以结构风险最小化原则取代传统机器学习方法中的经验风险最小化原则，在有限样本的机器学习中显示出优异的性能．将这一新的统计学习方法应用到多媒体交互作用的研究中，用支持向量回归的方法由语音预测唇动参数．通过对语音的线性预测系数进行主分量分析，有效地压缩了声学特征参数的维数．结合交叉校验和最速下降优化方法，选择最佳的支持向量回归学习参数．在汉语0～9的任意数字串上对唇高参数的预测实验结果达到了均方误差0．0096，平均幅度误差7．2％及相关系数0．8的效果．这一结果优于一个文中优化过的人工神经网络所达到的性能，说明这一方法很有潜力．     

Drug design by machine learning: support vector machines for pharmaceutical data analysis.

We show that the support vector machine (SVM) classification algorithm, a recent development from the machine learning community, proves its potential for structure-activity relationship analysis. In a benchmark test, the SVM is compared to several machine learning techniques currently used in the field. The classification task involves predicting the inhibition of dihydrofolate reductase by pyrimidines, using data obtained from the UCI machine learning repository. Three artificial neural networks, a radial basis function network, and a C5.0 decision tree are all outperformed by the SVM. The SVM is significantly better than all of these, bar a manually capacity-controlled neural network, which takes considerably longer to train.     

Drug design by machine learning: support vector machines for pharmaceutical data analysis

We show that the support vector machine (SVM) classification algorithm, a recent development from the machine learning community, proves its potential for structure–activity relationship analysis. In a benchmark test, the SVM is compared to several machine learning techniques currently used in the field. The classification task involves predicting the inhibition of dihydrofolate reductase by pyrimidines, using data obtained from the UCI machine learning repository. Three artificial neural networks, a radial basis function network, and a C5.0 decision tree are all outperformed by the SVM. The SVM is significantly better than all of these, bar a manually capacity-controlled neural network, which takes considerably longer to train.     

Multi-fault classification based on support vector machine trained by chaos particle swarm optimization

A novel method of training support vector machine (SVM) by using chaos particle swarm optimization (CPSO) is proposed. A multi-fault classification model based on the SVM trained by CPSO is established and applied to the fault diagnosis of rotating machines. The results show that the method of training SVM using CPSO is feasible, the proposed fault classification model outperforms the neural network trained by chaos particle swarm optimization and least squares support vector machine, the precision and reliability of the fault classification results can meet the requirement of practical application.     

A Self-Organizing TS-Type Fuzzy Network With Support Vector Learning and its Application to Classification Problems

A self-organizing Takagi-Sugeno (TS)-type fuzzy network with support vector learning (SOTFN-SV) is proposed in this paper. The proposed SOTFN-SV is inspired by analysis of TS-type fuzzy systems and composite-kernel support vector machine (SVM). SOTFN-SV is a fuzzy system constructed by the hybridization of fuzzy clustering and SVM. The antecedent part of SOTFN-SV is generated via fuzzy clustering of the input data, and then SVM is used to tune the consequent part parameters to give the network better generalization performance. For demonstration, SOTFN-SV is applied to several classification problems, especially the skin color classification problem. In the skin color classification application, each color pixel is represented by hue and saturation (HS) color space. To represent color information by histogram as accurately as possible, a nonuniform partition of HS space is proposed. For comparison, SVMs and other fuzzy systems trained by SVM or neural networks are applied to the same classification problems. The advantages of SOTFN-SV are verified by comparisons with the results of these methods.     

Minimal Structure of Self-Organizing HCMAC Neural Network Classifier

The authors previously proposed a self-organizing Hierarchical Cerebellar Model Articulation Controller (HCMAC) neural network containing a hierarchical GCMAC neural network and a self-organizing input space module to solve high-dimensional pattern classification problems. This novel neural network exhibits fast learning, a low memory requirement, automatic memory parameter determination and highly accurate high-dimensional pattern classification. However, the original architecture needs to be hierarchically expanded using a full binary tree topology to solve pattern classification problems according to the dimension of the input vectors. This approach creates many redundant GCMAC nodes when the dimension of the input vectors in the pattern classification problem does not exactly match that in the self-organizing HCMAC neural network. These redundant GCMAC nodes waste memory units and degrade the learning performance of a self-organizing HCMAC neural network. Therefore, this study presents a minimal structure of self-organizing HCMAC (MHCMAC) neural network with the same dimension of input vectors as the pattern classification problem. Additionally, this study compares the learning performance of this novel learning structure with those of the BP neural network,support vector machine (SVM), and original self-organizing HCMAC neural network in terms of ten benchmark pattern classification data sets from the UCI machine learning repository. In particular, the experimental results reveal that the self-organizing MHCMAC neural network handles high-dimensional pattern classification problems better than the BP, SVM or the original self-organizing HCMAC neural network. Moreover, the proposed self-organizing MHCMAC neural network significantly reduces the memory requirement of the original self-organizing HCMAC neural network, and has a high training speed and higher pattern classification accuracy than the original self-organizing HCMAC neural network in most testing benchmark data sets. The experimental results also show that the MHCMAC neural network learns continuous function well and is suitable for Web page classification.     

基于SVM预测器的传感器故障诊断与信号恢复研究

支持向量机(SVM)是一种新兴的基于统计学习理论的机器学习方法.简要介绍了SVM回归原理,据此建立了基于SVM的时间预测器并用于传感器的故障诊断和信号恢复,阐述了具体的实现方法和步骤.仿真结果表明:SVM预测器有效地克服了神经网络的不足,能准确预测和跟踪传感器的输出信号,并在传感器发生故障后一定的时间段内能较精确的估计传感器的正常输出.     

Support vector machine with adaptive parameters in financial time series forecasting

A novel type of learning machine called support vector machine (SVM) has been receiving increasing interest in areas ranging from its original application in pattern recognition to other applications such as regression estimation due to its remarkable generalization performance. This paper deals with the application of SVM in financial time series forecasting. The feasibility of applying SVM in financial forecasting is first examined by comparing it with the multilayer back-propagation (BP) neural network and the regularized radial basis function (RBF) neural network. The variability in performance of SVM with respect to the free parameters is investigated experimentally. Adaptive parameters are then proposed by incorporating the nonstationarity of financial time series into SVM. Five real futures contracts collated from the Chicago Mercantile Market are used as the data sets. The simulation shows that among the three methods, SVM outperforms the BP neural network in financial forecasting, and there are comparable generalization performance between SVM and the regularized RBF neural network. Furthermore, the free parameters of SVM have a great effect on the generalization performance. SVM with adaptive parameters can both achieve higher generalization performance and use fewer support vectors than the standard SVM in financial forecasting.     

基于线性规划支持向量回归的混沌系统预测

支持向量机是一种基于统计学习理论的新颖的机器学习方法,该方法已经广泛用于解决分类与回归问题。标准的支持向量机算法需要解一个二次规划问题,当训练样本较多时,其运算速度一般很慢。为了提高运算速度,介绍了一种基于线性规划的支持向量回归算法,并由此提出几种新的回归模型,同时将它们应用到混沌时间序列预测中,并比较了它们的预测性能。在实际应用中,可以根据具体情况灵活地选择所需模型。     

Implementing automated diagnostic systems for breast cancer detection

This paper intends to an integrated view of implementing automated diagnostic systems for breast cancer detection. The major objective of the paper is to be a guide for the readers, who want to develop an automated decision support system for detection of breast cancer. Because of the importance of making the right decision, better classification procedures for breast cancer have been searched. The classification accuracies of different classifiers, namely multilayer perceptron neural network (MLPNN), combined neural network (CNN), probabilistic neural network (PNN), recurrent neural network (RNN) and support vector machine (SVM), which were trained on the attributes of each record in the Wisconsin breast cancer database, were compared. The purpose was to determine an optimum classification scheme with high diagnostic accuracy for this problem. This research demonstrated that the SVM achieved diagnostic accuracies which were higher than that of the other automated diagnostic systems.     

Implementing automated diagnostic systems for breast cancer detection

This paper intends to an integrated view of implementing automated diagnostic systems for breast cancer detection. The major objective of the paper is to be a guide for the readers, who want to develop an automated decision support system for detection of breast cancer. Because of the importance of making the right decision, better classification procedures for breast cancer have been searched. The classification accuracies of different classifiers, namely multilayer perceptron neural network (MLPNN), combined neural network (CNN), probabilistic neural network (PNN), recurrent neural network (RNN) and support vector machine (SVM), which were trained on the attributes of each record in the Wisconsin breast cancer database, were compared. The purpose was to determine an optimum classification scheme with high diagnostic accuracy for this problem. This research demonstrated that the SVM achieved diagnostic accuracies which were higher than that of the other automated diagnostic systems.     

A comparative study for content-based dynamic spam classification using four machine learning algorithms

The growth of email users has resulted in the dramatic increasing of the spam emails during the past few years. In this paper, four machine learning algorithms, which are Naïve Bayesian (NB), neural network (NN), support vector machine (SVM) and relevance vector machine (RVM), are proposed for spam classification. An empirical evaluation for them on the benchmark spam filtering corpora is presented. The experiments are performed based on different training set size and extracted feature size. Experimental results show that NN classifier is unsuitable for using alone as a spam rejection tool. Generally, the performances of SVM and RVM classifiers are obviously superior to NB classifier. Compared with SVM, RVM is shown to provide the similar classification result with less relevance vectors and much faster testing time. Despite the slower learning procedure, RVM is more suitable than SVM for spam classification in terms of the applications that require low complexity.     

支持向量机与RBF神经网络回归性能比较研究

支持向量机与RBF神经网络相比各有优缺点,通过对支持向量机与RBF神经网络的研究,从理论上分析了这两种学习机在回归预测原理上的异同,通过仿真实验对比了两者在测试集上的逼近能力及泛化能力。仿真结果表明,对于小样本集,支持向量机的逼近能力及泛化能力要优于RBF神经网络。对实际应用中回归模型的选择问题提出了建议。     

Fault diagnosis of power transformer based on support vector machine with genetic algorithm

Diagnosis of potential faults concealed inside power transformers is the key of ensuring stable electrical power supply to consumers. Support vector machine (SVM) is a new machine learning method based on the statistical learning theory, which is a powerful tool for solving the problem with small sampling, nonlinearity and high dimension. The selection of SVM parameters has an important influence on the classification accuracy of SVM. However, it is very difficult to select appropriate SVM parameters. In this study, support vector machine with genetic algorithm (SVMG) is applied to fault diagnosis of a power transformer, in which genetic algorithm (GA) is used to select appropriate free parameters of SVM. The experimental data from several electric power companies in China are used to illustrate the performance of the proposed SVMG model. The experimental results indicate that the SVMG method can achieve higher diagnostic accuracy than IEC three ratios, normal SVM classifier and artificial neural network.     

Deep kernel learning in core vector machines

In machine learning literature, deep learning methods have been moving toward greater heights by giving due importance in both data representation and classification methods. The recently developed multilayered arc-cosine kernel leverages the possibilities of extending deep learning features into the kernel machines. Even though this kernel has been widely used in conjunction with support vector machines (SVM) on small-size datasets, it does not seem to be a feasible solution for the modern real-world applications that involve very large size datasets. There are lot of avenues where the scalability aspects of deep kernel machines in handling large dataset need to be evaluated. In machine learning literature, core vector machine (CVM) is being used as a scaling up mechanism for traditional SVMs. In CVM, the quadratic programming problem involved in SVM is reformulated as an equivalent minimum enclosing ball problem and then solved by using a subset of training sample (Core Set) obtained by a faster \((1+\epsilon )\) approximation algorithm. This paper explores the possibilities of using principles of core vector machines as a scaling up mechanism for deep support vector machine with arc-cosine kernel. Experiments on different datasets show that the proposed system gives high classification accuracy with reasonable training time compared to traditional core vector machines, deep support vector machines with arc-cosine kernel and deep convolutional neural network.