Chaotic antlion algorithm for parameter optimization of support vector machine

Support Vector Machine (SVM) is one of the well-known classifiers. SVM parameters such as kernel parameters and penalty parameter (C) significantly influence the classification accuracy. In this paper, a novel Chaotic Antlion Optimization (CALO) algorithm has been proposed to optimize the parameters of SVM classifier, so that the classification error can be reduced. To evaluate the proposed algorithm (CALO-SVM), the experiment adopted six standard datasets which are obtained from UCI machine learning data repository. For verification, the results of the CALO-SVM algorithm are compared with grid search, which is a conventional method of searching parameter values, standard Ant Lion Optimization (ALO) SVM, and three well-known optimization algorithms: Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and Social Emotional Optimization Algorithm (SEOA). The experimental results proved that the proposed algorithm is capable of finding the optimal values of the SVM parameters and avoids the local optima problem. The results also demonstrated lower classification error rates compared with GA, PSO, and SEOA algorithms.     

基于对支持向量机的多类分类算法在入侵检测中的应用

针对基于传统支持向量机(SVM)的多类分类算法在处理大规模数据时训练速度上存在的弱势,提出了一种基于对支持向量机(TWSVM)的多类分类算法。该算法结合二叉树SVM多类分类思想,通过在二叉树节点处构造基于TWSVM的分类器来达到分类目的。为减少二叉树SVM的误差累积,算法分类前首先通过聚类算法得到各类的聚类中心,通过比较各聚类中心之间的距离来衡量样本的差异以决定二叉树节点处类别的分离顺序,最后将算法用于网络入侵检测。实验结果表明,该算法不仅保持了较高的检测精度,在训练速度上还表现了一定优势,尤其在处理稍大规模数据时,这种优势更为明显,是传统二叉树SVM多类分类算法训练速度的近两倍,为入侵检测领域大规模数据处理提供了有效参考价值。     

面向超大数据集的SVM近似训练算法

标准SVM学习算法运行所需的时间和空间复杂度分别为O(l~3)和O(l~2),l为训练样本的数量,因此不适用于对超大数据集进行训练.提出一种基于近似解的SVM训练算法:Approximate Vector Machine(AVM).AVM采用增量学习的策略来寻找近似最优分类超平面,并且在迭代过程中采用热启动及抽样技巧来加快训练速度.理论分析表明,该算法的计算复杂度与训练样本的数量无关,因此具有良好的时间与空间扩展性.在超大数据集上的实验结果表明,该算法在极大提高训练速度的同时,仍然保持了原始分类器的泛化性能,并且训练完毕具有较少的支持向量,因此结果分类器具有更快的分类速度.     

An up-to-date comparison of state-of-the-art classification algorithms

Current benchmark reports of classification algorithms generally concern common classifiers and their variants but do not include many algorithms that have been introduced in recent years. Moreover, important properties such as the dependency on number of classes and features and CPU running time are typically not examined. In this paper, we carry out a comparative empirical study on both established classifiers and more recently proposed ones on 71 data sets originating from different domains, publicly available at UCI and KEEL repositories. The list of 11 algorithms studied includes Extreme Learning Machine (ELM), Sparse Representation based Classification (SRC), and Deep Learning (DL), which have not been thoroughly investigated in existing comparative studies. It is found that Stochastic Gradient Boosting Trees (GBDT) matches or exceeds the prediction performance of Support Vector Machines (SVM) and Random Forests (RF), while being the fastest algorithm in terms of prediction efficiency. ELM also yields good accuracy results, ranking in the top-5, alongside GBDT, RF, SVM, and C4.5 but this performance varies widely across all data sets. Unsurprisingly, top accuracy performers have average or slow training time efficiency. DL is the worst performer in terms of accuracy but second fastest in prediction efficiency. SRC shows good accuracy performance but it is the slowest classifier in both training and testing.     

基于随机森林的跌倒检测算法

针对现有跌倒检测算法由于缺乏真实老人跌倒样本以及使用年轻人仿真跌倒样本规模较小导致的过拟合和适应性不足等问题,提出了基于随机森林的跌倒检测算法.该算法采用滑动窗口机制,对窗口内的加速度数据进行时间域和变换域处理,提取时间域和变换域特征参数后,在所有样本集中进行有放回的Bootstrap随机抽样和属性随机选择,构建多个基于最佳属性分割的支持向量机(SVM)基本分类器.在线跌倒检测阶段,对多个SVM基本分类器的分类结果采用少数服从多数的原则,给出最终判定结果.实验表明,随机森林跌倒检测算法可获得95.2%的准确率、90.6%的敏感度和93.5%的特异性,明显优于基于SVM和反向传播(BP)神经网络跌倒检测算法,反映出随机森林跌倒检测算法能更准确地检测跌倒行为,具有较强的泛化能力和鲁棒性.     

Hierarchical linear support vector machine

The increasing size and dimensionality of real-world datasets make it necessary to design efficient algorithms not only in the training process but also in the prediction phase. In applications such as credit card fraud detection, the classifier needs to predict an event in 10 ms at most. In these environments the speed of the prediction constraints heavily outweighs the training costs. We propose a new classification method, called a Hierarchical Linear Support Vector Machine (H-LSVM), based on the construction of an oblique decision tree in which the node split is obtained as a Linear Support Vector Machine. Although other methods have been proposed to break the data space down in subregions to speed up Support Vector Machines, the H-LSVM algorithm represents a very simple and efficient model in training but mainly in prediction for large-scale datasets. Only a few hyperplanes need to be evaluated in the prediction step, no kernel computation is required and the tree structure makes parallelization possible. In experiments with medium and large datasets, the H-LSVM reduces the prediction cost considerably while achieving classification results closer to the non-linear SVM than that of the linear case.     

基于OC-SVM的大型数据集分类方法

支持向量机是最有效的分类技术之一,具有很高的分类精度和良好的泛化能力,但其应用于大型数据集时的训练过程还是非常复杂。对此提出了一种基于单类支持向量机的分类方法。采用随机选择算法来约简训练集,以达到提高训练速度的目的;同时,通过恢复超球体交集中样本在原始数据中的邻域来保证支持向量机的分类精度。实验证明,该方法能在较大程度上减小计算复杂度,从而提高大型数据集中的训练速度。     

面向概念漂移问题的渐进多核学习方法

针对概念漂移问题,构建数据特性随时间发生渐进变化特点的分类学习模型,提出一种基于渐进支持向量机（G-SVM）的渐进多核学习方法（G-MKL）。该方法采用支持向量机（SVM）为基本分类器,进行多区间上的子分类器耦合训练,并通过约束子分类器增量方式使模型适应数据渐进变化特性,最终将多个核函数以线性组合方式融入SVM求解框架。该方法综合发挥了各个核函数的优势,大大提高了模型适应性和有效性。在具有渐变特性的模拟数据集和真实数据集上将所提算法与多种经典算法进行了对比,验证了所提算法在处理非静态数据问题的有效性。     

最小类内方差支持向量引导的字典学习

支持向量引导的字典学习算法依据大间隔分类原则,仅考虑每类编码向量边界条件建立决策超平面,未利用数据的分布信息,在一定程度上限制了模型的泛化能力.为解决该问题,提出最小类内方差支持向量引导的字典学习算法.将融合Fisher线性鉴别分析和支持向量机大间隔分类准则的最小类内方差支持向量机作为鉴别条件,在模型分类器的交替优化过程中,充分考虑编码向量的分布信息,保障同类编码向量总体一致的同时降低向量间的耦合度并修正分类矢量,从而挖掘编码向量鉴别信息,使其更好地引导字典学习以提高算法分类性能.在人脸、物体和手写数字识别数据集上的实验结果表明,在大部分样本和原子数量条件下,该算法的识别率和原子鲁棒性均优于K奇异值分解、局部特征和类标嵌入约束等经典字典学习算法.     

基于模糊分割的支持向量机分类器

支持向量机算法对噪声点和异常点是敏感的,为了解决这个问题,人们提出了模糊支持向量机,但其中的模糊隶属度函数需要人为设置。提出基于模糊分割的支持向量机分类器。在该算法中,首先根据聚类有效性用模糊c-均值聚类分别对训练集中的正负类数据聚类;然后,选择距离最近的c个聚类对构成c个二分类问题;最后,对c个二分类器用加权平均策略得到最终分类结果。为了验证所提算法的有效性,对三个UCI数据集进行了数值实验,结果表明,该算法能有效提高带噪声点和异常点数据集分类的预测精度。     

基于SVM主动学习技术的PU文本分类

PU文本分类(以正例和未标识实例集训练分类器的分类方法)关键在于从U(未标识实例)集中提取尽可能多的可靠反例,然后在正例与可靠反例的基础上使用机器学习的方法构造有效分类器,而已有的方法可靠反例的数量少或不可靠,同样构造的分类器也精度不高,基于SVM主动学习技术的PU文本分类算法提出一种利用SVM与改进的Rocchio分类器进行主动学习的PU文本分类方法,并通过spy技术来提高SVM分类器的准确度,解决某些机器学习中训练样本获取代价过大,尤其是反例样本较难获取的实际问题。实验表明,该方法比目前其它的主动学习方法及面向PU的文本分类方法具有更高的准确率和召回率。     

An effective head pose estimation approach using Lie Algebrized Gaussians based face representation

The accuracy of head pose estimation is significant for many computer vision applications such as face recognition, driver attention detection and human-computer interaction. Most appearance-based head pose estimation works typically extract the low-dimensional face appearance features in some statistic subspaces, where the subspaces represent the underlying geometry structure of the pose space. However, there is an open problem, namely, how to effectively represent appearance-based subspace face for the head pose estimation problem. To address the problem, this paper proposes a head pose estimation approach based on the Lie Algebrized Gaussians (LAG) feature to model the pose characteristic. LAG is built on Gaussian Mixture Models (GMM), which actually not only models the distribution of local appearance features, but also captures the Lie group manifold structure of the feature space. Moreover, to keep multi-resolution structure information, LAG is operated on many subregions of the image. As a result, these properties of LAG enable it to effectively model the structure of subspace face which can lead to powerful discriminative ability for head pose estimation. After representing subspace face using the LAG, we treat the head pose estimation as a classification problem. The within-class covariance normalization (WCCN) based Support Vector Machine (SVM) classifier is employed to achieve robust performance as WCCN could reduce the within-class variabilities of the same pose. Extensive experimental analysis and comparison with both traditional and state-of-the-art algorithms on two challenging benchmarks demonstrate the effectiveness of our approach.     

Social media research: The application of supervised machine learning in organizational communication research.

Despite the online availability of data, analysis of this information in academic research is arduous. This article explores the application of supervised machine learning (SML) to overcome challenges associated with online data analysis. In SML classifiers are used to categorize and code binary data. Based on a case study of Dutch employees’ work-related tweets, this paper compares the coding performance of three classifiers, Linear Support Vector Machine, Naïve Bayes, and logistic regression. The performance of these classifiers is assessed by examining accuracy, precision, recall, the area under the precision-recall curve, and Krippendorf’s Alpha. These indices are obtained by comparing the coding decisions of the classifier to manual coding decisions. The findings indicate that the Linear Support Vector Machine and Naïve Bayes classifiers outperform the logistic regression classifier. This study also compared the performance of these classifiers based on stratified random samples and random samples of training data. The findings indicate that in smaller training sets stratified random training samples perform better than random training samples, in large training sets (n = 4000) random samples yield better results. Finally, the Linear Support Vector Machine classifier was trained with 4000 tweets and subsequently used to categorize 578,581 tweets obtained from 430 employees.     

基于神经网络的支持向量机学习方法研究

针对支持向量机（Support Vector Machine,SVM）对大规模样本分类效率低下的问题,提出了基于自适应共振理论（Adaptive Resonance Theory,ART）神经网络与自组织特征映射（Self-Organizing feature Map,SOM）神经网络的SVM训练算法,分别称为ART-SVM算法与SOM-SVM算法。这两种算法通过聚类压缩数据集,使SVM训练的速度大大提高,同时可获得令人满意的泛化能力。     

Brain tumor segmentation and classification in MRI using SVM and its variants: a survey

The process of separation of brain tumor from normal brain tissues is Brain tumor segmentation. Segmentation of tumor from the MR images is a very challenging task as brain tumors are of different shapes and sizes. There are multiple phases to achieve the segmentation and the phases are pre-processing, segmentation, feature extraction, feature reduction, and classification of the tumor into benign and malignant. In this paper, Otsu thresholding is used in segmentation phase, Discrete Wavelet Transform (DWT) in feature extraction phase, Principal Component Analysis (PCA) in feature reduction phase and Support Vector Machine (SVM), Least Squared-Support Vector Machine (LS-SVM), Proximal Support Vector Machine (PSVM) and Twin Support Vector Machine (TWSVM) in the classification phase. We have compared the performances of all these classifiers, where TWSVM outperformed all other classifiers with 100% accuracy.

     

A second-order cone programming formulation for twin support vector machines

Second-order cone programming (SOCP) formulations have received increasing attention as robust optimization schemes for Support Vector Machine (SVM) classification. These formulations study the worst-case setting for class-conditional densities, leading to potentially more effective classifiers in terms of performance compared to the standard SVM formulation. In this work we propose an SOCP extension for Twin SVM, a recently developed classification approach that constructs two nonparallel classifiers. The linear and kernel-based SOCP formulations for Twin SVM are derived, while the duality analysis provides interesting geometrical properties of the proposed method. Experiments on benchmark datasets demonstrate the virtues of our approach in terms of classification performance compared to alternative SVM methods.     

基于元学习和叠加法的双层支持向量机算法

提出一种模式识别算法——双层支持量机算法,用来提高表面肌电识别精度。该算法融合集成学习中元学习的并行方法和叠加法的递进思想,把基本SVM分类器并行分布在第1层,第1层的预测结果作为第2层的输入,由第2层再进行分类识别,从而通过多层分类器组合来融合多源特征。以手臂表面肌电数据集为测试数据,采用文中的双层支持向量机,各肌肉的肌电信号分别输入基支持向量机,组合器融合各肌肉电信号特征,集成识别前臂肌肉群的肌电信号,从而实现运动意图的精确识别。实验结果显示,在预测精度上,此算法优于单个SVM分类器。在预测性能上(识别精度、耗时、鲁棒性),此算法优于随机森林和旋转森林等集成分类器。     

New classification techniques for electroencephalogram (EEG) signals and a real-time EEG control of a robot

This paper studies the state-of-the-art classification techniques for electroencephalogram (EEG) signals. Fuzzy Functions Support Vector Classifier, Improved Fuzzy Functions Support Vector Classifier and a novel technique that has been designed by utilizing Particle Swarm Optimization and Radial Basis Function Networks (PSO-RBFN) have been studied. The classification performances of the techniques are compared on standard EEG datasets that are publicly available and used by brain–computer interface (BCI) researchers. In addition to the standard EEG datasets, the proposed classifier is also tested on non-EEG datasets for thorough comparison. Within the scope of this study, several data clustering algorithms such as Fuzzy C-means, K-means and PSO clustering algorithms are studied and their clustering performances on the same datasets are compared. The results show that PSO-RBFN might reach the classification performance of state-of-the art classifiers and might be a better alternative technique in the classification of EEG signals for real-time application. This has been demonstrated by implementing the proposed classifier in a real-time BCI application for a mobile robot control.     

基于VPRS理论的一种混合分类算法

在文本分类领域中,KNN与SVM算法都具有较高的分类准确率,但两者都有其内在的缺点,KNN算法会因为大量的训练样本而导致计算量过大;SVM算法对于噪声数据过于敏感,对分布在分类超平面附近的数据点无法进行准确的分类,基于此提出一种基于变精度粗糙集理论的混合分类算法,该算法能够充分利用二者的优势同时又能克服二者的弱点,最后通过实验证明混合算法能够有效改善计算复杂度与分类精度。     

Feature selection for support vector machines with RBF kernel

Linear kernel Support Vector Machine Recursive Feature Elimination (SVM-RFE) is known as an excellent feature selection algorithm. Nonlinear SVM is a black box classifier for which we do not know the mapping function F{\Phi} explicitly. Thus, the weight vector w cannot be explicitly computed. In this paper, we proposed a feature selection algorithm utilizing Support Vector Machine with RBF kernel based on Recursive Feature Elimination(SVM-RBF-RFE), which expands nonlinear RBF kernel into its Maclaurin series, and then the weight vector w is computed from the series according to the contribution made to classification hyperplane by each feature. Using w_i²{w_i^2} as ranking criterion, SVM-RBF-RFE starts with all the features, and eliminates one feature with the least squared weight at each step until all the features are ranked. We use SVM and KNN classifiers to evaluate nested subsets of features selected by SVM-RBF-RFE. Experimental results based on 3 UCI and 3 microarray datasets show SVM-RBF-RFE generally performs better than information gain and SVM-RFE.