A MapReduce-based distributed SVM algorithm for automatic image annotation

Machine learning techniques have facilitated image retrieval by automatically classifying and annotating images with keywords. Among them Support Vector Machines (SVMs) have been used extensively due to their generalization properties. However, SVM training is notably a computationally intensive process especially when the training dataset is large. This paper presents MRSMO, a MapReduce based distributed SVM algorithm for automatic image annotation. The performance of the MRSMO algorithm is evaluated in an experimental environment. By partitioning the training dataset into smaller subsets and optimizing the partitioned subsets across a cluster of computers, the MRSMO algorithm reduces the training time significantly while maintaining a high level of accuracy in both binary and multiclass classifications.     

A Resource Aware MapReduce Based Parallel SVM for Large Scale Image Classifications

Machine learning techniques have facilitated image retrieval by automatically classifying and annotating images with keywords. Among them support vector machines (SVMs) are used extensively due to their generalization properties. However, SVM training is notably a computationally intensive process especially when the training dataset is large. This paper presents RASMO, a resource aware MapReduce based parallel SVM algorithm for large scale image classifications which partitions the training data set into smaller subsets and optimizes SVM training in parallel using a cluster of computers. A genetic algorithm based load balancing scheme is designed to optimize the performance of RASMO in heterogeneous computing environments. RASMO is evaluated in both experimental and simulation environments. The results show that the parallel SVM algorithm reduces the training time significantly compared with the sequential SMO algorithm while maintaining a high level of accuracy in classifications.     

A comparison of methods for multiclass support vector machines

Support vector machines (SVMs) were originally designed for binary classification. How to effectively extend it for multiclass classification is still an ongoing research issue. Several methods have been proposed where typically we construct a multiclass classifier by combining several binary classifiers. Some authors also proposed methods that consider all classes at once. As it is computationally more expensive to solve multiclass problems, comparisons of these methods using large-scale problems have not been seriously conducted. Especially for methods solving multiclass SVM in one step, a much larger optimization problem is required so up to now experiments are limited to small data sets. In this paper we give decomposition implementations for two such "all-together" methods. We then compare their performance with three methods based on binary classifications: "one-against-all," "one-against-one," and directed acyclic graph SVM (DAGSVM). Our experiments indicate that the "one-against-one" and DAG methods are more suitable for practical use than the other methods. Results also show that for large problems methods by considering all data at once in general need fewer support vectors.     

Fast Multiclass SVM Classification Using Decision Tree Based One-Against-All Method

We present an improved version of One-Against-All (OAA) method for multiclass SVM classification based on a decision tree approach. The proposed decision tree based OAA (DT-OAA) is aimed at increasing the classification speed of OAA by using posterior probability estimates of binary SVM outputs. DT-OAA decreases the average number of binary SVM tests required in testing phase to a greater extent when compared to OAA and other multiclass SVM methods. For a balanced multiclass dataset with K classes, under best situation, DT-OAA requires only (K + 1)/2 binary tests on an average as opposed to K binary tests in OAA; however, on imbalanced multiclass datasets we observed DT-OAA to be much faster with proper selection of order in which the binary SVMs are arranged in the decision tree. Computational comparisons on publicly available datasets indicate that the proposed method can achieve almost the same classification accuracy as that of OAA, but is much faster in decision making.     

基于二叉树的多分类SVM算法在电子邮件过滤中的应用

目前性能较好的多分类算法有1-v-r支持向量机（SVM）、1-1-1SVM、DDAG SVM等，但存在大量不可分区域且训练时间较长的问题。提出一种基于二叉树的多分类SVM算法用于电子邮件的分类与过滤，通过构建二叉树将多分类转化为二值分类，算法采用先聚类再分类的思想，计算测试样本与子类中心的最大相似度和子类间的分离度，以构造决策节点的最优分类超平面。对于C类分类只需C-1个决策函数，从而可节省训练时间。实验表明，该算法得到了较高的查全率、查准率。     

超球体多类支持向量机理论

目前的多类分类器大多是经二分类器组合而成的,存在训练速度较慢的问题,在分类类别多的时候,会遇到很大困难,超球体多类支持向量机将超球体单类支持向量机扩展到多类问题,由于每类样本只参与一个超球体支持向量机的训练.因此,这是一种直接多类分类器,训练效率明显提高.为了有效训练超球体多类支持向量机,利用SMO算法思想,提出了超球体支持向量机的快速训练算法.同时对超球体多类支持向量机的推广能力进行了理论上的估计.数值实验表明,在分类类别较多的情况,这种分类器的训练速度有很大提高,非常适合解决类别数较多的分类问题.超球体多类支持向量机为研究快速直接多类分类器提供了新的思路.     

A Novel and Principled Multiclass Support Vector Machine

Support vector machines (SVMs) have been demonstrated very efficient for binary classification problems; however, computationally efficient and effective multiclass SVMs are still missing. Most existing multiclass SVM classifiers are constructed either by combining multiple binary SVM classifiers, which often perform moderately for some problems, or by converting multiclass problems into one single optimization problem, which is unfortunately computationally expensive. To address these issues, a novel and principled multiclass SVM based on geometric properties of hyperspheres, termed SVMGH, is proposed in this paper. Different from existing SVM‐based methods that seek a cutting hyperplane between two classes, SVMGH draws the discriminative information of each class by constructing a minimum hypersphere containing all class members, and then defines a label function based on the geometric properties of the minimum hyperspheres. We prove theoretically the geometric properties of the minimum hyperspheres to guarantee the validation of SVMGH. The computational efficiency is enhanced by a data reduction strategy as well as a fast training method. Experimental results demonstrate that the proposed SVMGH shows better performance and higher computational efficiency than the state of the art on multiclassification problems while maintaining comparable performance and efficiency on binary classification problems.     

液晶字符识别的CNN和SVM组合分类器

针对仪表液晶显示字符识别问题,提出一种结合了卷积神经网络(CNN)和支持向量机(SVM)的 字符识别方法。分别采用具有并联结构的 CNN 模型和基于梯度方向直方图(HOG)特征的 SVM 方法构建基本分 类器,当 2 个分类器的结果存在冲突时,利用 CNN 的 softmax 输出最大值判决最终结果,当其大于设定阈值 时采用 CNN 分类器的结果,反之采用 SVM 分类器的结果。建立字符图像的误差模型并利用仿真方法构建了 数据集用于分类器的训练和测试,给出一种基于投票原理的最优阈值的估计算法。在 MNIST 和仿真数据集上 的测试实验结果表明,最优阈值估计算法的结果可靠,组合分类器的准确率较 2 种单一分类器均有提高,在实 际测试系统上其准确率达到 99.81%,验证了该组合分类器方法对液晶字符识别问题的有效性;在 CIFAR-10 数 据集上的实验结果验证了该方法也可用于其他分类问题。     

基于马氏距离的孪生多分类支持向量机

孪生支持向量机(TWSVM)的研究是近来机器学习领域的一个热点。TWSVM具有分类精度高、训练速度快等优点,但训练时没有充分利用样本的统计信息。作为TWSVM的改进算法,基于马氏距离的孪生支持向量机(TMSVM)在分类过程中考虑了各类样本的协方差信息,在许多实际问题中有着很好的应用效果。然而TMSVM的训练速度有待提高,并且仅适用于二分类问题。针对这两个问题,将最小二乘思想引入TMSVM,用等式约束取代TMSVM中的不等式约束,将二次规划问题的求解简化为求解两个线性方程组,得到基于马氏距离的最小二乘孪生支持向量机(LSTMSVM),并结合有向无环图策略(DAG)设计出基于马氏距离的最小二乘孪生多分类支持向量机。为了减少DAG结构的误差累积,构造了基于马氏距离的类间可分性度量。人工数据集和UCI数据集上的实验均表明,所提算法不仅有效,而且相对于传统多分类SVM,其分类性能有明显提高。     

DESIGN OF DECISION TREE VIA KERNELIZED HIERARCHICAL CLUSTERING FOR MULTICLASS SUPPORT VECTOR MACHINES

As a very effective method for universal purpose pattern recognition, support vector machine (SVM) was proposed for dichotomic classification problem, which exhibits a remarkable resistance to overfitting, a feature explained by the fact that it directly implements the principle of structural risk minimization. However, in real world, most of classification problems consist of multiple categories. In an attempt to extend the binary SVM classifier for multiclass classification, decision-tree-based multiclass SVM was proposed recently, in which the structure of decision tree plays an important role in minimizing the classification error. The present study aims at developing a systematic way for the design of decision tree for multiclass SVM. Kernel-induced distance function between datasets was discussed and then kernelized hierarchical clustering was developed and used in determining the structure of decision tree. Further, simulation results on satellite image interpretation show the superiority of the proposed classification strategy over the conventional multiclass SVM algorithms.     

Multiclass Lagrangian support vector machine

A support vector machine (SVM) has been developed for two-class problems, although its application to multiclass problems is not straightforward. This paper proposes a new Lagrangian SVM (LSVM) for application to multiclass problems. The multiclass Lagrangian SVM is formulated as a single optimization problem considering all the classes together, and a training method tailored to the multiclass problem is presented. A multiclass output representation matrix is defined to simplify the optimization formulation and associated training method. The proposed method is applied to some benchmark datasets in repository, and its effectiveness is demonstrated via simulation.     

Using one-class and two-class SVMs for multiclass image annotation

We propose using one-class, two-class, and multiclass SVMs to annotate images for supporting keyword retrieval of images. Providing automatic annotation requires an accurate mapping of images' low-level perceptual features (e.g., color and texture) to some high-level semantic labels (e.g., landscape, architecture, and animals). Much work has been performed in this area; however, there is a lack of ability to assess the quality of annotation. In this paper, we propose a confidence-based dynamic ensemble (CDE), which employs a three-level classification scheme. At the base-level, CDE uses one-class support vector machines (SVMs) to characterize a confidence factor for ascertaining the correctness of an annotation (or a class prediction) made by a binary SVM classifier. The confidence factor is then propagated to the multiclass classifiers at subsequent levels. CDE uses the confidence factor to make dynamic adjustments to its member classifiers so as to improve class-prediction accuracy, to accommodate new semantics, and to assist in the discovery of useful low-level features. Our empirical studies on a large real-world data set demonstrate CDE to be very effective.     

Multiclass Support Vector Machines for Classification of ECG Data with Missing Values

The article presents an experimental study on multiclass Support Vector Machine (SVM) methods over a cardiac arrhythmia dataset that has missing attribute values for electrocardiogram (ECG) diagnostic application. The presence of an incomplete dataset and high data dimensionality can affect the performance of classifiers. Imputation of missing data and discriminant analysis are commonly used as preprocessing techniques in such large datasets. The article proposes experiments to evaluate performance of One-Against-All (OAA) and One-Against-One (OAO) approaches in kernel multiclass SVM for a heartbeat classification problem with imputation and dimension reduction techniques. The results indicate that the OAA approach has superiority over OAO in multiclass SVM for ECG data analysis with missing values.     

基于混合粒子群算法的运动估计研究

针对块匹配运动估计算法中传统搜索方法的不足,提出了一种新的基于混合粒子群的块匹配运动估计算法。在保留系统随机搜索性能的同时根据运动矢量特性合理地设计初始搜索种群,并通过混沌差分进化搜索协同粒子群算法迭代寻优,混沌序列用于优化差分变异算子,以提高算法的精细搜索能力。通过相同点检测技术和恰当的终止计划有效地降低了系统的运算复杂度。经实验测试与验证,该算法在搜索质量和运算复杂度中达到了一种动态平衡的状态,其整体性能高于传统的快速运动估计算法,效果更逼近于穷举搜索法。     

支持向量机多类分类算法研究

提出一种新的基于二叉树结构的支持向量(SVM)多类分类算法.该算法解决了现有主要算法所存在的不可分区域问题.为了获得较高的推广能力,必须让样本分布广的类处于二叉树的上层节点,才能获得更大的划分空间.所以,该算法采用最小超立方体和最小超球体类包含作为二叉树的生成算法.实验结果表明,该算法具有一定的优越性.     

快速的支持向量机多类分类研究

研究了支持向量机多类算法DAGSVM（Direct Acyclic Graph SVM）的速度优势,提出了结合DAGSVM和简化支持向量技术的一种快速支持向量机多类分类方法。该方法一方面减少了一次分类所需的两类支持向量机的数量,另一方面减少了支持向量的数量。实验采用UCI和Statlog数据库的多类数据,并和四种多类方法进行比较,结果表明该方法能有效地加快分类速度。     

一种新的二叉树多类支持向量机算法

采用二叉树结构对多个二值支持向量机(SVM)子分类器组合,可实现多类问题的分类,并且还可克服传统多类SVM算法存在的不可分区域的情况。针对现有二叉树多类SVM方法未采用有效的二叉树生成算法,该文采用聚类分析中的类距离思想,提出了一种新的基于二叉树的多类SVM分类方法。实验结果表明,新算法具有较高的推广性能。     

粗糙集理论和DT_SVM在Web信息过滤中的应用

针对Web信息过滤问题,提出一种将粗糙集理论和决策树SVM(DT_SVM)相结合进行数据分类、过滤的新方法。该方法运用改进的启发式相对属性约简算法消除冗余、降低样本空间维数,通过聚类和DT_SVM相结合来训练SVM,将多分类问题转化为二值分类问题,提高了训练速度及过滤精度。实验表明,该算法得到了较高的查全率、查准率,体现了将粗糙集理论与DT_SVM算法结合的优越性。