Dual margin approach on a Lagrangian support vector machine

In this paper, we propose a new support vector machine (SVM) called dual margin Lagrangian support vectors machine (DMLSVM). Unlike other SVMs which use only support vectors to determine the separating hyperplanes, DMLSVM utilizes all the available training data for training the classifier, thus producing robust performance. The training data are weighted differently depending on whether they are in a marginal region or surplus region. For fast training, DMLSVM borrows its training algorithm from Lagrangian SVM (LSVM) and tailors the algorithm to its formulation. The convergence of our training method is rigorously proven and its validity is tested on a synthetic test set and UCI dataset. The proposed method can be used in a variety of applications such as a recommender systems for web contents of IPTV services.     

Multiclass Lagrangian support vector machine

A support vector machine (SVM) has been developed for two-class problems, although its application to multiclass problems is not straightforward. This paper proposes a new Lagrangian SVM (LSVM) for application to multiclass problems. The multiclass Lagrangian SVM is formulated as a single optimization problem considering all the classes together, and a training method tailored to the multiclass problem is presented. A multiclass output representation matrix is defined to simplify the optimization formulation and associated training method. The proposed method is applied to some benchmark datasets in repository, and its effectiveness is demonstrated via simulation.     

An efficient weighted Lagrangian twin support vector machine for imbalanced data classification

In this paper, we propose an efficient weighted Lagrangian twin support vector machine (WLTSVM) for the imbalanced data classification based on using different training points for constructing the two proximal hyperplanes. The main contributions of our WLTSVM are: (1) a graph based under-sampling strategy is introduced to keep the proximity information, which is robustness to outliers, (2) the weight biases are embedded in the Lagrangian TWSVM formulations, which overcomes the bias phenomenon in the original TWSVM for the imbalanced data classification, (3) the convergence of the training procedure of Lagrangian functions is proven and (4) it is tested and compared with some other TWSVMs on synthetic and real datasets to show its feasibility and efficiency for the imbalanced data classification.     

一种聚类加权支持向量机算法及其在软测量中的应用

针对支持向量机应用于软测量建模时,工业过程数据中特异点影响建模精度的问题,提出聚类加权支持向量机方法.该方法首先对建模数据进行聚类分析,根据聚类结果,对各类数据的惩罚系数进行相应的加权,改变权值大小既能减小特异点对模型的影响程度,又能将其包含的生产过程信息引入到软测量模型中.聚丙烯熔融指数软测量的实例研究表明,通过对建模数据进行聚类分析和加权处理,聚类加权支持向量机比标准支持向量机建模更准确.     

基于数据场的SVM技术在雷暴预报中的应用

针对天气预报中样本不平衡造成漏报率高的问题,提出一种基于数据场的C加权支持向量机（SVM）技术。该技术对不平衡天气数据进行分类,采用叠加数据场势值作为数据重采样依据,筛选出最利于SVM分类器学习的样本作为训练样本,结合C加权方法进行训练。实验结果证明,在样本数量较多且不平衡性显著的雷暴天气中,该技术能缩减训练集规模,减少漏报,提升预报系统的g-means值。     

A novel fuzzy compensation multi-class support vector machine

This paper presents a novel fuzzy compensation multi-class support vector machine (FCM-SVM) to improve the outlier and noise sensitivity problem of traditional support vector machine (SVM) for multi-class data classification. The basic idea is to give the dual effects to penalty term through treating every data point as both positive and negative classes, but with different memberships. We fuzzify penalty term, compensate weight to classification, reconstruct the optimization problem and its restrictions, reconstruct {Lagrangian} formula, and present the theoretic deduction. By this way the new fuzzy compensation multi-class support vector machine is expected to have more generalization ability while preserving the merit of insensitive to outliers. Experimental results on benchmark data set and real data set show that the proposed method reduces the effect of noise data and yields higher classification rate than traditional multi-class SVM does.     

基于稳健最小二乘支持向量机的测深激光信号处理

针对浅海探测中激光回波噪声源多、信噪比低,传统非加权最小二乘支持向量机和加权最小二乘支持向量机对低信噪比信号滤波不足的问题,提出将稳健最小二乘法与加权最小二乘支持向量机相结合的滤波方法(HW-LS-SVM)。首先采用强淘汰权函数计算先验权值、残差和均方误差,然后采用权函数模型计算最小二乘支持向量机的权值,最后通过迭代计算实现回波信号滤波。通过仿真实验结果表明, HW-LS-SVM方法较最小二乘支持向量机、贝叶斯最小二乘支持向量机和传统加权最小二乘支持向量机滤波效果更加稳健,在噪声率为45%的情况下,滤波效果较为理想,水面和水底回波提取正确率为100%;对实测4组深水区和4组浅水区数据滤波后提取的海水深度均与背景资料的深度吻合。由此表明, HW-LS-SVM方法具有更好的抗噪性,更适合于对信噪比低的测深激光信号的滤波处理。     

Projection Support Vector Machine Generators

Large-scale Support Vector Machine (SVM) classification is a very active research line in data mining. In recent years, several efficient SVM generation algorithms based on quadratic problems have been proposed, including: Successive OverRelaxation (SOR), Active Support Vector Machines (ASVM) and Lagrangian Support Vector Machines (LSVM). These algorithms have been used to solve classification problems with millions of points. ASVM is perhaps the fastest among them. This paper compares a new projection-based SVM algorithm with ASVM on a selection of real and synthetic data sets. The new algorithm seems competitive in terms of speed and testing accuracy.     

Lagrangian relaxation for SVM feature selection

We discuss a Lagrangian-relaxation-based heuristics for dealing with feature selection in the Support Vector Machine (SVM) framework for binary classification. In particular we embed into our objective function a weighted combination of the L₁ and L₀ norm of the normal to the separating hyperplane. We come out with a Mixed Binary Linear Programming problem which is suitable for a Lagrangian relaxation approach.Based on a property of the optimal multiplier setting, we apply a consolidated nonsmooth optimization ascent algorithm to solve the resulting Lagrangian dual. In the proposed approach we get, at every ascent step, both a lower bound on the optimal solution as well as a feasible solution at low computational cost.We present the results of our numerical experiments on some benchmark datasets.     

神经网络和SVM多传感器融合的隧道CO体积分数研究

利用BP,RBF神经网络、支持向量机(SVM)不同信息融合模型对高速公路隧道中失效CO传感器数据融合研究,比较了单一融合模型融合效果和不同最优加权信息融合模型融合效果。仿真实验表明:3种单一融合模型对隧道CO体积分数融合的有效性,其中单一SVM融合模型效果最好。最优加权融合性能均优于单一融合模型,其中,BP与SVM最优加权融合精度最高。还分析单一融合模型输出之间的冗余度对最优加权融合精度的影响规律,该规律为参与最优加权融合的单一模型筛选提供了一种新方法。     

Weighted Dantzig–Wolfe Decomposition for Linear Mixed-integer Programming

Dantzig–Wolfe decomposition can be used to solve the Lagrangian dual of a linear mixed-integer programming problem ( MIP ) if the dual structure of the ( MIP ) is exploited via Lagrangian relaxation with respect to the complicating constraints. In the so-called weighted Dantzig–Wolfe decomposition algorithm, instead of the optimal solution of the Dantzig–Wolfe master problem a specially weighted average of the previously constructed Lagrangian multipliers and the optimal solution of the master problem is used as Lagrangian multiplier for the next Lagrangian subproblem to be solved. A convergence proof of the weighted Dantzig–Wolfe decomposition algorithm is given, and some properties of this procedure together with computational results for the capacitated facility location problem are discussed.     

Fuzzy C-means based support vector machine for channel equalisation

This paper proposes a new classification network, the fuzzy C-means based support vector machine (FCM–SVM) and applies it to channel equalisation. In contrast to a kernel-based SVM, the FCM–SVM has a smaller number of parameters while retaining the SVM's good generalisation ability. In FCM–SVM, input training data is clustered by FCM. The output of FCM–SVM is a weighted sum of the degrees where each input data belongs to the clusters. To achieve high generalisation ability, FCM–SVM weights are learned through linear kernel based SVM. Computer simulations illustrate the performance of the suggested network, where the FCM–SVM is used as a channel equaliser. Simulations with white Gaussian and coloured Gaussian noise are performed. This paper also compares simulation results from the FCM–SVM, the Gaussian kernel based SVM and the optimal equaliser.     

一种改进的局部支持向量机算法

局部支持向量机是一种用途广泛的分类器,无论在理论研究还是实际应用方面,局部支持向量机都受到越来越多的关注。目前,许多传统的局部支持向量机算法都存在一个问题,即模型中样本比例失衡,导致无法提高分类精度。在加权支持向量机的启发下,提出了将加权思想应用在局部支持向量机Falk-SVM中的WFalk-SVM算法,并通过实验分析验证了WFalk-SVM的可行性及其有效性,最后对WFalk-SVM算法进行分析总结。     

用于支持向量机拒识区域的加权k近邻法

为解决1-v-r和1-v-1支持向量机中存在的拒识区域问题,提出一种加权k近邻法。该方法计算落入拒识区域中的样本,即拒识样本到所有训练样本的距离,选择最近的k个样本为拒识样本的类别投票,并根据距离大小进行加权,得票多的类即拒识样本的所属类。实验结果表明,加权k近邻法实现了零拒识,提高了传统多分类支持向量机的分类性能。     

基于快速SVM的大规模网络流量分类方法

支持向量机方法具有良好的分类准确率、稳定性与泛化性,在网络流量分类领域已有初步应用,但在面对大规模网络流量分类问题时却存在计算复杂度高、分类器训练速度慢的缺陷。为此,提出一种基于比特压缩的快速SVM方法,利用比特压缩算法对初始训练样本集进行聚合与压缩,建立具有权重信息的新样本集,在损失尽量少原始样本信息的前提下缩减样本集规模,进一步利用基于权重的SVM算法训练流量分类器。通过大规模样本集流量分类实验对比,快速SVM方法能在损失较少分类准确率的情况下,较大程度地缩减流量分类器的训练时间以及未知样本的预测时间,同时,在无过度压缩前提下,其分类准确率优于同等压缩比例下的随机取样SVM方法。本方法在保留SVM方法较好分类稳定性与泛化性能的同时,有效提升了其应对大规模流量分类问题的能力。     

支持向量机中优化算法

1 引言对基于数掘的机器学习研究是目前计算智能技术的基础性工作之一,旨在从观测数据出发基于统计学理论采用相关技术对研究对象建立可信的模型,利用这一模型对未来实验作出指导性的预测。相关技术包括聚类分析、模式识别、神经网络等等。现有的学习方法多是基于样本数趋于无穷大时的渐进理论。可是在实际科研工作中,有时候很难获得较为充足     

基于鲁棒最小二乘支持向量机的气动参数拟合

最小二乘支持向量机（LS－SVM）比标准支持向量机具有更高的计算效率,但是却散失了标准支持向量机的稀疏特性,而且当考虑异常值或者误差变量的高斯假设不成立时,会导致不稳健的估计结果。为了克服这两个缺点,在飞行器的气动参数拟合计算中引入了一种鲁棒最小二乘支持向量机（RLS－SVM）,该方法通过加权的支持向量机来获得鲁棒估计,并通过对支持值谱进行剪枝最终得到稀疏解。仿真结果表明：RLS－SVM方法简单,学习速度快,拟合精度高,鲁棒性强,是一种在飞行器轨迹计算中值得推广和采用的方法。     

基于后验概率的支持向量机

在支持向量机(support vector machines,SVM)中,训练样本总是具有明确的类别信息,而对于一些不确定性问题并不恰当．受贝叶斯决策规则的启发,利用样本的后验概率来表示这种不确定性．将贝叶斯决策规则与SVM相结合,建立后验概率支持向量机(posteriori probability support vector machine,PPSVM)的体系框架．并详细讨论线性可分性、间隔、最优超平面以及软间隔算法,得到了一个新的优化问题,同时给出了一个支持向量的新定义．实际上,后验概率支持向量机是建立于统计学习理论(statistical learning theory)基础之上,是标准SVM的扩展．针对数据,还提出了一个确定后验概率的经验性方法．实验也证明了后验概率支持向量机的合理性、有效性．     

Engineering fast multilevel support vector machines

The computational complexity of solving nonlinear support vector machine (SVM) is prohibitive on large-scale data. In particular, this issue becomes very sensitive when the data represents additional difficulties such as highly imbalanced class sizes. Typically, nonlinear kernels produce significantly higher classification quality to linear kernels but introduce extra kernel and model parameters which requires computationally expensive fitting. This increases the quality but also reduces the performance dramatically. We introduce a generalized fast multilevel framework for regular and weighted SVM and discuss several versions of its algorithmic components that lead to a good trade-off between quality and time. Our framework is implemented using PETSc which allows an easy integration with scientific computing tasks. The experimental results demonstrate significant speed up compared to the state-of-the-art nonlinear SVM libraries. Reproducibility: our source code, documentation and parameters are available at https://github.com/esadr/mlsvm.

     

基于正样本和未标记样本的遥感图像分类方法

传统分类器的构建需要正样本和负样本两类数据。在遥感影像分类中,常出现这样一类情形：感兴趣的地物只有一种。由于标记样本耗时耗力,未标记样本往往容易获取并且包含有用信息,鉴于此,提出了一种基于正样本和未标记样本的遥感图像分类方法（PUL）。首先,根据正样本固有特征并结合支持向量数据描述（SVDD）从未标记集筛选出可信正负样本,再将其从未标记集中剔除;接着将其带入SVM训练,根据未标记集在分类器中的表现设立阈值,再从未标记集中筛选出相对可靠的正负样本;最后是加权SVM（Weighted SVM）过程,初始正样本及提取出的可靠正负样本权重为1,SVM训练筛选出的样本权重范围0~1。为验证PUL的有效性,在遥感影像进行分类实验,并与单类支持向量机（OC-SVM）、高斯数据描述（GDD）、支持向量数据描述（SVDD）、有偏SVM（Biased SVM）以及多类SVM分类对比,实验结果表明PUL提高了分类效果,优于上述单类分类方法及多类SVM方法。