一种支持向量逐步回归机算法研究

支持向量机是解决非线性问题的重要工具,对多元线性回归模型和支持向量机的原始形式进行比较,拟定从样本子集的多元线性回归模型出发,逐步搜索支持向量,提出了一种建立支持向量回归机的快速算法,以降低核矩阵的规模从而降低解凸二次规划的复杂度;最后,分析了该算法的复杂度,并提供了一个算例。     

支持向量机最优模型选择的研究

通过对核矩阵的研究,利用核矩阵的对称正定性,采用核校准的方法提出了一种SVM最优模型选择的算法——OMSA算法．利用训练样本不通过SVM标准训练和测试过程而寻求最优的核参数和相应的最优学习模型,弥补了传统SVM在模型选择上经验性强和计算量大的不足．采用该算法在UCI标准数据集和FERET标准人脸库上进行了实验,结果表明,通过该算法找到的核参数以及相应的核矩阵是最优的,得到的SVM分类器的错误率最小．该算法为SVM最优模型选择提供了一种可行的方法,同时对其他基于核的学习方法也具有一定的参考价值．     

支持向量机的算法及应用

统计学习理论是由Vapnik建立的一种专门研究小样本情况下机器学习规律的理论,支持向量机（SVM）是在这一理论基础上发展而来的一种新的通用学习方法。目前SVM已成为国际上机器学习领域新的研究热点,本文是一篇综述,旨在介绍SVM的一般理论、算法及应用,以引起国内学者的进一步关注。     

鲁棒的光滑支持向量机

光滑支持向量机(Smooth Support Vector Machine,SSVM)是传统支持向量机的一种改进模型,它利用光滑方法将传统支持向量机的二次规划问题转化成无约束优化问题,并使用Newton-Armijo算法求解该无约束优化问题。在光滑支持向量机的基础上提出了鲁棒的光滑支持向量机(Robust Smooth Support Vector Machine,RSSVM),其利用M-estimator代替SSVM中基于L2范数的正则化项,并利用半二次最小化优化方法求解相应的最优化问题。实验结果表明所提方法可以有效地提高SSVM的抗噪声能力。     

自适应模糊支持向量机算法研究

一个有效的核方法通常取决于选择一个合适的核函数。目前研究核方法的热点是从数据中自动地进行核学习。提出基于最优分类标准的核学习方法,这个标准类似于线性鉴别分析和核Fisher判别式。并把此算法应用于模糊支持向量机多类分类器设计上,在ORL人脸数据集和Iris数据集上的实验验证了该算法的可行性。     

支持向量机研究进展

基于统计学习理论的支持向量机((Support vector machines, SVM)以其优秀的学习能力受到广泛的关注。但传统支持向量机在处理大规模二次规划问题时会出现训练时间长、效率低下等问题。对SVM训练算法的最新研究成果进行了综述,对主要算法进行了比较深入的分析和比较,指出了各自的优点及其存在的问题,并且着重介绍了目前研究的新进展—模糊SVM和粒度SVM。接着论述了SVM主要的两方面应用—分类和回归。最后给出了今后SVM研究方向的预见。     

一种基于支持向量机的半监督分类方法

如何有效利用海量的数据是当前机器学习面临的一个重要任务,传统的支持向量机是一种有监督的学习方法,需要大量有标记的样本进行训练,然而有标记样本的数量是十分有限的并且非常不易获取.结合Co-training算法与Tri-training算法的思想,给出了一种半监督SVM分类方法.该方法采用两个不同参数的SVM分类器对无标记样本进行标记,选取置信度高的样本加入到已标记样本集中.理论分析和计算机仿真结果都表明,文中算法能有效利用大量的无标记样本,并且无标记样本的加入能有效提高分类的正确率.     

基于约简核矩阵的稀疏最小二乘支持向量机

为了解决最小二乘支持向量机模型稀疏性不足的问题,提出了一种约简核矩阵的LS-SVM稀疏化方法.按照空间两点的欧式距离寻找核矩阵中相近的行(列),并通过特定的规则进行合并,以减小核矩阵的规模,进而求得稀疏LS-SVM模型.以高斯径向基核函数为例,详细阐述了改进方法的实现步骤,并通过仿真表明了采用该方法求得的稀疏LS-SVM模型泛化能力良好.     

一种改进的临近支持向量机

基于线性临近支持向量机,提出一种改进的分类器一直接支持向量机.该分类器与临近支持向量机相比,对线性分类二者相同;对于非线性分类,直接支持向量机的Lagrangian乘子求解公式和分类器的表达式都更加简单,计算复杂度降低一半,且通过替代核函数就可实现线性与非线性的统一,可使用相同的算法代码,改正了临近支持向量机的不足.数值实验表明,非线性分类时,直接支持向量机的训练速度比临近支持向量机要快一倍左右,而测试速度则快更多,且分类精度并没有降低.     

基于半监督支持向量机的交互式遗传算法

为减轻用户疲劳并将交互式遗传算法应用于复杂的优化问题中,提出一种基于半监督支持向量机的交互式遗传算法。根据标记样本和未标记样本几何特性派生出数据依赖的核函数,以此构建半监督支持向量机,再以自训练方法进行高可信未标记样本的批量选择,实现用户评价代理模型的高泛化性能。将该方法应用于基于内容的图像检索系统,结果表明其能有效加快进化收敛的速度,提高优化成功率。     

用核K-means聚类和半定规划SVM实现垃圾标签检测

提出使用核K-means聚类算法从样本集中抽取特征向量集来训练SVM,达到减少SVM规模的目的。SVM核函数的选择会影响SVM模型的分类效果,提出将多个非线性映射能力不同的核函数进行线性组合,在特征训练集上构造出组合SVM的半定规划模型,用内点法求解出最优组合系数,得到非线性映射能力更强的半定规划SVM,并用做垃圾标签检测。在UCI数据集上与双层减样支持向量机方法进行比较,实验结果表明,新的垃圾标签检测法提高了识别率,并大幅度减少了训练时间。     

Discretization method for semi-definite programming

Semi-definite programs are convex optimization problems arising in a wide variety of applications and the extension of linear programming. Most methods for linear programming have been generalized to semi-definite programs. This paper discusses the discretization method in semi-definite programming. The convergence and the convergent rate of error between the optimal value of the semi-definite programming problems and the optimal value of the discretized problems are obtained. An approximately optimal division is given under certain assumptions. With the significance of the convergence property, the duality result in semi-definite programs is proved in a simple way which is different from the other common proofs.     

Robust controllability assessment via semi-definite programming

This paper presents a new framework for systematic assessment of the controllability of uncertain linear time-invariant (LTI) systems. The objective is to evaluate controllability of an uncertain system with norm-bounded perturbation over the entire uncertain region. The method is based on a singular-value minimization problem, over the entire complex plane. To solve the problem, first, a necessary and sufficient condition is proposed to avert the difficulties of griding over the complex plane, and to verify the controllability of directed and undirected networks in a single step. Secondly, the results are utilized to formulate the problem as two Lyapunov-based linear matrix inequalities (LMIs) for undirected networks, and four LMIs, for directed ones. The proposed approach is subsequently extended to evaluate the maximum guaranteed distance to uncontrollability of a system through a quasi-convex optimization problem whose solution is guaranteed to be globally optimal. By duality, analogous results are established for robust observability of directed and undirected networks. The proposed framework is implemented efficiently using LMI tools, and provides a fast and reliable tool for the assessment of robust controllability (and by duality, robust observability). It is also extendable to robust control system design problems such as control node selection for uncertain systems.     

Learning linear PCA with convex semi-definite programming

The aim of this paper is to learn a linear principal component using the nature of support vector machines (SVMs). To this end, a complete SVM-like framework of linear PCA (SVPCA) for deciding the projection direction is constructed, where new expected risk and margin are introduced. Within this framework, a new semi-definite programming problem for maximizing the margin is formulated and a new definition of support vectors is established. As a weighted case of regular PCA, our SVPCA coincides with the regular PCA if all the samples play the same part in data compression. Theoretical explanation indicates that SVPCA is based on a margin-based generalization bound and thus good prediction ability is ensured. Furthermore, the robust form of SVPCA with a interpretable parameter is achieved using the soft idea in SVMs. The great advantage lies in the fact that SVPCA is a learning algorithm without local minima because of the convexity of the semi-definite optimization problems. To validate the performance of SVPCA, several experiments are conducted and numerical results have demonstrated that their generalization ability is better than that of regular PCA. Finally, some existing problems are also discussed.     

A primal-dual semi-definite programming approach to linearquadratic control

We study a deterministic linear-quadratic (LQ) control problem over an infinite horizon, without the restriction that the control cost matrix R or the state cost matrix Q be positive-definite. We develop a general approach to the problem based on semi-definite programming (SDP) and related duality analysis. We show that the complementary duality condition of the SDP is necessary and sufficient for the existence of an optimal LQ control under a certain stability condition (which is satisfied automatically when Q is positive-definite). When the complementary duality does hold, an optimal state feedback control is constructed explicitly in terms of the solution to the primal SDP     

基于RSS的无线传感器网络半定规划定位算法研究

在无线传感器网络定位中,基于RSS测量的定位方法是最常用的方法之一。由于传统的最大似然估计(MLE)算法的目标函数具有非线性和非凸性,在应用于无线传感器网络定位时,会产生多个局部最优值。针对该问题提出一种基于半定规划(SDP)的凸优化定位方法。首先采用泰勒级数近似对目标函数进行线性化处理,然后通过引入冗余变量将原无约束优化问题转化为约束优化问题,最后应用半定松弛(SDR)技术将约束优化问题转化为半定规划(SDP)凸优化问题进行求解。通过仿真实验的比较,说明本文提出的优化算法在定位精度、鲁棒性方面优于已有算法。     

A flexible semi-definite programming approach for source localization problems

In this paper, a new semi-definite programming approach is devised for approximating nonlinear estimation problems. The main idea is to include the noise components as parameters of interests, which increases the flexibility in the convex optimization formulation. Using the source localization as an illustration, we develop semi-definite relaxation (SDR) positioning algorithms using angle-of-arrival, time-of-arrival and time-difference-of-arrival measurements. Numerical examples are included to show the effectiveness of the proposed SDR approach.     

Outlier detection and least trimmed squares approximation using semi-definite programming

Robust linear regression is one of the most popular problems in the robust statistics community. It is often conducted via least trimmed squares, which minimizes the sum of the k smallest squared residuals. Least trimmed squares has desirable properties and forms the basis on which several recent robust methods are built, but is very computationally expensive due to its combinatorial nature. It is proven that the least trimmed squares problem is equivalent to a concave minimization problem under a simple linear constraint set. The “maximum trimmed squares”, an “almost complementary” problem which maximizes the sum of the q smallest squared residuals, in direct pursuit of the set of outliers rather than the set of clean points, is introduced. Maximum trimmed squares (MTS) can be formulated as a semi-definite programming problem, which can be solved efficiently in polynomial time using interior point methods. In addition, under reasonable assumptions, the maximum trimmed squares problem is guaranteed to identify outliers, no mater how extreme they are.     

A second-order Mehrotra-type predictor-corrector algorithm with a new wide neighbourhood for semi-definite programming

In this paper, we present a new second-order Mehrotra-type predictor–corrector algorithm for semi-definite programming (SDP). The proposed algorithm is based on a new wide neighbourhood. We are particularly concerned with an important inequality. Based on the inequality, the convergence is shown for a specific class of search directions. In particular, the complexity bound is O(√nlog?^?1) for the Nesterov–Todd search direction and O(n log?^?1) for the Helmberg-Kojima-Monteiro search direction. The derived complexity bounds coincide with the currently best known theoretical complexity bounds obtained so far for SDP. We provide some preliminary numerical results as well.