基于黎曼度量的最小二乘支持向量机模型选择

针对最小二乘支持向量机(LS-SVM)模型选择效果不稳定、易于过学习的问题,提出了一种基于黎曼度量的模型选择方法.首先,基于信息几何理论,证明了LS-SVM模型泛化能力受样本点二阶协变张量的影响;其次,进一步证明了同时最小化所有样本点的黎曼度量之和与权重向量的L2范数即可提高模型泛化能力;在此基础上,将LS-SVM模型选择转换为一个多目标优化问题,引入多目标粒子群算法选取最优超参数.采用仿真与真实UCI数据集对所提方法进行了对比实验,结果表明,与传统LS-SVM与基于留一法模型选择的LS-SVM相比,所提方法可以取得更小的泛化误差,同时数值稳定性更好.

Model Selection of Least Squares Support Vector Machine Based on Riemannian Metric

Least squares support vector machine(LS-SVM) cannot guarantee the performance of model selection and tend to be over-fitting in many cases.In this paper,a new model selection approach of LS-SVM is proposed based on Riemannian metric.First,based on the information geometry,this paper proves the generalization performance of LS-SVM can be characterized by Riemannian metric.This paper further proves the generalization ability can be developed by minimizing the Frobenius norm of Riemannian metric tensor and L2 norm of weight vector.Finally,multi-objective particle swarm optimization algorithm is introduced to solve this problem.Experiments on simulated and real-life UCI data sets are conducted,demonstrating that,compared with the conventional LS-SVM and leave-one-out cross validation,the proposed approach generally obtains better generalization performance in terms of generalization error and numerical stability.