基于输入K-近邻的正则化路径上SVR贝叶斯组合

在ε-不敏感支持向量回归(ε-insensitive support vector regression,ε-SVR)正则化路径的基础上,提出基于输入K-近邻的三步式SVR模型组合方法。在整个样本集上进行训练,求得ε-SVR的正则化路径。由SVR正则化路径的分段线性性质确定初始模型集合,并应用平均贝叶斯信息准则(Bayesian Information Criterion,BIC)策略对初始模型集合进行修剪以获得候选模型集合。该修剪策略可减小候选模型集合的规模,提高模型组合的计算效率和预测性能。在预测或测试阶段,根据样本输入向量采用K-近邻法确定最终组合模型集合,并实现贝叶斯组合预测。证明了ε-SVR模型组合的Lε-风险一致性,给出了SVR模型组合基于样本的合理性解释。试验结果验证了正则化路径上基于输入K-近邻的ε-SVR模型组合的有效性。

Bayesian combination of SVR on regularization path based on KNN of input

A model combination method of ε-insensitive support vector regression(ε-SVR)based on regularization path with K-Nearest Neighbor(KNN)of input was proposed. The model set was constructed with ε-SVR regularization path, which was trained by using the same original training set. The initial model set was obtained according to the piecewise linearity of SVR regularization path. The average of Bayesian Information Criterion(BIC)was applied to exclude models with poor performance and prune the initial model set. In the testing or predicting phase, the combination model set was determined with the KNN method, and Bayesian combination was performed. The pruning policy improves not only the computational efficiency of model combination but also the generalization performance. The L_ε-risk consistency for model combination of ε-SVR was defined and proved, which gave the mathematical foundation of the proposed method. Experimental results demonstrated the effectiveness and efficiency of the Bayesian combination of ε-SVR on regularization path.