基于贝叶斯最优化的Xgboost算法的改进及应用

在使用Xgboost框架时，经常涉及各种参数的调整，并且参数组合的选取对模型的分类性能影响较大. 传统的参数寻优方法，通常先导出一个惩罚函数，然后运用经验或者穷举法调整参数值来最大化或最小化这个惩罚函数，但是经常会遇到某个模型没有一个显式的表达式情况. 这类模型的参数寻优就非常麻烦，同时又会给算法带来一定的不确定性和随机性. 本文基于高斯法（GP）的贝叶斯最优化算法对Xgboost框架进行参数寻优，提出了一种新的算法GP_Xgboost，并通过多组数值进行实验. 结果表明本文改进的算法分类效果要优于人工调优和穷举法，从而证明了该算法的可行性和有效性.

The Improvement and Application of Xgboost Method Based on the Bayesian Optimization

When the Xgboost framework is in use, it is often involved in the adjustment of various parameters, and the selection of parameters has a great influence on the classification performance of the model. The traditional parameter optimization method usually first derives a penalty function, and then the empirical or exhaustive method is used to adjust the parameter value to maximize or minimize the penalty function, but often encounters a model without an explicit expression. The optimization of the parameters of this model is very troublesome, also bringing some uncertainty and randomness to the algorithm. The Bayesian optimization algorithm based on Gaussian method (GP) is used to optimize the parameters of the Xgboost framework. A new algorithm, GP_Xgboost, is proposed and experimented by multiple sets of numerical values. The results show that the proposed algorithm is superior to the manual tuning and exhaustive method, which proves the feasibility and effectiveness of the proposed algorithm.