基于Spark框架的用于金融信贷风险控制的加权随机森林算法

为解决互联网时代线上贷款业务量庞大带来的困扰,优化快速迭代的数据模型,从线上贷款业务的特点出发,以Spark分布式计算引擎为核心设计并实现了能够并行处理非平衡数据的加权随机森林算法.该算法从特征切分点抽样统计、特征分箱、逐层训练三个角度对加权随机森林算法进行并行化优化.该算法有效提高了随机森林算法的分类准确率,同时有效降低了决策过程中出现的平局现象.对非平衡数据,该文章通过SMOTE算法对数据进行重构,较好的保留了原有数据集信息.实验表明,该算法能够有效提高放贷效率性与及时性,极大的提高了生产力.

Weighted Random Forest Algorithm Based on Spark Framework for Financial Credit Risk Control

In order to solve the problems caused by huge online loan business in the Internet era and optimize the fast iterative data model,starting from the characteristics of online load business,based on Spark distributed computing engine,a weighted random forest algorithm capable of parallel processing unbalanced data is designed and implemented.In this algorithm,the weighted random forest algorithm is parallelized and optimized from three aspects:sampling statistics of feature points,feature box,and layer by layer training.This algorithm improves the classification accuracy of the random forest algorithm and reduces the phenomenon of tie in the decisionmaking processing effectively.For the unbalanced data,SMOTE algorithm is used to reconstruct the data in the paper,and the original data set information is preserved well.Experiments show that the system can effectively improve the effectively and timeliness during the lending,and greatly improve the productivity.