旋转森林与极限学习相结合的遥感影像分类方法

针对旋转森林算法(rotation forest,RF)处理遥感影像分类时容易出现过拟合现象,以及极限学习算法(extreme learning machine,ELM)泛化性能较差问题,提出一种将旋转森林与极限学习相结合(RF-ELM)的影像分类算法。该方法首先用旋转森林算法对基分类器进行训练,然后利用极限学习算法作为基分类器解决旋转森林中存在的过拟合问题。通过利用Landsat-8遥感影像分别对比RF、ELM、Bag-ELM和RF-ELM进行分类实验。结果表明,所提出的集成方法比RF、ELM单一算法具有更高的分类精度,相比Bag-ELM具有更高泛化能力,有效改善了分类过拟合现象,计算效率也继承了ELM快速运算的特点。     

基于ELM和FOA的股票价格预测

针对股票价格预测中应用极限学习机预测存在稳定性不理想的问题,提出了一种改进果蝇优化极限学习机（IFOA-ELM）预测模型的算法。在该算法中,果蝇群通过不断调整群半径来优化ELM的输入层与隐含层连接权值和隐含层阈值,并以优化后的结果为基础,构建ELM预测模型。将IFOA-ELM模型用于股票价格预测。实验表明,与ELM和FOA-ELM相比,IFOA-ELM在股票价格预测中具有更高的预测精度和更好的稳定性。     

基于旋转森林的分类器集成算法研究

为提高决策树的集成分类精度,介绍了一种基于特征变换的旋转森林分类器集成算法,通过对数据属性集的随机分割,并在属性子集上对抽取的子样本数据进行主成分分析,以构造新的样本数据,达到增大基分类器差异性及提高预测准确率的目的。在Weka平台下,分别采用Bagging、AdaBoost及旋转森林算法对剪枝与未剪枝的J48决策树分类算法进行集成的对比试验,以10次10折交叉验证的平均准确率为比较依据。结果表明旋转森林算法的预测精度优于其他两个算法,验证了旋转森林是一种有效的决策树分类器集成算法。     

基于输出不一致测度的极限学习机集成的基因表达数据分类

选择性集成学习已经成为分析基因表达数据、获取生物学信息的有力工具.为了更好地挖掘基因表达数据,利用极限学习机的集成,克服单个ELM用于数据分类时性能欠稳定的缺点,文中提出了一种基于输出不一致测度的ELM相异性集成算法(D-D-ELM).算法首先以输出不一致测度为标准对多个ELM模型进行相异性判断,其次根据ELM的平均分类精度剔除掉相应的模型,最后对筛选后的分类模型用多数投票法进行集成.算法被运用到Breast、Leukemia、Colon、Heart基因表达数据上,并通过理论和实验得到验证.实验结果的统计学分析表明D-D-ELM能够以更少的模型数量达到较稳定的分类精度.     

基于标记分布学习的异态集成学习算法

为了提高预测的准确性,文中结合机器学习中堆积(Stacking)集成框架,组合多个分类器对标记分布进行学习,提出基于标记分布学习的异态集成学习算法(HELA-LDL).算法构造两层模型框架,通过第一层结构将样本数据采用组合方式进行异态集成学习,融合各分类器的学习结果,将融合结果输入到第二层分类器,预测结果是带有置信度的标记分布.在专用数据集上的对比实验表明,HELA-LDL可以发挥各种算法在不同场景下的性能较优,稳定性分析进一步说明算法的有效性.     

基于改进极限学习机的泥石流发生预测

为提高泥石流预测预报的准确性,提出一种基于DBSCAN聚类的改进极限学习机(ELM)算法。首先,利用DBSCAN算法对泥石流发生训练的数据进行聚类处理;其次,将聚类得到的不同训练集分类训练ELM分类器;最后,利用ELM分类器对预测集数据进行预测。实验结果表明,利用改进ELM算法对泥石流发生预测的平均准确率达到91.6%,改进ELM算法的稳定性与传统ELM算法相比有明显提高,与传统ELM算法、BP神经网络和Fisher预测法相比,改进ELM算法的预测精度更高。     

基于隐特征空间的极限学习机模型选择

针对极限学习机（ELM）中冗余的隐神经元会削弱模型泛化能力的缺点,提出了一种基于隐特征空间的ELM模型选择算法。首先,为了寻找合适的ELM隐层,在ELM中添加正则项,该项为现有隐层空间到低维隐特征空间的映射函数矩阵的Frobenius范数;其次,为解决该非凸问题,采用交替优化的策略,并通过凸二次型优化学习该隐空间;最终自适应得到最优映射函数和ELM模型。分别采用UCI标准数据集和载荷识别工程数据对所提算法进行测试,结果表明,与经典ELM相比,该算法可有效提高预测精度和数值稳定性,与现有模型选择算法相比,该算法预测精度相当,但运行时间则大幅降低。     

深度森林在用户购买预测中的应用

针对电商大数据时代用户未来购买行为预测，在京东平台真实数据集上，提出时间滑动窗口技术和窗口权重递减设置，从五方面构建整体用户行为特征，综合考虑深度学习的表征学习能力和集成学习的训练效率，引入多层异源集成算法，将随机森林、XGBoost等多种算法进行组合，搭建基于深度森林模型的用户购买行为预测算法框架，实现准确高效的用户购买预测结果。算法训练时间为68 s,预测准确率达89.3%,相对于集成学习算法和深度神经网络模型取得了更好的效果。     

集成学习算法的研究与应用

集成学习算法的思想就是集成多个学习器,并组合它们的预测结果,以形成最终的结论。典型的学习模型组合方法有投票法,专家混合方法,堆叠泛化法与级联法,但这些方法的性能都有待进一步提高。提出了一种新颖的集成学习算法--增强的集成学习算法（ReinforcedEnsemble）。ReinforcedEnsemble集成算法由两大部分组成：ReinforcedEnsemble特征提取算法与ReinforcedEnsemble基分类器。通过实验,将ReinforcedEnsemble算法与其他集成学习算法进行了性能比较。实验结果表明,所提出的算法在多项指标上均达到最优。     

基于CMAES集成学习方法的地表水质分类

为了提高人民生活质量,政府部门不断加强水质管理,然而人工分类方法无法满足实时处理的需求,传统机器学习方法的分类准确率又不够高。集成学习使用多种学习算法来获得比单一学习算法更好的预测性能。首先,对集成学习进行概述,简要介绍了Bagging和Boosting算法,并提出基于协方差自适应调整的进化策略算法(CMAES)的集成学习方法。接着,介绍了数据处理方式、模型评估方法和评价指标。最后,用CMAES集成学习方法对逻辑回归、线性判别分析、支持向量机、决策树、完全随机树、朴素贝叶斯、K-邻近算法、随机森林、完全随机树林、深度级联森林十种模型进行集成。实验结果表明,CMAES集成学习方法优于所有其他模型,该方法将继续被应用到未来的研究之中。     

Random forests-based extreme learning machine ensemble for multi-regime time series prediction

Accurate and timely predicting values of performance parameters are currently strongly needed for important complex equipment in engineering. In time series prediction, two problems are urgent to be solved. One problem is how to achieve the accuracy, stability and efficiency together, and the other is how to handle time series with multiple regimes. To solve these two problems, random forests-based extreme learning machine ensemble model and a novel multi-regime approach are proposed respectively, and these two approaches can be integrated to achieve better performance. First, the extreme learning machine (ELM) is used in the proposed model because of its efficiency. Then the regularized ELM and ensemble learning strategy are used to improve generalization performance and prediction accuracy. The bootstrap sampling technique is used to generate training sample sets for multiple base-level ELM models, and then the random forests (RF) model is used as the combiner to aggregate these ELM models to achieve more accurate and stable performance. Next, based on the specific properties of turbofan engine time series, a multi-regime approach is proposed to handle it. Regimes are first separated, then the proposed RF-based ELM ensemble model is used to learn models of all regimes, individually, and last, all the learned regime models are aggregated to predict performance parameter at the future timestamp. The proposed RF-based ELM ensemble model and multi-regime approaches are evaluated by using NN3 time series and NASA turbofan engine time series, and then the proposed model is applied to the exhaust gas temperature prediction of CFM engine. The results demonstrate that the proposed RF-based ELM ensemble model and multi-regime approach can be accurate, stable and efficient in predicting multi-regime time series, and it can be robust against overfitting.     

Decay-weighted extreme learning machine for balance and optimization learning

The original extreme learning machine (ELM) was designed for the balanced data, and it balanced misclassification cost of every sample to get the solution. Weighted extreme learning machine assumed that the balance can be achieved through the equality of misclassification costs. This paper improves previous weighted ELM with decay-weight matrix setting for balance and optimization learning. The decay-weight matrix is based on the sample number of each class, but the weight sum values of each class are not necessarily equal. When the number of samples is reduced, the weight sum is also reduced. By adjusting the decaying velocity, classifier could achieve more appropriate boundary position. From the experimental results, the decay-weighted ELM obtains the better effects in solving the imbalance classification tasks, particularly in multiclass tasks. This method was successfully applied to build the prediction model in the urban traffic congestion prediction system.     

Nonlinear regression in environmental sciences using extreme learning machines: A comparative evaluation

The extreme learning machine (ELM), a single-hidden layer feedforward neural network algorithm, was tested on nine environmental regression problems. The prediction accuracy and computational speed of the ensemble ELM were evaluated against multiple linear regression (MLR) and three nonlinear machine learning (ML) techniques – artificial neural network (ANN), support vector regression and random forest (RF). Simple automated algorithms were used to estimate the parameters (e.g. number of hidden neurons) needed for model training. Scaling the range of the random weights in ELM improved its performance. Excluding large datasets (with large number of cases and predictors), ELM tended to be the fastest among the nonlinear models. For large datasets, RF tended to be the fastest. ANN and ELM had similar skills, but ELM was much faster than ANN except for large datasets. Generally, the tested ML techniques outperformed MLR, but no single method was best for all the nine datasets.     

Dynamic ensemble extreme learning machine based on sample entropy

Extreme learning machine (ELM) as a new learning algorithm has been proposed for single-hidden layer feed-forward neural networks, ELM can overcome many drawbacks in the traditional gradient-based learning algorithm such as local minimal, improper learning rate, and low learning speed by randomly selecting input weights and hidden layer bias. However, ELM suffers from instability and over-fitting, especially on large datasets. In this paper, a dynamic ensemble extreme learning machine based on sample entropy is proposed, which can alleviate to some extent the problems of instability and over-fitting, and increase the prediction accuracy. The experimental results show that the proposed approach is robust and efficient.     

An improved extreme learning machine with self-recurrent hidden layer

Extreme learning machine (ELM) is widely used in complex industrial problems, especially the online-sequential extreme learning machine (OS-ELM) plays a good role in industrial online modeling. However, OS-ELM requires batch samples to be pre-trained to obtain initial weights, which may reduce the timeliness of samples. This paper proposes a novel model for the online process regression prediction, which is called the Recurrent Extreme Learning Machine (Recurrent-ELM). The nodes between the hidden layers are connected in Recurrent-ELM, thus the input of the hidden layer receives both the information from the current input layer and the previously hidden layer. Moreover, the weights and biases of the proposed model are generated by analysis rather than random. Six regression applications are used to verify the designed Recurrent-ELM, compared with extreme learning machine (ELM), fast learning network (FLN), online sequential extreme learning machine (OS-ELM), and an ensemble of online sequential extreme learning machine (EOS-ELM), the experimental results show that the Recurrent-ELM has better generalization and stability in several samples. In addition, to further test the performance of Recurrent-ELM, we employ it in the combustion modeling of a 330 MW coal-fired boiler compared with FLN, SVR and OS-ELM. The results show that Recurrent-ELM has better accuracy and generalization ability, and the theoretical model has some potential application value in practical application.     

Variable activation function extreme learning machine based on residual prediction compensation

For solving the problem that extreme learning machine (ELM) algorithm uses fixed activation function and cannot be residual compensation, a new learning algorithm called variable activation function extreme learning machine based on residual prediction compensation is proposed. In the learning process, the proposed method adjusts the steep degree, position and mapping scope simultaneously. To enhance the nonlinear mapping capability of ELM, particle swarm optimization algorithm is used to optimize variable parameters according to root-mean square error for the prediction accuracy of the mode. For further improving the predictive accuracy, the auto-regressive moving average model is used to model the residual errors between actual value and predicting value of variable activation function extreme learning machine (V-ELM). The prediction of residual errors is used to rectify the prediction value of V-ELM. Simulation results verified the effectiveness and feasibility of this method by using Pole, Auto-Mpg, Housing, Diabetes, Triazines and Stock benchmark datasets. Also, it was implemented to develop a soft sensor model for the gasoline dry point in delayed coking and some satisfied results were obtained.     

A novel ensemble-based wrapper method for feature selection using extreme learning machine and genetic algorithm

This paper presents a novel wrapper feature selection algorithm for classification problems, namely hybrid genetic algorithm (GA)- and extreme learning machine (ELM)-based feature selection algorithm (HGEFS). It utilizes GA to wrap ELM to search for the optimum subsets in the huge feature space, and then, a set of subsets are selected to make ensemble to improve the final prediction accuracy. To prevent GA from being trapped in the local optimum, we propose a novel and efficient mechanism specifically designed for feature selection problems to maintain GA’s diversity. To measure each subset’s quality fairly and efficiently, we adopt a modified ELM called error-minimized extreme learning machine (EM-ELM) which automatically determines an appropriate network architecture for each feature subsets. Moreover, EM-ELM has good generalization ability and extreme learning speed which allows us to perform wrapper feature selection processes in an affordable time. In other words, we simultaneously optimize feature subset and classifiers’ parameters. After finishing the search process of GA, to further promote the prediction accuracy and get a stable result, we select a set of EM-ELMs from the obtained population to make the final ensemble according to a specific ranking and selecting strategy. To verify the performance of HGEFS, empirical comparisons are carried out on different feature selection methods and HGEFS with benchmark datasets. The results reveal that HGEFS is a useful method for feature selection problems and always outperforms other algorithms in comparison.     

结合期望风险的极限学习机的研究

对极限学习机的模型进行了研究,提出了一种结合期望风险最小化的极限学习机的预测模型。其基本思想是同时考虑结构风险和期望风险,根据期望风险和经验风险之间的关系,将期望风险转换成经验风险,进行最小化期望风险的极限学习机预测模型求解。利用人工数据集和实际数据集进行回归问题的数值实验,并与极限学习机（ELM）和正则极限学习机（RELM）两种算法的性能进行了比较,实验结果表明,所提方法能有效提高了泛化能力。     

基于极限学习机的航空发动机传感器故障诊断

针对当前应用于航空发动机传感器故障诊断中的基于梯度的传统学习算法多存在参数选择困难、容易陷入局部最小化、过拟合等问题,提出了基于极限学习机(ELM)的航空发动机传感器故障诊断方法。算法只需设置隐含层神经元的个数,能够较好地避免上述问题,缩短故障诊断时间、提升诊断精度。通过仿真试验表明:基于ELM算法所建的航空发动机传感器故障诊断模型要比基于BP神经网络算法所建的模型耗时短且精度高。