Analysis of preprocessing vs. cost-sensitive learning for imbalanced classification. Open problems on intrinsic data characteristics

Class imbalance is among the most persistent complications which may confront the traditional supervised learning task in real-world applications. The problem occurs, in the binary case, when the number of instances in one class significantly outnumbers the number of instances in the other class. This situation is a handicap when trying to identify the minority class, as the learning algorithms are not usually adapted to such characteristics.The approaches to deal with the problem of imbalanced datasets fall into two major categories: data sampling and algorithmic modification. Cost-sensitive learning solutions incorporating both the data and algorithm level approaches assume higher misclassification costs with samples in the minority class and seek to minimize high cost errors. Nevertheless, there is not a full exhaustive comparison between those models which can help us to determine the most appropriate one under different scenarios.The main objective of this work is to analyze the performance of data level proposals against algorithm level proposals focusing in cost-sensitive models and versus a hybrid procedure that combines those two approaches. We will show, by means of a statistical comparative analysis, that we cannot highlight an unique approach among the rest. This will lead to a discussion about the data intrinsic characteristics of the imbalanced classification problem which will help to follow new paths that can lead to the improvement of current models mainly focusing on class overlap and dataset shift in imbalanced classification.     

针对不平衡数据的过采样和随机森林改进算法

针对数据不平衡带来的少数类样本识别率低的问题,提出通过加权策略对过采样和随机森林进行改进的算法,从数据预处理和算法两个方面降低数据不平衡对分类器的影响。数据预处理阶段应用合成少数类过采样技术(Synthetic Minority Oversampling Technique,SMOTE)降低数据不平衡度,每个少数类样本根据其相对于剩余样本的欧氏距离分配权重,使每个样本合成不同数量的新样本。算法改进阶段利用Kappa系数评价随机森林中决策树训练后的分类效果,并赋予每棵树相应的权重,使分类能力更好的树在投票阶段有更大的投票权,提高随机森林算法对不平衡数据的整体分类性能。在KEEL数据集上的实验表明,与未改进算法相比,改进后的算法对少数类样本分类准确率和整体样本分类性能有所提升。     

一种基于GMM-EM的非平衡数据的概率增强算法

非平衡数据的分类问题是机器学习领域的一个重要研究课题.在一个非平衡数据里,少数类的训练样本明显少于多数类,导致分类结果往往偏向多数类.针对非平衡数据分类问题,提出一种基于高斯混合模型-均值最大化方法(GMM-EM)的概率增强算法.首先,通过高斯混合模型(GMM)与均值最大化算法(EM)建立少数类数据的概率密度函数;其次,根据高概率密度的样本生成新样本的能力比低概率密度的样本更强的性质,建立一种基于少数类样本密度函数的过采样算法,该算法保证少数类数据集在平衡前后的概率分布的一致性,从数据集的统计性质使少数类达到平衡;最后,使用决策树分类器对已经达到平衡的数据集进行分类,并且利用评价指标对分类效果进行评判.通过从UCI和KEEL数据库选出的8组数据集的分类实验,表明了所提出算法比现有算法更有效.     

一种基于特征选择的不平衡数据分类算法

不平衡数据分类是当前机器学习的研究热点,传统分类算法通常基于数据集平衡状态的前提,不能直接应用于不平衡数据的分类学习.针对不平衡数据分类问题,文章提出一种基于特征选择的改进不平衡分类提升算法,从数据集的不同类型属性来权衡对少数类样本的重要性,筛选出对有效预测分类出少数类样本更意义的属性,同时也起到了约减数据维度的目的.然后结合不平衡分类算法使数据达到平衡状态,最后针对原始算法错分样本权值增长过快问题提出新的改进方案,有效抑制权值的增长速度.实验结果表明,该算法能有效提高不平衡数据的分类性能,尤其是少数类的分类性能.     

Multi-window based ensemble learning for classification of imbalanced streaming data

Imbalanced streaming data is commonly encountered in real-world data mining and machine learning applications, and has attracted much attention in recent years. Both imbalanced data and streaming data in practice are normally encountered together; however, little research work has been studied on the two types of data together. In this paper, we propose a multi-window based ensemble learning method for the classification of imbalanced streaming data. Three types of windows are defined to store the current batch of instances, the latest minority instances, and the ensemble classifier. The ensemble classifier consists of a set of latest sub-classifiers, and the instances employed to train each sub-classifier. All sub-classifiers are weighted prior to predicting the class labels of newly arriving instances, and new sub-classifiers are trained only when the precision is below a predefined threshold. Extensive experiments on synthetic datasets and real-world datasets demonstrate that the new approach can efficiently and effectively classify imbalanced streaming data, and generally outperforms existing approaches.     

面向不平衡数据集的改进型SMOTE算法

针对SMOTE（synthetic minority over-sampling technique）在合成少数类新样本时存在的不足,提出了一种改进的SMOTE算法GA-SMOTE。该算法的关键将是遗传算法中的3个基本算子引入到SMOTE中,利用选择算子实现对少数类样本有区别的选择,使用交叉、变异算子实现对合成样本质量的控制.结合GA-SMOTE与SVM（support vector machine）算法来处理不平衡数据的分类问题.UCI数据集上的大量实验表明,GA-SMOTE在新样本的整体合成效果上表现出色,有效提高了SVM在不平衡数据集上的分类性能。     

多任务学习的不平衡SVM+算法

处理不平衡数据分类时,传统支持向量机技术（SVM）对少数类样本识别率较低。鉴于SVM+技术能利用样本间隐藏信息的启发,提出了多任务学习的不平衡SVM+算法（MTL-IC-SVM+）。MTL-IC-SVM+基于SVM+将不平衡数据的分类表示为一个多任务的学习问题,并从纠正分类面的偏移出发,分别赋予多数类和少数类样本不同的错分惩罚因子,且设置少数类样本到分类面的距离大于多数类样本到分类面的距离。UCI数据集上的实验结果表明,MTL-IC-SVM+在不平衡数据分类问题上具有较高的分类精度。     

基于改进SMOTE的非平衡数据集分类研究

针对SMOTE(Synthetic Minority Over-sampling Technique)在合成少数类新样本时存在的不足,提出了一种改进的SMOTE算法(SSMOTE)。该算法的关键是将支持度概念和轮盘赌选择技术引入到SMOTE中,并充分利用了异类近邻的分布信息,实现了对少数类样本合成质量和数量的精细控制。将SSMOTE与KNN(K-Nearest Neighbor)算法结合来处理不平衡数据集的分类问题。通过在UCI数据集上与其他重要文献中的相关算法进行的大量对比实验表明,SSMOTE在新样本的整体合成效果上表现出色,有效提高了KNN在非平衡数据集上的分类性能。     

一种基于聚类提升的不平衡数据分类算法

不平衡数据分类是机器学习研究领域中的一个热点问题。针对传统分类算法处理不平衡数据的少数类识别率过低问题,文章提出了一种基于聚类的改进AdaBoost分类算法。算法首先进行基于聚类的欠采样,在多数类样本上进行K均值聚类,之后提取聚类质心,与少数类样本数目一致的聚类质心和所有少数类样本组成新的平衡训练集。为了避免少数类样本数量过少而使训练集过小导致分类精度下降,采用少数过采样技术过采样结合聚类欠采样。然后,借鉴代价敏感学习思想,对AdaBoost算法的基分类器分类误差函数进行改进,赋予不同类别样本非对称错分损失。实验结果表明,算法使模型训练样本具有较高的代表性,在保证总体分类性能的同时提高了少数类的分类精度。     

基于各类支持度阈值独立挖掘的关联改进算法

关联分类及较多的改进算法很难同时既具有较高的整体准确率又有较好的小类分类性能。针对此问题,提出了一种基于类支持度阈值独立挖掘的关联分类改进算法—ACCS。ACCS算法的主要特点是:(1)根据训练集中各类数量大小给出每个类类支持度阈值的设定方法,并基于各类的类支持度阈值独立挖掘该类的关联分类规则,尽量使小类生成更多高置信度的规则;(2)采用类支持度对置信度相同的规则排序,提高小类规则的优先级;(3)用综合考虑置信度和提升度的新的规则度量预测未知实例。在多个数据集上的实验结果表明,相比多种关联分类改进算法,ACCS算法有更高的整体分类准确率,且在不平衡数据上也能取得较好的小类分类性能。     

融合SLDA主题模型的不均衡文本分类方法

在标签均衡分布且标注样本足够多的数据集上,监督式分类算法通常可以取得比较好的分类效果。然而,在实际应用中样本的标签分布通常是不均衡的,分类算法的分类性能就变得比较差。为此,结合SLDA（Supervised LDA）有监督主题模型,提出一种不均衡文本分类新算法ITC-SLDA（Imbalanced Text Categorization based on Supervised LDA）。基于SLDA主题模型,建立主题与稀少类别之间的精确映射,以提高少数类的分类精度。利用SLDA模型对未标注样本进行标注,提出一种新的未标注样本的置信度计算方法,以及类别约束的采样策略,旨在有效采样未标注样本,最终降低不均衡文本的倾斜度,提升不均衡文本的分类性能。实验结果表明,所提方法能明显提高不均衡文本分类任务中的Macro-F1和G-mean值。     

SMOTE过采样及其改进算法研究综述

近年来不平衡分类问题受到广泛关注。SMOTE过采样通过添加生成的少数类样本改变不平衡数据集的数据分布,是改善不平衡数据分类模型性能的流行方法之一。本文首先阐述了SMOTE的原理、算法以及存在的问题,针对SMOTE存在的问题,分别介绍了其4种扩展方法和3种应用的相关研究,最后分析了SMOTE应用于大数据、流数据、少量标签数据以及其他类型数据的现有研究和面临的问题,旨在为SMOTE的研究和应用提供有价值的借鉴和参考。     

Imbalanced classification of mental workload using a cost-sensitive majority weighted minority oversampling strategy

Identifying the temporal variations in mental workload level (MWL) is crucial for enhancing the safety of human–machine system operations, especially when there is cognitive overload or inattention of human operator. This paper proposed a cost-sensitive majority weighted minority oversampling strategy to address the imbalanced MWL data classification problem. Both the inter-class and intra-class imbalance problems are considered. For the former, imbalance ratio is defined to determine the number of the synthetic samples in the minority class. The latter problem is addressed by assigning different weights to borderline samples in the minority class based on the distance and density meaures of the sample distribution. Furthermore, multi-label classifier is designed based on an ensemble of binary classifiers. The results of analyzing 21 imbalanced UCI multi-class datasets showed that the proposed approach can effectively cope with the imbalanced classification problem in terms of several performance metrics including geometric mean (G-mean) and average accuracy (ACC). Moreover, the proposed approach was applied to the analysis of the EEG data of eight experimental participants subject to fluctuating levels of mental workload. The comparative results showed that the proposed method provides a competing alternative to several existing imbalanced learning algorithms and significantly outperforms the basic/referential method that ignores the imbalance nature of the dataset.     

非平衡数据集分类方法探讨

由于数据集中类分布极不平衡,很多分类算法在非平衡数据集上失效,而非平衡数据集中占少数的类在现实生活中通常具有显著意义,因此如何提高非平衡数据集中少数类的分类性能成为近年来研究的热点。详细讨论了非平衡数据集分类问题的本质、影响非平衡数据集分类的因素、非平衡数据集分类通常采用的方法、常用的评估标准以及该问题中存在的问题与挑战。     

一种用于非平衡数据分类的集成学习模型

针对非平衡数据分类问题,提出了一种改进的SVM-KNN分类算法,在此基础上设计了一种集成学习模型.该模型采用限数采样方法对多数类样本进行分割,将分割后的多数类子簇与少数类样本重新组合,利用改进的SVM-KNN分别训练,得到多个基本分类器,对各个基本分类器进行组合.采用该模型对UCI数据集进行实验,结果显示该模型对于非平衡数据分类有较好的效果.     

邻域自适应SMOTE算法研究

针对SMOTE(synthetic minority over-sampling technique)等基于近邻值的传统过采样算法在处理类不平衡数据时近邻参数不能根据少数类样本的分布及时调整的问题,提出邻域自适应SMOTE算法AdaN_SMOTE.为使合成数据保留少数类的原始分布,跟踪精度下降点确定每个少数类数据的近邻值,并根据噪声、小析取项或复杂的形状及时调整近邻值的大小;合成数据保留了少数类的原始分布,算法分类性能更佳.在KE E L数据集上进行实验对比验证,结果表明AdaN_SMOTE分类性能优于其他基于近邻值的过采样方法,且在有噪声的数据集中更有效.     

一种改善非平衡分布数据SVM分类能力的新策略

支持向量机利用接近边界的少数向量来构造一个最优分类面。但是若两分类问题中的样本呈现非平衡分布时，即两类样本数目相差很大时，分类能力就会有所下降。提出分别使用重复数量少的一类样本、选择数量多的类样本以及引入类惩罚因子的三个方法来改善分类能力。实验表明，三种方法对不同类型数据集合，一定程度上都改善了支持向量的分类能力。     

An Imbalanced Dataset and Class Overlapping Classification Model for Big Data

Most modern technologies, such as social media, smart cities, and the internet of things (IoT), rely on big data. When big data is used in the real-world applications, two data challenges such as class overlap and class imbalance arises. When dealing with large datasets, most traditional classifiers are stuck in the local optimum problem. As a result, it’s necessary to look into new methods for dealing with large data collections. Several solutions have been proposed for overcoming this issue. The rapid growth of the available data threatens to limit the usefulness of many traditional methods. Methods such as oversampling and undersampling have shown great promises in addressing the issues of class imbalance. Among all of these techniques, Synthetic Minority Oversampling TechniquE (SMOTE) has produced the best results by generating synthetic samples for the minority class in creating a balanced dataset. The issue is that their practical applicability is restricted to problems involving tens of thousands or lower instances of each. In this paper, we have proposed a parallel mode method using SMOTE and MapReduce strategy, this distributes the operation of the algorithm among a group of computational nodes for addressing the aforementioned problem. Our proposed solution has been divided into three stages. The first stage involves the process of splitting the data into different blocks using a mapping function, followed by a pre-processing step for each mapping block that employs a hybrid SMOTE algorithm for solving the class imbalanced problem. On each map block, a decision tree model would be constructed. Finally, the decision tree blocks would be combined for creating a classification model. We have used numerous datasets with up to 4 million instances in our experiments for testing the proposed scheme’s capabilities. As a result, the Hybrid SMOTE appears to have good scalability within the framework proposed, and it also cuts down the processing time.     

面向不均衡数据集的ISMOTE算法

为了提高不均衡数据集中少数类的分类性能,提出ISMOTE算法。它是在少数类实例及其最近邻少数类实例构成的n维球体内进行随机插值,从而来改进数据分布的不均衡程度。通过实际数据集上的实验,与SMOTE算法和直接分类不均衡数据算法的性能比较结果表明,ISMOTE算法具有更高的分类精度,可以有效地改进分类器的性能。     

基于边界混合采样的非均衡数据处理算法

针对非均衡数据分类效果差的问题,提出一种新的基于边界混合采样的非均衡数据处理方法(BMS).首先通过引进“变异系数”找出样本的边界域和非边界域;然后对边界域中的少数类样本进行过采样,对非边界域中的多数类样本进行随机欠采样,以期达到训练数据基本平衡的目标.实验结果表明,BMS方法比其他3种流行的非均衡数据处理方法在对7个公开数据集的分类性能上平均提高了5%左右,因此,该方法可以广泛应用于非均衡数据的处理和分类中.