软件缺陷预测模型间的比较实验: 问题、进展与挑战

近年来,研究者提出了大量的软件缺陷预测模型,新模型往往通过与过往模型进行比较实验来表明其有效性.然而,研究者在设计新旧模型间的比较实验时并没有达成共识,不同的工作往往采用不完全一致的比较实验设置,这可能致使在对比模型时得到误导性结论,最终错失提升缺陷预测能力的机会.对近年来国内外学者所做的缺陷预测模型间的比较实验进行系统性的总结:首先,阐述缺陷预测模型间的比较实验的研究问题;然后,分别从缺陷数据集、数据集划分、基线模型、性能指标、分类阈值这5个方面对现有的比较实验进行总结;最后,指出目前在进行缺陷预测模型间比较实验时面临的挑战,并给出建议的研究方向.

Comparing Software Defect Prediction Models: Research Problem, Progress, and Challenges

In recent years, a large number of software defect prediction models have been proposed. Once a new defect prediction model is proposed, it is often compared with previous defect prediction models to evaluate its effectiveness. However, there is no consensus on how to compare the newly proposed defect prediction model with previous defect prediction models. Different studies often adopt different settings for comparison, which may lead to misleading conclusions in the comparisons of prediction models, and consequently lead to missing the opportunity to improve the effectiveness of defect prediction. This study systematically reviews the comparative experiments of software defect prediction models conducted by worldwide scholars in recent years. First, the comparisons of defect prediction models are introduced. Then, the research progress is summarized from the perspectives of defect dataset, dataset split, baseline models, performance indicators, and classification thresholds, respectively, in the comparisons. Finally, the opportunities and challenges are summarized in comparative experiments of defect prediction models and the research directions in the future are outlined.