CODAS:一个易扩展的静态代码缺陷分析服务

利用静态代码缺陷分析技术对软件进行早期缺陷检测,是提高软件质量的重要途径。静态代码缺陷分析工具(如FINDBUGS,JLINT,ESC/JAVA,PMD,COVERITY等)已经被证实可以成功地识别出大量的软件潜在缺陷[1-3]。然而,这类工具在可用性和有效性方面的不足严重限制了它们的进一步广泛使用。可用性不足包括a)每个独立缺陷检测工具只擅于检测特定类型的缺陷,需要配合使用才能全面检测缺陷;b)每个缺陷检测工具的安装、配置和运行占用了用户大量的时间、精力。有效性不足包括静态缺陷分析结果往往存在大量误报,并且会包括许多不重要的(不会引起程序员修复行为的)缺陷报告。为了解决上述问题,提出并构建了一个易扩展的"静态代码缺陷分析"服务(Code Defect Analysis Service,CODAS)。CODAS基于一个高度可扩展的架构设计,对多个独立的缺陷检测工具进行了封装和集成,并对缺陷检测报告进行了有效汇总和排序,从而充分发挥了各个独立工具的优势,大大提升了静态缺陷分析工具的可用性和有效性。     

C/C++程序静态内存泄漏警报自动确认方法

内存泄漏是C/C++程序的一种常见的、难以发现的缺陷,一直困扰着软件开发者,尤其是针对长时间运行的程序或者系统软件,内存泄漏的后果十分严重.针对内存泄漏的检测,目前主要有静态分析和动态测试两种方法.动态测试实际运行程序,具有较大开销,同时依赖测试用例的质量;静态分析技术及自动化工具已经被学术界和工业界广泛运用于内存泄漏缺陷检测中,然而由于静态分析采取了保守的策略,其结果往往包含数量巨大的误报,需要通过进一步人工确认来甄别误报,但人工确认静态分析的结果耗时且容易出错,严重限制了静态分析技术的实用性.本文提出了一种基于混合执行测试的静态内存泄漏警报的自动化确认方法.首先,针对静态分析报告的目标程序中内存泄漏的静态警报,对目标程序进行控制流分析,并计算警报的可达性,形成制导信息;其次,基于警报制导信息对目标程序进行混合执行测试;最后,在混合执行测试过程中,监控追踪内存对象的状态,判定内存泄漏是否发生,对静态警报进行动态确认并分类.实验结果表明该方法可以对静态内存泄漏警报进行有效的分类,显著降低了人工确认的工作量.实验详情参见：http://ssthappy.github.io/memleak/.     

基于目标制导符号执行的静态缓冲区溢出警报自动确认技术

缓冲区溢出漏洞是一类严重的安全性缺陷。目前存在动态测试和静态分析技术来检测缓冲区溢出缺陷:动态测试技术的有效性取决于测试用例的设计,而且往往会引入执行开销;静态分析技术及自动化工具已经被广泛运用于缓冲区溢出缺陷检测中,然而静态分析由于采取了保守的策略,其结果往往包含数量巨大的误报,需要通过进一步人工确认来甄别误报,但人工确认静态分析的结果耗时且容易出错,严重限制了静态分析技术的实用性。符号执行技术使用符号代替实际输入,能系统地探索程序的状态空间并生成高覆盖度的测试用例。本文提出一种基于目标制导符号执行的静态缓冲区溢出警报确认方法,使用静态分析工具的输出结果作为目标,制导符号执行确认警报。我们的方法分为3步:首先在过程间控制流图中检测静态分析警报路径片段的可达性,并将可达的警报路径片段集合映射为用于确认的完整确认路径集合;其次在符号执行中通过修剪与溢出缺陷疑似语句无关的路径,指导符号执行沿特定确认路径执行;最后在溢出缺陷疑似语句收集路径约束并加入溢出条件,通过约束求解的结果,对静态分析的警报进行分类。基于上述方法我们实现了原型工具BOVTool,实验结果表明在实际开源程序上BOVTool能够代替人工减少检查59.9%的缓冲区溢出误报。     

Automatic Buffer Overflow Warning Validation

Static buffer overflow detection techniques tend to report too many false positives fundamentally due to the lack of software execution information. It is very time consuming to manually inspect all the static warnings. In this paper, we propose BovInspector, a framework for automatically validating static buffer overflow warnings and providing suggestions for automatic repair of true buffer overflow warnings for C programs. Given the program source code and the static buffer overflow warnings, BovInspector first performs warning reachability analysis. Then, BovInspector executes the source code symbolically under the guidance of reachable warnings. Each reachable warning is validated and classified by checking whether all the path conditions and the buffer overflow constraints can be satisfied simultaneously. For each validated true warning, BovInspector provides suggestions to automatically repair it with 11 repair strategies. BovInspector is complementary to prior static buffer overflow discovery schemes. Experimental results on real open source programs show that BovInspector can automatically validate on average 60% of total warnings reported by static tools.

     

Improving software security with static automated code analysis in an industry setting

Software security can be improved by identifying and correcting vulnerabilities. In order to reduce the cost of rework, vulnerabilities should be detected as early and efficiently as possible. Static automated code analysis is an approach for early detection. So far, only few empirical studies have been conducted in an industrial context to evaluate static automated code analysis. A case study was conducted to evaluate static code analysis in industry focusing on defect detection capability, deployment, and usage of static automated code analysis with a focus on software security. We identified that the tool was capable of detecting memory related vulnerabilities, but few vulnerabilities of other types. The deployment of the tool played an important role in its success as an early vulnerability detector, but also the developers perception of the tools merit. Classifying the warnings from the tool was harder for the developers than to correct them. The correction of false positives in some cases created new vulnerabilities in previously safe code. With regard to defect detection ability, we conclude that static code analysis is able to identify vulnerabilities in different categories. In terms of deployment, we conclude that the tool should be integrated with bug reporting systems, and developers need to share the responsibility for classifying and reporting warnings. With regard to tool usage by developers, we propose to use multiple persons (at least two) in classifying a warning. The same goes for making the decision of how to act based on the warning. Copyright © 2012 John Wiley & Sons, Ltd.     

On the value of static analysis for fault detection in software

No single software fault-detection technique is capable of addressing all fault-detection concerns. Similarly to software reviews and testing, static analysis tools (or automated static analysis) can be used to remove defects prior to release of a software product. To determine to what extent automated static analysis can help in the economic production of a high-quality product, we have analyzed static analysis faults and test and customer-reported failures for three large-scale industrial software systems developed at Nortel Networks. The data indicate that automated static analysis is an affordable means of software fault detection. Using the orthogonal defect classification scheme, we found that automated static analysis is effective at identifying assignment and checking faults, allowing the later software production phases to focus on more complex, functional, and algorithmic faults. A majority of the defects found by automated static analysis appear to be produced by a few key types of programmer errors and some of these types have the potential to cause security vulnerabilities. Statistical analysis results indicate the number of automated static analysis faults can be effective for identifying problem modules. Our results indicate static analysis tools are complementary to other fault-detection techniques for the economic production of a high-quality software product.     

基于缺陷关联的静态分析优化

缺陷检测一般包括静态分析与人工审查两个阶段.静态检测工具报告大量缺陷,但是主要的缺陷确认工作仍由人工完成,这是一件费时、费力的工作.巨大的审查开销可能会导致软件开发人员拒绝使用该静态缺陷检测工具.提出一种可靠的基于缺陷关联的静态分析优化方法,能够分组静态检测工具所报告的缺陷,在分组后的任意一组缺陷中,如果其主导缺陷被证明是误报(或者是真实的),就能确认其他缺陷也是误报(也是真实的).实验结果表明,基于缺陷关联的静态分析优化方法在较小的时间和空间开销下减少了22%的缺陷审查工作,能够较好地适应于大型的关键嵌入式系统程序缺陷检测.     

Bug localization using latent Dirichlet allocation

ContextSome recent static techniques for automatic bug localization have been built around modern information retrieval (IR) models such as latent semantic indexing (LSI). Latent Dirichlet allocation (LDA) is a generative statistical model that has significant advantages, in modularity and extensibility, over both LSI and probabilistic LSI (pLSI). Moreover, LDA has been shown effective in topic model based information retrieval. In this paper, we present a static LDA-based technique for automatic bug localization and evaluate its effectiveness.ObjectiveWe evaluate the accuracy and scalability of the LDA-based technique and investigate whether it is suitable for use with open-source software systems of varying size, including those developed using agile methods.MethodWe present five case studies designed to determine the accuracy and scalability of the LDA-based technique, as well as its relationships to software system size and to source code stability. The studies examine over 300 bugs across more than 25 iterations of three software systems.ResultsThe results of the studies show that the LDA-based technique maintains sufficient accuracy across all bugs in a single iteration of a software system and is scalable to a large number of bugs across multiple revisions of two software systems. The results of the studies also indicate that the accuracy of the LDA-based technique is not affected by the size of the subject software system or by the stability of its source code base.ConclusionWe conclude that an effective static technique for automatic bug localization can be built around LDA. We also conclude that there is no significant relationship between the accuracy of the LDA-based technique and the size of the subject software system or the stability of its source code base. Thus, the LDA-based technique is widely applicable.     

Extending the reflexion method for consolidating software variants into product lines

Software variants emerge from ad-hoc copying in-the-large with adaptations to a specific context. As the number of variants increases, maintaining such software variants becomes more and more difficult and expensive. In contrast to such ad-hoc reuse, software product lines offer organized ways of reuse, taking advantage of similarities of different products. To re-gain control, software variants may be consolidated as organized software product lines. In this paper, we describe a method and supporting tools to compare software variants at the architectural level extending the reflexion method to software variants. Murphy’s reflexion method allows one to reconstruct the module view, a static architectural view describing the static components, their interfaces and dependencies and their grouping as layers and subsystems. The method consists of the specification of the module view and the mapping of implementation components onto the module view. An automatic analysis determines differences between the module view and its implementation. We extend the reflexion method from single systems to software variants. Because software variants share a very large amount of code, we can expect components of one variant to re-occur in the other variant either identically or at least similarly. We use similarity metrics to determine this type of correspondence among implementation components between two variants. Because the similarity metrics are expensive to compute, we use clone detection techniques to reduce the number of candidate pairs of implementation components for which the similarity is computed. The correspondence is then used to transfer as much as possible of the mapping for the analyzed variants to the next variant to be analyzed. Furthermore, we describe how to unify the individual product architectures into a software product line architecture.     

A bidirectional-transformation-based framework for software visualization and visual editing

Software visualization and visual editing are important and practical techniques to improve the development of complex software systems. A challenge when applying the two technologies is how to realize the correspondence, a bidirectional relationship, between the data and its visual representation correctly. Although many tools and frameworks have been developed to support the construction of visual tools, it is still compli- cated and error-prone to realize the bidirectional relationship. In this paper, we propose a model-driven and bidirectional-transformation-based framework for data visualization and visual editing. Our approach mainly focuses on 1） how to define and manage graphical symbols in the model form and 2） how to specify and im- plement the bidirectional relationship based on the technique of bidirectional model transformation. Then, a prototype tool and four case studies are presented to evaluate the feasibility of our work.     

Coding-error based defects in enterprise resource planning software: Prevention,discovery, elimination and mitigation

Software defects due to coding errors continue to plague the industry with disastrous impact, especially in the enterprise application software category. Identifying how much of these defects are specifically due to coding errors is a challenging problem. In this paper, we investigate the best methods for preventing new coding defects in enterprise resource planning (ERP) software, and discovering and fixing existing coding defects. A large-scale survey-based ex-post-facto study coupled with experiments involving static code analysis tools on both sample code and real-life million lines of code open-source ERP software were conducted for such purpose. The survey-based methodology consisted of respondents who had experience developing ERP software. This research sought to determine if software defects could be merely mitigated or totally eliminated, and what supporting policies, procedures and infrastructure were needed to remedy the problem. In this paper, we introduce a hypothetical framework developed to address our research questions, the hypotheses we have conjectured, the research methodology we have used, and the data analysis methods used to validate the stated hypotheses. Our study revealed that: (a) the best way for ERP developers to discover coding-error based defects in existing programs is to choose an appropriate programming language; perform a combination of manual and automated code auditing, static code analysis, and formal test case design, execution and analysis, (b) the most effective ways to mitigate defects in an ERP system is to track the defect densities in the ERP software, fix the defects found, perform regression testing, and update the resulting defect density statistics, and (c) the impact of epistemological and legal commitments on the defect densities of ERP systems is inconclusive.We feel that our proposed model has the potential to vastly improve the quality of ERP and other similar software by reducing the coding-error defects, and recommend that future research aimed at testing the model in actual production environments.     

Empirical analysis on the correlation between GCC compiler warnings and revision numbers of source files in five industrial software projects

This article discusses whether using warnings generated by the GNU C++ compiler can be used effectively to identify source code files that are likely to be error prone. We analyze five industrial projects written in C++ and belonging to the telecommunication domain. We find a significant positive correlation between the number of compiler warnings and the number of source files changes. We use such correlation to conclude that compiler warnings may be used as an indicator for the presence of software defects in source code. The result of this research is useful for finding defect-prone modules in newer projects, which lack change history.

SUDS: an infrastructure for creating dynamic software defect detection tools

SUDS is a powerful infrastructure for creating dynamic software defect detection tools. It contains phases for both static analysis and dynamic instrumentation allowing users to create tools that take advantage of both paradigms. The results of static analysis phases can be used to improve the quality of dynamic defect detection tools created with SUDS by focusing the instrumentation on types of defects, sources of data, or regions of code. The instrumentation engine is designed in a manner that allows users to create their own correctness models quickly but is flexible to support construction of a wide range of different tools. The effectiveness of SUDS is demonstrated by showing that it is capable of finding bugs and that performance improves when static analysis is used to eliminate unnecessary instrumentation.     

Comparing four approaches for technical debt identification

Software systems accumulate technical debt (TD) when short-term goals in software development are traded for long-term goals (e.g., quick-and-dirty implementation to reach a release date versus a well-refactored implementation that supports the long-term health of the project). Some forms of TD accumulate over time in the form of source code that is difficult to work with and exhibits a variety of anomalies. A number of source code analysis techniques and tools have been proposed to potentially identify the code-level debt accumulated in a system. What has not yet been studied is if using multiple tools to detect TD can lead to benefits, that is, if different tools will flag the same or different source code components. Further, these techniques also lack investigation into the symptoms of TD “interest” that they lead to. To address this latter question, we also investigated whether TD, as identified by the source code analysis techniques, correlates with interest payments in the form of increased defect- and change-proneness. Comparing the results of different TD identification approaches to understand their commonalities and differences and to evaluate their relationship to indicators of future TD “interest.” We selected four different TD identification techniques (code smells, automatic static analysis issues, grime buildup, and Modularity violations) and applied them to 13 versions of the Apache Hadoop open source software project. We collected and aggregated statistical measures to investigate whether the different techniques identified TD indicators in the same or different classes and whether those classes in turn exhibited high interest (in the form of a large number of defects and higher change-proneness). The outputs of the four approaches have very little overlap and are therefore pointing to different problems in the source code. Dispersed Coupling and Modularity violations were co-located in classes with higher defect-proneness. We also observed a strong relationship between Modularity violations and change-proneness. Our main contribution is an initial overview of the TD landscape, showing that different TD techniques are loosely coupled and therefore indicate problems in different locations of the source code. Moreover, our proxy interest indicators (change- and defect-proneness) correlate with only a small subset of TD indicators.     

Mining architectural violations from version history

Software architecture conformance is a key software quality control activity that aims to reveal the progressive gap normally observed between concrete and planned software architectures. However, formally specifying an architecture can be difficult, as it must be done by an expert of the system having a high level understanding of it. In this paper, we present a lightweighted approach for architecture conformance based on a combination of static and historical source code analysis. The proposed approach relies on four heuristics for detecting absences (something expected was not found) and divergences (something prohibited was found) in source code based architectures. We also present an architecture conformance process based on the proposed approach. We followed this process to evaluate the architecture of two industrial-strength information systems, achieving an overall precision of 62.7 % and 53.8 %. We also evaluated our approach in an open-source information retrieval library, achieving an overall precision of 59.2 %. We envision that an heuristic-based approach for architecture conformance can be used to rapidly raise architectural warnings, without deeply involving experts in the process.     

多元统计过程监控与安全生产

统计过程控制是一种改善产品质量及保证安全生产的有力工具。针对现有多元统计监控技术大多假定所考察的生产过程本身仅存在一个标准运行条件,导致实际应用时往往引发大量的连续报警的问题,本文基于主角度建立了任意两个主元模型相似性的度量,提出了一种基于多主元模型的过程监控方法。通过该方法能有效地检测、诊断工业过程中的异常,以避免事故的发生,将带来巨大的经济效益。最后,讨论了相应的软件实现平台EZMon及其应用。     

反模式检测研究综述

反模式是一类在软件设计和实现过程中重复出现的不良解决方案, 这种方案会妨碍软件的维护和演化。反模式检测不仅能够帮助识别软件缺陷, 避免未来的工作中出现同样的错误, 还能够对软件重构提供指导, 对于软件的质量保证有着重要意义。从静态反模式检测和动态反模式检测入手, 详细分析了目前反模式检测领域的相关方法, 对相关检测工具进行了调查分析, 总结了各种检测方法的应用以及存在的不足。     

An evaluation of expert system development tools

Abstract: There is a significant amount of interest in expert systems and the software tools that are available for developing these systems. Most tools that are developed at university research labs are described in some detail in books, articles, or technical reports. However, there is a need for objective information on commercial software tools for building expert systems. These tools can be used for research, prototyping and developing end-user applications. Necessarily we must establish a set of criteria which can be used to evaluate these tools. These criteria include an evaluation of the basic features, the development environment, what a tool can be used for, how easy it is to learn and use, how much it costs, and how it is supported. In the first part of this paper, a set of criteria is described for evaluating expert system software tools. In the second part, these criteria are used to evaluate several currently available commercial tools.     

Software Process Improvement Problems in Twelve Software Companies: An Empirical Analysis

In this paper we discuss our study of the problems 12 software companies experienced in software development. In total we present qualitative data collected from 45 focus groups that involved over 200 software staff. We look at how different practitioner groups respond to software process improvement problems. We show our classification and analysis of this data using correspondence analysis. Correspondence analysis is a graphical data representation method new to software development research. The aim of the work we present is to develop a more holistic understanding of the problems practitioners are experiencing in their attempts to improve their software processes. Our main finding is that there is an association between a company's capability maturity and patterns of reported problems. Organizational problems are more associated with high maturity companies than with low maturity companies. Low maturity companies are closely linked to problems relating directly to projects such as documentation, timescales, tools and technology. Our findings also confirm differences in practitioner group problems. Senior managers cite problems with goals, culture and politics. Project managers are concerned with timescales, change management, budgets and estimates. Developers are experiencing problems with requirements, testing, documentation, communication, tools and technology. These associations are displayed graphically through correspondence analysis maps.     

神经程序修复领域数据泄露问题的实证研究

修复软件缺陷是软件工程领域一个无法回避的重要问题,而程序自动修复技术则旨在自动、准确且高效地修复存在缺陷的程序,以缓解软件缺陷所带来的问题.近年来,随着深度学习的快速发展,程序自动修复领域兴起了一种使用深度神经网络去自动捕捉缺陷程序和其补丁之间关系的方法,被称为神经程序修复.从在基准测试上被正确修复的缺陷的数量上看,神经程序修复工具的修复性能已经显著超过了非学习的程序自动修复工具.然而,近期有研究发现神经程序修复系统性能的提升可能得益于测试数据在训练数据中存在,即数据泄露.受此启发,为进一步探究神经程序修复系统数据泄露的原因及影响,更公平地评估现有的系统,本文(1)对现有神经程序修复系统进行了系统的分类和总结,根据分类结果定义了神经程序修复系统的数据泄露,并为每个类别的系统设计了数据泄露的检测方法;(2)依照上一步骤中的数据泄露检测方法对现有模型展开了大规模检测,并探究了数据泄露对模型真实性能与评估性能间差异的影响以及对模型本身的影响;(3)分析现有神经程序修复系统数据集的收集和过滤策略,加以改进和补充,在现有流行的数据集上基于改进后的策略构建了一个纯净的大规模程序修复训练数据集,并验证了该数据集避免数据泄露的有效性.实验结果发现:本次调研的10个神经程序修复系统在基准测试集上均出现了数据泄露,其中神经程序修复系统RewardRepair的数据泄露问题较为严重,在基准测试集Defects4J(v1.2.0)上的数据泄露达24处,泄露比例高达53.33%.此外,数据泄露对神经程序修复系统的鲁棒性也造成了影响,调研的5个神经程序修复系统均因数据泄露产生了鲁棒性降低的问题.由此可见,数据泄露是一个十分常见的问题,且会使神经程序修复系统得到不公平的性能评估结果,以及影响系统本身的性质.研究人员在训练神经程序修复模型时,应尽可能避免出现数据泄露,且要考虑数据泄露问题对神经程序修复系统性能评估产生的影响,尽可能更公平地评估系统.