Model checking driven static analysis for the real world: designing and tuning large scale bug detection

Model checking and static analysis are traditionally seen as two separate approaches to software analysis and verification. In this work we define a model, checking approach for the static analysis of large C/C++ source code bases to detect potential run-time issues such as program crashes, security vulnerabilities and memory leaks. Working on the intersection of software model checking and automated static bug detection for real-life systems, we address a number of issues: how to scale for real-life systems of 1,000,000 LoC or more, how to quickly write new checks, and most importantly how to distinguish between relevant and irrelevant bugs and fine tune the analysis accordingly. We define our model checking-based static analysis approach implemented in our tool Goanna, illustrate a number of design and implementation decisions to obtain practical outcomes and relevant results, and present our findings by empirical data obtained from regularly analyzing large industrial and open source code bases such as the Firefox Web browser.     

A type system for static and dynamic checking of C++ pointers

Object-oriented programming is the most used programming paradigm when dealing with large-scale, modular software. In this field, the two leading languages are Java and C++. The former has superior qualities in terms of safety and ease of programming, whereas the latter is often considered an “old” language, too complex and potentially unsafe.In this paper, we describe a new type system designed to analyze the security problems derived from pointer manipulation in C++. This type system tries to trap the most common errors through static analysis, i.e., at compile-time, and only when static analysis fails it generates and embeds code fragments that apply runtime checks on specific instructions. The aim of this new type system is to give C++ the same safety of Java in the most important memory-related operations, without adding much runtime overhead. An experimental implementation of the type system is also presented, embedded in a C++ analysis tool called GPCC.     

Code Analysis for Software and System Security Using Open Source Tools

ABSTRACT

Software security helps in identifying and managing risks. One of the effective ways to identify software vulnerabilities is to analyze its code. Code analysis (Chess & West, 2007 Chess, B. and West, J. 2007. Secure programming with static analysis, Boston, MA: Addison-Wesley.  [Google Scholar]) helps in catching common coding mistakes such as buffer overflow, unused variables, memory leaks, and various race conditions, which in turn optimizes computer programs, both in storage and computation aspects. Software developers use either open source tools or commercial tools for verification and validation of software. Without proper validation of a software/system using some standard guidelines, potential attackers can find ways to exploit vulnerabilities and bugs and then can gain control over a system, if they are successful. In this paper, we discuss some of the open source static code analysis and dynamic analysis tools, their merits, and limitations with respect to some target codes that contain possible threats. We consider C/C++ and Java programming languages for our experiments. For static code analyzers, we consider Flawfinder, Splint, and Cppcheck; PMD, Findbugs, and Valgrind for dynamic code analysis, and its plug-in, Memcheck, to perform dynamic analysis on executables. We provide our observations in a comparison table, highlighting these tools strengths and weaknesses.     

基于代码属性图及注意力双向LSTM的漏洞挖掘方法

随着信息安全愈发严峻的趋势,软件漏洞已成为计算机安全的主要威胁之一.如何准确地挖掘程序中存在的漏洞,是信息安全领域的关键问题.然而,现有的静态漏洞挖掘方法在挖掘漏洞特征不明显的漏洞时准确率明显下降.一方面,基于规则的方法通过在目标源程序中匹配专家预先定义的漏洞模式挖掘漏洞,其预定义的漏洞模式较为刻板单一,无法覆盖到细节特征,导致其存在准确率低、误报率高等问题;另一方面,基于学习的方法无法充分地对程序源代码的特征信息进行建模,并且无法有效地捕捉关键特征信息,导致其在面对漏洞特征不明显的漏洞时,无法准确地进行挖掘.针对上述问题,提出了一种基于代码属性图及注意力双向LSTM的源码级漏洞挖掘方法.该方法首先将程序源代码转换为包含语义特征信息的代码属性图,并对其进行切片以剔除与敏感操作无关的冗余信息;其次,使用编码算法将代码属性图编码为特征张量;然后,利用大规模特征数据集训练基于双向LSTM和注意力机制的神经网络;最后,使用训练完毕的神经网络实现对目标程序中的漏洞进行挖掘.实验结果显示,在SARD缓冲区错误数据集、SARD资源管理错误数据集及它们两个C语言程序构成的子集上,该方法的F1分数分别达到了82.8%,77.4%,82.5%和78.0%,与基于规则的静态挖掘工具Flawfinder和RATS以及基于学习的程序分析模型TBCNN相比,有显著的提高.     

A static technique for detecting input validation vulnerabilities in Android apps

Input validation vulnerabilities are common in Android apps, especially in inter-component communications. Malicious attacks can exploit this kind of vulnerability to bypass Android security mechanism and compromise the integrity, confidentiality and availability of Android devices. However, so far there is not a sound approach at the source code level for app developers aiming to detect input validation vulnerabilities in Android apps. In this paper, we propose a novel approach for detecting input validation flaws in Android apps and we implement a prototype named EasyIVD, which provides practical static analysis of Java source code. EasyIVD leverages backward program slicing to extract transaction and constraint slices from Java source code. Then EasyIVD validates these slices with predefined security rules to detect vulnerabilities in a known pattern. To detect vulnerabilities in an unknown pattern, EasyIVD extracts implicit security specifications as frequent patterns from the duplicated slices and verifies them. Then EasyIVD semi-automatically confirms the suspicious rule violations and reports the confirmed ones as vulnerabilities. We evaluate EasyIVD on four versions of original Android apps spanning from version 2.2 to 5.0. It detects 58 vulnerabilities including confused deputy attacks and denial of service attacks. Our results prove that EasyIVD can provide a practical defensive solution for app developers.     

基于XML的软件安全静态检测方法研究

安全关键软件设计使用的C/C++语言含有大量未定义行为,使用不当可能产生重大安全隐患。软件静态检测是从软件代码和结构中找出安全缺陷的重要手段。从安全规则的角度,提出了基于XML（eXtensible Markup Language）中间模型的静态检测方法。该方法将C/C++源代码解释为XML中间模型,将安全规则转化为缺陷模式,利用Xquery查询表达式对软件安全缺陷进行定位。基于该方法的原型系统检验结果表明：该方法能够有效地检测出违反安全规则的软件缺陷,并具有安全规则可定制的特点。     

Static vulnerability detection in Java service-oriented components

Extensible component-based platforms allow dynamic discovery, installation and execution of components. Such platforms are service-oriented, as components may directly interact with each other via the services they provide. Even robust languages such as Java were not designed to handle safe code interaction between trusted and untrusted parties. Dynamic installation of code provided by different third parties leads to several security issues. The different security layers adopted by Java or component-based platforms cannot fully address the problem of untrusted components trying to tamper with other components via legitimate interactions. A malicious component might even use vulnerable ones to compromise the whole component-based platform. Our approach identifies vulnerable components in order to prevent them from threatening services security. We use static analysis to remain as exhaustive as possible and to avoid the need for non-standard or intrusive environments. We show that a static analysis through tainted object propagation is well suited to detect vulnerabilities in Java service-oriented components. We present STOP, a Service-oriented Tainted Object Propagation tool, which applies this technique to statically detect those security flaws. Finally, the audit of several trusted Apache Felix bundles shows that nowadays component-based platforms are not prepared for malicious Java interactions.     

Improving software security with static automated code analysis in an industry setting

Software security can be improved by identifying and correcting vulnerabilities. In order to reduce the cost of rework, vulnerabilities should be detected as early and efficiently as possible. Static automated code analysis is an approach for early detection. So far, only few empirical studies have been conducted in an industrial context to evaluate static automated code analysis. A case study was conducted to evaluate static code analysis in industry focusing on defect detection capability, deployment, and usage of static automated code analysis with a focus on software security. We identified that the tool was capable of detecting memory related vulnerabilities, but few vulnerabilities of other types. The deployment of the tool played an important role in its success as an early vulnerability detector, but also the developers perception of the tools merit. Classifying the warnings from the tool was harder for the developers than to correct them. The correction of false positives in some cases created new vulnerabilities in previously safe code. With regard to defect detection ability, we conclude that static code analysis is able to identify vulnerabilities in different categories. In terms of deployment, we conclude that the tool should be integrated with bug reporting systems, and developers need to share the responsibility for classifying and reporting warnings. With regard to tool usage by developers, we propose to use multiple persons (at least two) in classifying a warning. The same goes for making the decision of how to act based on the warning. Copyright © 2012 John Wiley & Sons, Ltd.     

Java Insecurity: Accounting for Subtleties That Can Compromise Code

Conscientious Java developers are typically aware of the numerous coding guidelines that they should follow when writing code, such as validating inputs, minimizing accessibility to classes and members, and avoiding public static nonfinal fields. Java developers follow such guidelines to avoid common programming pitfalls (often called antipatterns), thereby reducing the likelihood of bugs or security vulnerabilities in their programs.     

Static analyzer Svace for finding defects in a source program code

This paper describes Svace, a tool for static program analysis developed at the Institute for Systems Programming, Russian Academy of Sciences. This tool allows one to find defects and potential vulnerabilities in the source program code written in C/C++ languages. The main features of the tool are simplicity of use, wide variety of supported types of warnings, scalability up to programs of millions of code lines, and acceptable quality of analysis (30–80% of true positive warnings).     

Certifying a java type resolution function using program transformation,annotation, and reflection

In Java, type resolution is a function that takes a reference to a type occurring in a given context as input and returns the canonical name of that type. This information is fundamental to static analysis—a “must have” function underlying virtually all forms of semantic-based analysis. In the case of Java, this function is also complex and it is quite common to encounter tools where it is implemented incorrectly. This paper presents a novel approach for certifying the correctness of a given type resolution function with respect to an arbitrary Java source code base. The approach uses program transformation to instrument a subject code base in such a way that reflection can then be used to certify the correctness of the type resolution function against the function used by the Java compiler. In this form of certification, the type resolution function of the Java compiler serves as the test oracle.     

缓冲区溢出的建模分析

论文介绍了一种检测缓冲区溢出漏洞的建模方法。该方法的关键是对C语言的源代码做静态分析,把缓冲区溢出这个事件作为一个整数区域问题来看待,从而建立缓冲区溢出的数学模型。该模型的一个优点是在代码编译以前可以通过静态分析的方法来发现潜在的缓冲区溢出漏洞。     

A comparison of the efficiency and effectiveness of vulnerability discovery techniques

ContextSecurity vulnerabilities discovered later in the development cycle are more expensive to fix than those discovered early. Therefore, software developers should strive to discover vulnerabilities as early as possible. Unfortunately, the large size of code bases and lack of developer expertise can make discovering software vulnerabilities difficult. A number of vulnerability discovery techniques are available, each with their own strengths.ObjectiveThe objective of this research is to aid in the selection of vulnerability discovery techniques by comparing the vulnerabilities detected by each and comparing their efficiencies.MethodWe conducted three case studies using three electronic health record systems to compare four vulnerability discovery techniques: exploratory manual penetration testing, systematic manual penetration testing, automated penetration testing, and automated static analysis.ResultsIn our case study, we found empirical evidence that no single technique discovered every type of vulnerability. We discovered that the specific set of vulnerabilities identified by one tool was largely orthogonal to that of other tools. Systematic manual penetration testing found the most design flaws, while automated static analysis found the most implementation bugs. The most efficient discovery technique in terms of vulnerabilities discovered per hour was automated penetration testing.ConclusionThe results show that employing a single technique for vulnerability discovery is insufficient for finding all types of vulnerabilities. Each technique identified only a subset of the vulnerabilities, which, for the most part were independent of each other. Our results suggest that in order to discover the greatest variety of vulnerability types, at least systematic manual penetration testing and automated static analysis should be performed.     

C/C++静态代码安全检查工具研究

静态代码安全检查工具是一种能够帮助程序员自动检测出源程序中是否存在安全缺陷的软件。它通过逐行分析程序的源代码,发现软件中潜在的安全漏洞。本文针对C／C＋＋语言程序设计中容易存在的多种安全问题,分别分析了问题的根源,给出了具体可行的分析及检测方法。最后通过对静态代码安全检查工具优缺点的比较,给出了一些提高安全检查效果的建议。     

Lightweight Bytecode Verification

In this paper, we provide a theoretical foundation for and improvements to the existing bytecode verification technology, a critical component of the Java security model, for mobile code used with the Java “micro edition” (J2ME), which is intended for embedded computing devices. In Java, remotely loaded “bytecode” class files are required to be bytecode verified before execution, that is, to undergo a static type analysis that protects the platform's Java run-time system from so-called type confusion attacks such as pointer manipulation. The data flow analysis that performs the verification, however, is beyond the capacity of most embedded devices because of the memory requirements that the typical algorithm will need. We propose to take a proof-carrying code approach to data flow analysis in defining an alternative technique called “lightweight analysis” that uses the notion of a “certificate” to reanalyze a previously analyzed data flow problem, even on poorly resourced platforms. We formally prove that the technique provides the same guarantees as standard bytecode safety verification analysis, in particular that it is “tamper proof” in the sense that the guarantees provided by the analysis cannot be broken by crafting a “false” certificate or by altering the analyzed code. We show how the Java bytecode verifier fits into this framework for an important subset of the Java Virtual Machine; we also show how the resulting “lightweight bytecode verification” technique generalizes and simulates the J2ME verifier (to be expected as Sun's J2ME “K-Virtual machine” verifier was directly based on an early version of this work), as well as Leroy's “on-card bytecode verifier,” which is specifically targeted for Java Cards.     

CrasyDSE: A framework for solving Dyson–Schwinger equations

Dyson–Schwinger equations are important tools for non-perturbative analyses of quantum field theories. For example, they are very useful for investigations in quantum chromodynamics and related theories. However, sometimes progress is impeded by the complexity of the equations. Thus automating parts of the calculations will certainly be helpful in future investigations. In this article we present a framework for such an automation based on a C++ code that can deal with a large number of Green functions. Since also the creation of the expressions for the integrals of the Dyson–Schwinger equations needs to be automated, we defer this task to a Mathematica notebook. We illustrate the complete workflow with an example from Yang–Mills theory coupled to a fundamental scalar field that has been investigated recently. As a second example we calculate the propagators of pure Yang–Mills theory. Our code can serve as a basis for many further investigations where the equations are too complicated to tackle by hand. It also can easily be combined with DoFun, a program for the derivation of Dyson–Schwinger equations.¹Program summaryProgram title: CrasyDSECatalogue identifier: AEMY _v1_0Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AEMY_v1_0.htmlProgram obtainable from: CPC Program Library, Queen’s University, Belfast, N. IrelandLicensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.htmlNo. of lines in distributed program, including test data, etc.: 49030No. of bytes in distributed program, including test data, etc.: 303958Distribution format: tar.gzProgramming language: Mathematica 8 and higher, C++.Computer: All on which Mathematica and C++ are available.Operating system: All on which Mathematica and C++ are available (Windows, Unix, Mac OS).Classification: 11.1, 11.4, 11.5, 11.6.Nature of problem: Solve (large) systems of Dyson–Schwinger equations numerically.Solution method: Create C++ functions in Mathematica to be used for the numeric code in C++. This code uses structures to handle large numbers of Green functions.Unusual features: Provides a tool to convert Mathematica expressions into C++ expressions including conversion of function names.Running time: Depending on the complexity of the investigated system solving the equations numerically can take seconds on a desktop PC to hours on a cluster.     

Secure coding in C and C++ of strings and integers

Although the flexibility and performance of C and C++ aren't in question, security has increasingly become an issue. The C and C++ communities have recognized this and have taken steps to improve security at all levels, including improved standards, compiler implementations, and static and runtime analysis tools.