Easy Test Generation PLAs

Test Generation for large circuits may be extremely difficult.One of the approaches to alleviatingthis problem is to consider the difficulties during the design cycle.This paper proposes a design of EasyTest Generation Programmable Logic Arrays(ETG PLAs),for which test generation is basically notrequired,since a complete test set can be generated while the test is applied.This paper also presents aprocedure which makes a PLA an ETG PLA by following some design rules and providing reasonableextra hardware.     

Goal-oriented dynamic test generation

ContextMemory safety errors such as buffer overflow vulnerabilities are one of the most serious classes of security threats. Detecting and removing such security errors are important tasks of software testing for improving the quality and reliability of software in practice.ObjectiveThis paper presents a goal-oriented testing approach for effectively and efficiently exploring security vulnerability errors. A goal is a potential safety violation and the testing approach is to automatically generate test inputs to uncover the violation.MethodWe use type inference analysis to diagnose potential safety violations and dynamic symbolic execution to perform test input generation. A major challenge facing dynamic symbolic execution in such application is the combinatorial explosion of the path space. To address this fundamental scalability issue, we employ data dependence analysis to identify a root cause leading to the execution of the goal and propose a path exploration algorithm to guide dynamic symbolic execution for effectively discovering the goal.ResultsTo evaluate the effectiveness of our proposed approach, we conducted experiments against 23 buffer overflow vulnerabilities. We observed a significant improvement of our proposed algorithm over two widely adopted search algorithms. Specifically, our algorithm discovered security vulnerability errors within a matter of a few seconds, whereas the two baseline algorithms failed even after 30 min of testing on a number of test subjects.ConclusionThe experimental results highlight the potential of utilizing data dependence analysis to address the combinatorial path space explosion issue faced by dynamic symbolic execution for effective security testing.     

组合测试用例生成技术

组合测试是一种科学有效的软件测试方法,该方法旨在使用较少的测试用例有效地检测软件系统中各个因素以及它们之间的相互作用对系统产生的影响,实践证明其具有较高的错误检测能力。当前组合测试研究的热点之一是组合测试用例生成问题,即如何针对具体待测软件,在满足给定组合覆盖要求的前提下,生成规模尽可能小的测试用例集,以便在保证错误检测能力的前提下尽可能降低测试成本。从N维组合覆盖和变力度组合覆盖等两类不同的组合覆盖标准出发,简要介绍了迄今为止人们在组合测试用例生成领域所取得的研究成果,对现有的组合用例生成方法进行了分类和总结。此外,还对优先级、组合约束、错误定位等条件和应用场景下的组合测试用例生成技术进行了介绍。最后,分析了现有成果中存在的问题,并对该领域未来的研究方向进行了分析和讨论。     

Analysing sanity of requirements for avionics systems

In the last decade it became a common practice to formalise software requirements to improve the clarity of users’ expectations. In this work we build on the fact that functional requirements can be expressed in temporal logic and we propose new sanity checking techniques that automatically detect flaws and suggest improvements of given requirements. Specifically, we describe and experimentally evaluate approaches to consistency and redundancy checking that identify all inconsistencies and pinpoint their exact source (the smallest inconsistent set). We further report on the experience obtained from employing the consistency and redundancy checking in an industrial environment. To complete the sanity checking we also describe a semi-automatic completeness evaluation that can assess the coverage of user requirements and suggest missing properties the user might have wanted to formulate. The usefulness of our completeness evaluation is demonstrated in a case study of an aeroplane control system.     

基于MDA的软件测试用例生成

将MDA中模型驱动的软件代码自动化生成思想应用于模型驱动的软件测试用例自动化生成。从UML/OCL模型出发,采用缺陷测试理论、变异分析技术,结合约束处理规则,开发一个可以自动生成单元测试用例的框架,提高软件测试的自动化程度,从整体上提高软件的开发效率。     

Code-coverage guided prioritized test generation

Most automatic test generation research focuses on generation of test data from pre-selected program paths or input domains or program specifications. This paper presents a methodology for a full solution to code-coverage-based test case generation, which includes code coverage-based path selection, test data generation and actual test case representation in program’s original languages. We implemented this method in an automatic testing framework, eXVantage. Experimental results and industrial trials show that the framework is able to generate tests to achieve program line coverage from 20% to 98% with reduced overall testing effort. Our major contributions include an innovative coverage-based program prioritization algorithm, a novel path selection algorithm that takes into consideration program priority and functional calling relationship, and a constraint solver for test data generation that derives constraints from bytecode and solves complex constraints involving strings and dynamic objects.     

Automated software test data generation

An alternative approach to test-data generation based on actual execution of the program under test, function-minimization methods and dynamic data-flow analysis is presented. Test data are developed for the program using actual values of input variables. When the program is executed, the program execution flow is monitored. If during program execution an undesirable execution flow is observed then function-minimization search algorithms are used to automatically locate the values of input variables for which the selected path is traversed. In addition, dynamic data-flow analysis is used to determine those input variables responsible for the undesirable program behavior, significantly increasing the speed of the search process. The approach to generating test data is then extended to programs with dynamic data structures and a search method based on dynamic data-flow analysis and backtracking is presented. In the approach described, values of array indexes and pointers are known at each step of program execution; this information is used to overcome difficulties of array and pointer handling     

Constraint-based automatic test data generation

A novel technique for automatically generating test data is presented. The technique is based on mutation analysis and creates test data that approximate relative adequacy. It is a fault-based technique that uses algebraic constraints to describe test cases designed to find particular types of faults. A set of tools (collectively called Godzilla) that automatically generates constraints and solves them to create test cases for unit and module testing has been implemented. Godzilla has been integrated with the Mothra testing system and has been used as an effective way to generate test data that kill program mutants. The authors present an initial list of constraints and discuss some of the problems that have been solved to develop the complete implementation of the technique     

High-level test generation for VLSI

The authors survey high-level approaches to test generation for VLSI circuits, which can significantly reduce test generation time while still providing good fault coverage. High-level approaches view the circuit with less structural detail, that is, from a more abstract viewpoint and often hierarchically. The authors first review some basic circuit and fault models and the two most widely known test-generation algorithms as a basis for comparison between high-level and low-level techniques. The authors then examine the more important high-level approaches, which fall into two broad classes, namely algorithmic and heuristic     

面向测试生成的ASM模型约简研究

基于模型的Web应用程序测试是软件测试的一个重要方法。ASM模型从源码解析的角度,基于Web应用程序表示层建立模型,描述了Web应用程序的交互性、动态性和低耦合性。基于ASM模型的测试用例生成,考虑用户的非预期行为,在主要路径的基础上,通过添加无效访问状态和无效迁移路径,扩充测试用例。然而,随着Web应用程序规模的扩大,无效访问状态和无效迁移路径的增加导致测试用例空间爆炸。在研究ASM模型的基础上,通过定义基于ASM模型测试生成的等价迁移和等价状态,合并迁移和状态,从而有效地对ASM模型进行约简,减少了无效访问状态和无效迁移路径的数量,实现测试用例空间约减。 对一个实际Web应用程序系统的评估结果表明,基于模型约简的测试用例优化,有效约减了74.38%的测试用例空间,并且对原子段的覆盖率和错误检测数目没有产生影响。     

基于FSM的测试序列生成方法研究

为了降低UIO序列方法的测试序列长度,通过研究现有的测试序列生成方法,将可逆序列引入到测试序列的生成算法中,将其作为所有转移和状态的连接序列,并利用中国农村邮递员问题的解法构造一条最短遍历路径,使得各个状态的UIO序列之间的重复部分达到最大,测试序列的整体长度被缩短。对测试序列的实验结果表明,算法能够有效降低测试序列的长度。     

BUIO生成及测试序列生成算法研究

在通信协议的一致性测试中,生成较短的测试序列是一个重要问题。提出一种利用UIO生成 BUIO（Backward UIO）的方法,部分UIO直接转换成BUIO,可以降低BUIO的生成费用。对基于UIO和BUIO的启发式测试序列生成方法进行了分析,完善了该方法的生成算法,保证测试序列的自动生成。以ECMA-203协议为例,应用这种算法得到了测试用例,与基于UIO序列和中国邮路算法生成的测试序列进行了比较。     

Effectiveness of microarchitecture test program generation

Investigation of a list of design errors typically encountered in industry is undertaken to determine if our microarchitecture test programs can detect them. Two metrics, functional and timing deviation are used to determine design error coverage     

Model extraction and test generation from JUnit test suites

In this paper, we describe how to infer state machine models of systems from legacy unit test suites and how to generate new tests from those models. The novelty of our approach is to combine control dependencies and data dependencies in the same model, in contrast to most other work in this area. Combining both kinds of dependencies helps us to build more expressive models, which in turn allows us to produce smarter tests. We illustrate those techniques with real examples produced by our implementation, the James tool, designed to apply these techniques in practice to Java code and tests.     

Plain random test generation with PRTest

Automatic test-suite generation tools are often complex and their behavior is not predictable. To provide a minimum baseline that test-suite generators should be able to surpass, we present PRTest, a random black-box test-suite generator for C programs: To create a test, PRTest natively executes the program under test and creates a new, random test value whenever an input value is required. After execution, PRTest checks whether any new program branches were covered and, if this is the case, the created test is added to the test suite. This way, tests are rapidly created either until a crash is found, or until the user aborts the creation. While this naive mechanism is not competitive with more sophisticated, state-of-the-art test-suite generation tools, it is able to provide a good baseline for Test-Comp and a fast alternative for automatic test-suite generation for programs with simple control flow. PRTest is publicly available and open source.

     

Automated test generation from timed automata

Testing is the most dominant validation activity used by industry today, and there is an urgent need for improving its effectiveness, both with respect to the time and resources for test generation and execution, and obtained test coverage. We present a new technique for automatic generation of real-time black-box conformance tests for non-deterministic systems from a determinizable class of timed automata specifications with a dense time interpretation. In contrast to other attempts, our tests are generated using a coarse equivalence class partitioning of the specification. To analyze the specification, to synthesize the timed tests, and to guarantee coverage with respect to a coverage criterion, we use the efficient symbolic techniques recently developed for model checking of real-time systems. Application of our prototype tool to a realistic specification shows promising results in terms of both the test suite size, and the time and space used for test generation.     

Cloud-based automatic test data generation framework

Designing test cases is one of the most crucial activities in software testing process. Manual test case design might result in inadequate testing outputs due to lack of expertise and/or skill requirements. This article delivers automatic test data generation framework by effectively utilizing soft computing technique with Apache Hadoop MapReduce as the parallelization framework. We have evaluated and analyzed statistically our proposed framework using real world open source libraries. The experimental results conducted on Hadoop cluster with ten nodes are effective and our framework significantly outperforms other existing cloud-based testing models.     

Survey on test data generation tools

Automating the process of software testing is a very popular research topic and of real interest to industry. Test automation can take part on different levels, e.g., test execution, test case generation, test data generation. This survey gives an overview of state-of-the art test data generation tools, either academic or commercial. The survey focuses on white- and gray-box techniques. The list of existing tools was filtered with respect to their public availability, their maturity, and activity. The remaining seven tools, i.e., AgitarOne, CodePro AnalytiX, AutoTest, C++test, Jtest, RANDOOP, and PEX, are briefly introduced and their evaluation results are summarized. For the evaluation we defined 31 benchmark tests, which check the tools capabilities to generate test data that satisfies a given specification: 24 primitive type benchmarks and 7 non-primitive type and more complex with respect to the specification benchmarks. Most of the commercial tools implement a test data strategy that uses constant values found in the method under test or values that are slightly modified by means of mathematical operations. This strategy turns out to be very effective. In general, all tools that combine multiple techniques perform very well. For example PEX uses constraint solving techniques, but in cases where the constraint solver reaches its limitations it uses random based techniques to overcome those limitations. Especially, the two commercial tools AgitarOne and PEX that combine multiple approaches to test data generation are able to pass all 31 tests. This survey reflects the status in 2011.     

BPEL的测试用例生成研究

BPEL是能表达复杂的并发行为的一个语言,被用于描述Web服务的组合.提出一个算法来生成BPEL过程的单元测试用例.首先通过给定的转换规则把用BPEL描述的过程转换成BPEL流图(FGBPEL),然后提出CTP算法来生成测试用例.对提出的方法做适当的扩展,也可以运用到其它的业务过程处理语言中.