随机测试中测试数据的编码方案

1 引言软件测试过程中,测试用例的生成是软件测试的关键。目前,测试用例的生成主要靠手工完成,因此测试效率低下,软件成本居高不下。为此,迫切需要开发一些测试用例的自动生成工具,提高软件测试效率,降低软件成本,保证软件质量,提高软件测试的自动化程度。随机测试是对软件进行操作性测试,即测试用例是从程序的输入域中随机选择,让错误自己暴露出来。在某些情况下,随机测试技术比启发式测试技术效果要好得多。所以在软件测试中占据着重要地位。但是,目前国内外对随机测试数据自动生成的研究非常不成熟,尚无一套完整的方案解决此     

关于蜕变测试和特殊用例测试的实例研究

提出了基于蜕变测试方法的集成测试环境MTest,进而为检验蜕变测试方法的能力和效率,以稀疏矩阵乘法程序为例设计了一组实验.该实验基于变异分析技术,分别以mutation score和错误发现率为度量指标,定量地分析和对比了特殊用例测试,以特殊测试用例和随机测试用例为源测试用例的蜕变测试这3种方法的测试能力和效率.该实验可在MTest测试环境下自动完成.实验结果表明,蜕变测试与特殊用例测试之间是互补的,而且就蜕变测试的源测试用例而言,随机测试用例在测试能力和效率上优于特殊测试用例.     

CART:组合自适应随机测试方法

自适应随机测试方法将测试用例均匀分布于整个输入空间,以提高测试效率.然而,当被测软件的输入参数存在错误相关性,使失效区域形状成为“片状”时,传统的自适应随机测试方法效率将急剧下降.针对“片状”失效区域的特点,本文提出了一种新的自适应随机测试方法:组合自适应随机测试方法.该方法将输入参数划分为多个不同的组;每一组被视作一个独立的输入空间并通过自适应随机测试方法生成“准”测试用例序列;最终的测试用例将由各组“准”测试用例组合而成.实验表明,组合自适应随机测试方法能大幅度提高测试用例发现错误的效率.     

一种基于测试状态的回归测试方法

软件测试在软件生命周期中是一个非常重要的过程,而回归测试则又在软件测试中占有极其重要的地住。本文提出的RTSPS方法综合考虑了测试状态即测试频次、测试成本、错误检测率要求等多种因素来选择合适的回归测试策略,并对测试用例进行优先排序,可以有效地提高测试效率。     

改进ROS2.1的软件业务流程测试方法

在红旗办公服务器—Redoffice Document Server 2.0（RDS2.0）中,软件业务流程的测试用例繁复冗余,而且软件业务流程测试效率不高。针对这些问题,在RDS2.0的升级版本Red Office Server 2.1（ROS2.1）中使用了改进后的软件业务流程测试方法。这些方法减少测试用例的数量和测试工作量,同时提高软件测试覆盖率和效率。     

基于代数式规范的变异测试方法的设计与实现

软件测试是软件工程中保证软件产品质量的重要组成部分.变异测试是一种衡量测试用例集完备性的测试策略,也被用于生成完备的测试用例集.为了提出一种基于代数式规范的新的变异测试方法,为此设计了12类针对代数式规范的变异操作符,对5个代数式规范进行了实验,并进行了结果分析.结果表明基于代数式规范的变异测试方法相比基于代码的传统变异测试方法,生成更少的变异体,也大幅度提升了变异测试的效率.     

基于测试流程的软件测试设计

随着软件开发规模的增大、复杂程度的增加,以寻找软件中的错误为目的的测试工作就显得更加困难。通过分析软件测试流程中的主要活动,介绍了软件测试类型、测试技术和测试方法,给出了测试用例设计及实现的原则,介绍了测试执行方法及缺陷跟踪的目标,描述了测试评估的主要内容。阐述了提高软件产品的质量,加强软件测试过程控制的重要性。     

基于PEFSM行为模型的黑盒测试用例生成方法

随着计算机软件在医疗、航天、金融等领域的广泛应用,人们对软件系统可靠性的要求越来越严格。软件测试是保证软件安全可靠的有效手段,测试用例的优劣会直接影响测试效果及测试成本。针对嵌入式系统黑盒测试问题,提出了基于概率扩展有限状态机(PEFSM)行为模型的测试用例生成方法,通过两个假设给出了该方法的适用场景,设计了正则表达式转化和展开算法,并将该方法应用于Android智能电视的黑盒测试。该方法的特点是:1)根据用户对待测系统各类操作的使用频率信息,优先测试用户常用操作,从而缩减测试用例的数量和长度;2)可人为指定测试用例的初始状态和结束状态,设置闭包循环次数和迁移之间的等待时长,从而保证测试方法的灵活性和适用性。对比实验结果表明,该方法能够降低软件测试成本,提高测试用例的错误探测效率。     

确定性有限状态机的最小测试成本迁移覆盖准则

软件测试中的一个重要问题是测试成本和测试效率的平衡问题.依据程序中错误分布的2-8定律,将测试分为两个阶段,以解决该问题.第1阶段采用最小代价发现软件中的错误,第2阶段针对第1阶段中发现的错误补充设计测试用例,探测软件中潜在的错误.重点是第1阶段的实现.依据确定性有限状态机和集合划分的理论,提出了确定性有限状态机的最小...     

面向对象软件的测试方法研究

面向对象软件测试是保证面向对象软件开发项目质量的重要手段,而测试用例设计是实施面向对象软件测试的重要方面。从软件测试的技术出发,讨论了软件测试的技术特点,如何选择合适的测试方法,如何进行测试用例设计与工具选择,结合项目说明了测试实施和阶段划分。该领域的测试理论和技术还待完善,最终目的是克服软件测试的局限性,提高软件质量。     

A combinatorial testing strategy for concurrent programs

One approach to testing concurrent programs is called reachability testing, which derives test sequences automatically and on‐the‐fly, without constructing a static model. Existing reachability testing algorithms are exhaustive in that they are intended to exercise all possible synchronization sequences of a concurrent program with a given input. In this paper, we present a new testing strategy, called t‐way reachability testing, that adopts the dynamic framework of reachability testing but selectively exercises a subset of synchronization sequences. The selection of the synchronization sequences is based on a combinatorial testing strategy called t‐way testing. We present an algorithm that implements t‐way reachability testing, and report the results of several case studies that were conducted to evaluate its effectiveness. The results indicate that t‐way reachability testing can substantially reduce the number of synchronization sequences exercised during reachability testing while still effectively detecting faults. Copyright © 2007 John Wiley & Sons, Ltd.     

A rigorous approach towards test case generation

This paper presents a method to generate test cases for sequential programs and concurrent programs written in a flow based program language. Test cases of sequential programs are generated based on condition calculation, and can be combined together to form SYN-sequences for concurrent program testing. Semantics of the language provides an infrastructure for the test case generation, and thus our method may be considered as a rigorous and systematic approach to the program testing. Compared with some formal testing methods, our method can avoid hitting state explosion problem in the test formation. Besides, the complexity analysis reveals that our method is time saving. Our method has been applied to generate test cases for PPP over ATM, a subsystem of IAD that runs data and voice over DSL.     

ASRSP——一种新的并发程序测试准则

可达性测试是目前较为成熟的一种并发程序测试方法,该方法解决了如何生成最小完备偏序测试序列集的问题.但研究表明,对于一般规模的并发程序,这一测试序列集仍然太大,以至穷尽测试无法完成.因此,目前亟需能投入实际应用的并发程序测试准则和相应的测试序列生成算法.本文提出了一种实用性较高的并发程序测试准则:全发送接收语句对(ASRSP),并针对该准则提出了一种新的并发程序测试方法:全发送接收语句对可达性测试(ASR-SP-RT).该方法利用可达性测试生成测试序列集的完备性来保证覆盖所有的发送接收语句对,并在每次生成新序列 之后及时去掉对覆盖剩下发送接收语句对无作用的序列,从而达到约简测试序列集的目的.     

Variable strength combinatorial testing of concurrent programs

Reachability testing is an important approach to testing concurrent programs. It generates and exercises synchronization sequences automatically and on-the-fly without saving any test history. Existing reachability testing can be classified into exhaustive and t-way testing. Exhaustive testing is impractical in many cases while t-way testing may decrease the capability of fault detection in some cases. In this paper, we present a variable strength reachability testing strategy, which adopts the dynamic framework of reachability testing and uses a variable strength combinatorial strategy. Different parameter groups are provided with different covering strength. Variable strength testing covers no t-way combinations but the necessary combinations of parameters having mutual interactions in a concurrent program. It is more reasonable than t-way testing because uniform interactions between parameters do not often exist in concurrent systems. We propose a merging algorithmthat implements the variable strength combinatorial testing strategy and conduct our experiment on several concurrent programs. The experimental results indicate that our variable strength reachability testing reaches a good tradeoff between the effectiveness and efficiency. It can keep the same capability of fault detection as exhaustive reachability testing while substantially reducing the number of synchronization sequences and decreasing the execution time in most cases.     

Incremental integration testing of concurrent programs

We present a method for selecting test sequences for concurrent programs from labeled transitions systems (LTS). A common approach to selecting test sequences from a set of LTSs is to derive a global LTS, called the reachability graph, and then force deterministic program executions according to paths selected from the graph. However, using a reachability graph for test path selection introduces a state explosion problem. To overcome this problem, a reduced graph can be generated using incremental reachability analysis, which consists of repeatedly generating a reachability graph for a subset of LTSs, reducing this graph, and using the reduced graph in place of the original LTSs. Unfortunately, existing incremental reachability analysis techniques generate reduced graphs with insufficient information for deterministic testing. We present an incremental approach to testing concurrent programs. Incremental testing consists of incremental reachability analysis for test path selection and deterministic testing for test execution. We define a new type of reachability graph for incremental analysis, called an annotated labeled transition system (ALTS). An ALTS is an LTS annotated with information necessary for deterministic testing. We propose practical coverage criteria for selecting tests paths from an ALTS and present an ALTS reduction algorithm. The results of several case studies are reported     

CMuJava:一个面向Java程序并发变异体生成系统

并发程序由多个共享存储空间并发执行的流程组成.由于流程之间执行次序的不确定性,使得并发软件系统的测试比较困难.变异测试是一种基于故障的软件测试技术,广泛用于评估测试用例集的充分性和测试技术的有效性.将变异测试应用于并发程序的一个关键问题是,如何高效地生成大量的模拟并发故障的变异体集合.给出了一种并发程序的变异测试框架,...     

Test case selection for black-box regression testing of database applications

ContextThis paper presents an approach for selecting regression test cases in the context of large-scale database applications. We focus on a black-box (specification-based) approach, relying on classification tree models to model the input domain of the system under test (SUT), in order to obtain a more practical and scalable solution. We perform an experiment in an industrial setting where the SUT is a large database application in Norway’s tax department.ObjectiveWe investigate the use of similarity-based test case selection for supporting black box regression testing of database applications. We have developed a practical approach and tool (DART) for functional black-box regression testing of database applications. In order to make the regression test approach scalable for large database applications, we needed a test case selection strategy that reduces the test execution costs and analysis effort. We used classification tree models to partition the input domain of the SUT in order to then select test cases. Rather than selecting test cases at random from each partition, we incorporated a similarity-based test case selection, hypothesizing that it would yield a higher fault detection rate.MethodAn experiment was conducted to determine which similarity-based selection algorithm was the most suitable in selecting test cases in large regression test suites, and whether similarity-based selection was a worthwhile and practical alternative to simpler solutions.ResultsThe results show that combining similarity measurement with partition-based test case selection, by using similarity-based test case selection within each partition, can provide improved fault detection rates over simpler solutions when specific conditions are met regarding the partitions.ConclusionsUnder the conditions present in the experiment the improvements were marginal. However, a detailed analysis concludes that the similarity-based selection strategy should be applied when a large number of test cases are contained in each partition and there is significant variability within partitions. If these conditions are not present, incorporating similarity measures is not worthwhile, since the gain is negligible over a random selection within each partition.     

A systematic review of approaches for testing concurrent programs

Concurrent programs are replacing the sequential programs as they utilize the true capabilities of multicore architecture. The extensive use of multicore systems and multithreaded paradigms warrants more attention to the testing of the concurrent programs. The testing concurrent program is not a new field as it has been more than 40 years because the first problem related to the testing concurrent program was addressed by the researchers. The field covers various domains, which include concurrency problems, testing approaches, techniques, graphical representations, tools, and subject systems. This paper aims at providing an overview of research in the domain of testing concurrent programs by classifying it into eight categories: (a) reachability testing, (b) structural testing, (c) model‐based testing, (d) mutation‐based testing, (e) slicing‐based testing, (f) formal methods, (g) random testing, and (h) search‐based testing. The survey is focused on the techniques applied, methodologies followed, and tools used in these aforementioned approaches. Furthermore, the gaps are also identified in different approaches. The paper concludes with the consolidation of various testing parameters along with the future directions. Copyright © 2015 John Wiley & Sons, Ltd.     

Distributed reachability testing of concurrent programs

Reachability testing is an approach to verifying concurrent programs. During reachability testing, every partially ordered synchronization sequence of a program with a given input is exercised exactly once. In this paper, we present the design and implementation of a distributed reachability testing algorithm for a cluster of workstations. This algorithm allows different test sequences to be exercised concurrently by different workstations without any synchronization, and without any duplication of sequences among workstations. Dynamic load balancing is performed using a work‐stealing scheme. A novel aspect of this scheme is that work‐stealing requests progress in rounds. This round‐based structure identifies overloaded workstations to target for work stealing. Empirical studies show good speedup for four benchmark Java programs and one Lotos specification. Copyright © 2010 John Wiley & Sons, Ltd.     

一种形式化的协议互操作性测试方法

对协议实现的互操作性测试是保证网络产品之间互通的一种重要手段。在本文中我们提出了一种基于并发ＴＴＣＮ操作语义的互操作性测试系统的构造方法。