Enhancing software safety by fault trees: experiences from an application to flight critical software

The fault tree analysis is a well-established method in system safety and reliability assessment. We transferred the principles of this technique to an assembler code analysis, regarding any incorrect output of the software as the undesired top-level event. Starting from the instructions providing the outputs and tracking back to all instructions contributing to these outputs a hierarchical system of references is generated that may graphically be represented as a fault tree. To cope with the large number of relations in the code, a tool suite has been developed, which automatically creates these references and checks for unfulfilled preconditions of instructions. The tool was applied to the operational software of an inertial measurement unit, which provides safety critical signals for artificial stabilization of an aircraft. The method and its implementation as a software tool is presented and the benefits, surprising results, and limitations we have experienced were discussed.     

TREE-EXPERT: a tree-based expert system for fault tree construction

This paper addresses the topic of automatic fault tree construction, utilizing an expert system with Artificial Intelligence (AI) techniques and presents the related software tool, TREE-EXPERT—an expert system for automatic fault tree construction. In the light of the features involved in developing a fault tree, a new and more reasonable structure of knowledge representation, which is knowledge tree based, has been established. The knowledge tree provides the means by which component failure behaviors can be described by a group of particular fault tree modules instead of production rules. By introducing the conditional branch function, the new design of the knowledge base incorporates many good features such as strong expressivity, flexibility and ease of extension and it takes advantage of the user's familiarity with the field of fault tree analysis. Additionally, the design of the inference engine is original in that it deals with nodes, which it treats, as special components, so that many complicated engineering cases, such as the application of success criteria, and the problems of flow diversions and flow reversals in a process system, can be well managed and the function of the expert system is improved as a whole. TREE-EXPERT can be used to deal with large-scale and complicated engineering systems, and many engineering factors can be considered, e.g. more than one system parameter and the effect on them switching of the system operating states, bi-directional inference, human error failure, common-cause failure, maintenance and test, etc. On the other hand, the software uses P & ID (Pipe & Instrument Diagram) type interface to describe the system topology, which provides an easier man-machine interface with powerful graphics functions. This software can handle not only ‘process’ systems but also, with appropriate additions to the generic knowledge base, electrical systems and other similar systems.     

知识库构建工具软件的设计与实现

人工智能研究领域之一的专家系统在工程设备上的应用需求较为广泛,但满足用户需求的应用实例很少,主要原因在于知识库构建复杂且困难,其质量得不到保障。针对这个问题,研究了知识获取存在的问题,开发了以故障树为核心表达方式的知识库构建工具软件,实现了故障诊断功能。根据工程实际需要,采用J2EE技术开发了一套B/S（browser/server,浏览器/服务器）模式知识库构建工具软件,并对知识库各模块进行了需求分析设计,包括知识模型的数据结构设计和业务层逻辑方法的设计。此外,还探讨了知识模型的多样性表达,以3种命名方式来表达完整的故障树。最后,通过实例说明了该知识库构建工具软件的可行性。研究结果表明：知识获取模块采用故障树表达方式,有利于知识库质量的提高;选择网页Web形式,可以实现多用户/多工位知识编辑和输入,显著提高知识获取效率。该知识获取辅助系统具有强通用性,为领域专家和工程师构建知识库提供了有力支持。     

Design and implementation of knowledge base building tool software

The expert system, one of the artificial intelligence research field, is widely used to meet the need of engineering equipment. However, the examples of satisfying the requirements of users are quite scarce due to the complexity of establishing the knowledge base and the lack of the guarantee of the quality. To solve this problem, the problems of knowledge acquisition were studied and the knowledge base building tool software based on fault tree was developed, which realized the function of fault diagnosis. According to the actual demand, the J2EE technique was used to develop a set of tool software for establishing knowledge base on B/S (browser/server) model, and the requirement analysis design of each module of knowledge base were carried out, which involved the design of model data structure and the design of business layer logical method. In addition, the diversity expression of knowledge model was discussed, and the complete fault tree was expressed in three naming ways. Finally, an example was given to illustrate the feasibility of the knowledge base building tool software. The results indicated that the expression way of fault tree was used as the core of knowledge-acquiring module to improve the quality of knowledge base. In the meanwhile, the Web form was selected to realize editing and inputting knowledge in multi-user/multi-workstation model to raise the efficiency of acquiring knowledge. Thus, this auxiliary system for knowledge acquirement has strong universality, and it can provide powerful support for domain experts and engineers to establish knowledge base.     

An analysis of safety-critical digital systems for risk-informed design

This paper quantitatively presents the results of a case study which examines the fault tree analysis framework of the safety of digital systems. The case study is performed for the digital reactor protection system of nuclear power plants. The broader usage of digital equipment in nuclear power plants gives rise to the need for assessing safety and reliability because it plays an important role in proving the safety of a designed system in the nuclear industry. We quantitatively explain the relationship between the important characteristics of digital systems and the PSA result using mathematical expressions. We also demonstrate the effect of critical factors on the system safety by sensitivity study and the result which is quantified using the fault tree method shows that some factors remarkably affect the system safety. They are the common cause failure, the coverage of fault tolerant mechanisms and software failure probability.     

Fault tree developed by an object-based method improves requirements specification for safety-related systems

Fault tree analysis is frequently used to improve system reliability and safety. To be suitable for analysis of software in computerised safety-related systems, it has to be modified accordingly. This paper presents a new application: the fault trees developed by an object-based method. The object-based method integrates structural and behavioural models of a system. The developed fault tree includes information on structure and the failure behaviours of classes of the system. Away from traditional use of the fault tree, which for traditional systems emphasises qualitative and quantitative results, the result of the new application emphasises the process of fault tree development and its qualitative results. Such fault tree application reduces the probability of failures in the requirements specification phase within the software life cycle, which increases the reliability of its product; however, it does not confirm this in a quantitative manner.     

SMV model-based safety analysis of software requirements

Fault tree analysis (FTA) is one of the most frequently applied safety analysis techniques when developing safety-critical industrial systems such as software-based emergency shutdown systems of nuclear power plants and has been used for safety analysis of software requirements in the nuclear industry. However, the conventional method for safety analysis of software requirements has several problems in terms of correctness and efficiency; the fault tree generated from natural language specifications may contain flaws or errors while the manual work of safety verification is very labor-intensive and time-consuming. In this paper, we propose a new approach to resolve problems of the conventional method; we generate a fault tree from a symbolic model verifier (SMV) model, not from natural language specifications, and verify safety properties automatically, not manually, by a model checker SMV. To demonstrate the feasibility of this approach, we applied it to shutdown system 2 (SDS2) of Wolsong nuclear power plant (NPP). In spite of subtle ambiguities present in the approach, the results of this case study demonstrate its overall feasibility and effectiveness.     

测试性建模中故障模式概率更新算法研究

为适应复杂装备全寿命周期内的测试性分析与评估,测试性建模中多采用层次化的建模方式．针对建模过程中信号(功能)概率和故障模式概率的冲突问题,提出了两种故障模式概率更新算法,并通过某装备模块的故障模式概率的更新验证了算法的有效性．提高了测试性分析与评估的精度,为模型的修正和故障诊断策略的生成提供了更加可靠的依据．     

基于LASAR的数字电路可测试性设计仿真

随着故障诊断技术的发展,利用专业的仿真工具对实际电路进行可测试性分析仿真用的越来越普遍。LASAR(逻辑自动激励与响应)就是一套优秀的用于数字电路测试开发和逻辑分析的仿真软件系统。介绍了利用LASAR故障仿真进行数字电路可测试性分析的方法。通过对一个实际电路进行仿真,具体说明了该方法在实际工程当中的应用。     

Quantitative demonstration and cost considerations of a software fault removal methodology

This paper presents a demonstration of a methodology for fault removal during software development. The methodology encompasses the entire development history, from system and software requirements generation to system test. Thus it considers not only the faults during software testing after formal configuration controls have been invoked, but also the faults discovered prior to that phase: during system and software requirements generation, preliminary design, detailed design and code and unit testing. The agents for fault discovery used in verification and validation are called activities, techniques and tools (AT & Ts) in this paper, each having a certain maximum potential or capability for fault discovery. The AT & Ts considered include the usual specification review activities, and also certain tools not normally applied in ‘standard’ software development, such as automated requirements aids. Application of the methodology yields numbers of residual faults as of each phase of development, including those remaining to be discovered during operations and maintenance. Some previous experience and data on residual faults correspond to these results, indicating that the methodology and choice of parameters are reasonable. The methodology also allows one to calculate a relative loss due to delay in fault discovery, which, as is well known, rises rapidly when faults are not discovered during the phase in which they are generated.     

TOPAAS: An Alternative Approach to Software Reliability Quantification

In the realm of safety related systems, a growing number of functions are realized by software, ranging from ‘firmware’ to autonomous decision‐taking software. To support (political) real‐world decision makers, quantitative risk assessment methodology quantifies the reliability of systems. The optimal choice of safety measures with respect to the available budget, for example, the UK (as low as reasonably practicable approach), requires quantification. If a system contains software, some accepted methods on quantification of software reliability exist, but none of them is generally applicable, as we will show. We propose a model bringing software into the quantitative risk assessment domain by introducing failure of software modules (with their probabilities) as basic events in a fault tree. The method is known as ‘TOPAAS’ (Task‐Oriented Probability of Abnormalities Analysis for Software). TOPAAS is a factor model allowing the quantification of the basic ‘software’ events in fault tree analyses. In this paper, we argue that this is the best approach currently available to industry. Task‐Oriented Probability of Abnormalities Analysis for Software is a practical model by design and is currently put to field testing in risk assessments of programmable electronic safety‐related systems in tunnels and control systems of movable storm surge barriers in the Netherlands. The TOPAAS model is constructed to incorporate detailed fields of knowledge and to provide focus toward reliability quantification in the form of a probability measure of mission failure. Our development also provides context for further in‐depth research. Copyright © 2013 John Wiley & Sons, Ltd.     

Optimization of safety equipment outages improves safety

Testing and maintenance activities of safety equipment in nuclear power plants are an important potential for risk and cost reduction. An optimization method is presented based on the simulated annealing algorithm. The method determines the optimal schedule of safety equipment outages due to testing and maintenance based on minimization of selected risk measure. The mean value of the selected time dependent risk measure represents the objective function of the optimization. The time dependent function of the selected risk measure is obtained from probabilistic safety assessment, i.e. the fault tree analysis at the system level and the fault tree/event tree analysis at the plant level, both extended with inclusion of time requirements. Results of several examples showed that it is possible to reduce risk by application of the proposed method. Because of large uncertainties in the probabilistic safety assessment, the most important result of the method may not be a selection of the most suitable schedule of safety equipment outages among those, which results in similarly low risk. But, it may be a prevention of such schedules of safety equipment outages, which result in high risk. Such finding increases the importance of evaluation speed versus the requirement of getting always the global optimum no matter if it is only slightly better that certain local one.     

Analysis and measurement of fault coverage in a combined ATE and BIST environment

This paper analyzes an environment which utilizes built-in self-test (BIST) and automatic test equipment (ATE), and presents closed-form expressions for fault coverage as a function of the number of BIST and ATE test vectors. This requires incorporating the time to switch from BIST to ATE (referred to as switchover time), and utilizing ATE generated vectors to finally achieve the desired level of fault coverage. For this environment, we model fault coverage as a function of the testability of the circuit under test and the numbers of vectors which are supplied by the BIST circuitry and the ATE. A novel approach is proposed; this approach is initially based on fault simulation using a small set of random vectors; an estimate of the so-called detection profile of the circuit under test is established as the basis of the test model. This analytical model effectively relates the testable features of the circuit under test to detection using both BIST and ATE as related testing processes.     

An imperfect software debugging model considering irregular fluctuation of fault introduction rate

ABSTRACT

In general, software testing is a complicated and uncertain process. New faults can be introduced into the software during each fault removal. This process is called imperfect debugging. For simplicity, fault introduction rates are generally assumed to be constant. However, software debugging can be affected by many factors, such as subjective and objective influences, the difficulty and complexity of fault removal, the dependent relationships among faults, the changes in different phases of software testing, and the test schedules. Thus, the rate of fault introduction is not a constant, but is an irregularly fluctuating variable in software debugging. In this article, we propose a model with imperfect software debugging considering the irregular changes in fault introduction rates during software debugging. Experimental results reveal that our proposed model has good fitting capability and considerably stronger forecasting performance than that of the other models, and that the proposed model assumptions are close to the actual software debugging situation. Moreover, research on the irregular fluctuation of the fault introduction rate in software debugging has a certain reference value and important significance for software-intensive product testing, for instance, cloud computing.     

基于TEAMS的惯性测量组合测试性建模与分析

惯性测量组合(IMU)是飞行器控制系统中的核心部件,其结构复杂、精度要求高,是故障率最高的部件之一。基于TEAMS平台建立某惯组系统的多信号模型,并对其测试性进行分析与评估,针对固有测试在故障定位和诊断中存在的不足,提出两种测试点的改进方案。分析结果表明:改进后的IMU测试性得到了显著提高,基于多信号模型的测试性分析与评估方法对增强装备保障能力是有效的。     

Systematic evaluation of fault trees using real-time model checker UPPAAL

Fault tree analysis, the most widely used safety analysis technique in industry, is often applied manually. Although techniques such as cutset analysis or probabilistic analysis can be applied on the fault tree to derive further insights, they are inadequate in locating flaws when failure modes in fault tree nodes are incorrectly identified or when causal relationships among failure modes are inaccurately specified. In this paper, we demonstrate that model checking technique is a powerful tool that can formally validate the accuracy of fault trees. We used a real-time model checker UPPAAL because the system we used as the case study, nuclear power emergency shutdown software named Wolsong SDS2, has real-time requirements. By translating functional requirements written in SCR-style tabular notation into timed automata, two types of properties were verified: (1) if failure mode described in a fault tree node is consistent with the system's behavioral model; and (2) whether or not a fault tree node has been accurately decomposed. A group of domain engineers with detailed technical knowledge of Wolsong SDS2 and safety analysis techniques developed fault tree used in the case study. However, model checking technique detected subtle ambiguities present in the fault tree.     

Integration of safety analysis in model-driven software development

Safety critical software requires integrating verification techniques in software development methods. Software architectures must guarantee that developed systems will meet safety requirements and safety analyses are frequently used in the assessment. Safety engineers and software architects must reach a common understanding on an optimal architecture from both perspectives. Currently both groups of engineers apply different modelling techniques and languages: safety analysis models and software modelling languages. The solutions proposed seek to integrate both domains coupling the languages of each domain. It constitutes a sound example of the use of language engineering to improve efficiency in a software-related domain. A model-driven development approach and the use of a platform-independent language are used to bridge the gap between safety analyses (failure mode effects and criticality analysis and fault tree analysis) and software development languages (e.g. unified modelling language). Language abstract syntaxes (metamodels), profiles, language mappings (model transformations) and language refinements, support the direct application of safety analysis to software architectures for the verification of safety requirements. Model consistency and the possibility of automation are found among the benefits.