Analyzing impact of experience curve on ROI in the software product line adoption process

ContextExperience curve is a well-known concept in management and education science, which explains the phenomenon of increased worker efficiency with repetitive production of a good or service.ObjectiveWe aim to analyze the impact of the experience curve effect on the Return on Investment (ROI) in the software product line engineering (SPLE) process.MethodWe first present the results of a systematic literature review (SLR) to explicitly depict the studies that have considered the impact of experience curve effect on software development in general. Subsequently, based on the results of the SLR, the experience curve effect models in the literature, and the SPLE cost models, we define an approach for extending the cost models with the experience curve effect. Finally, we discuss the application of the refined cost models in a real industrial context.ResultsThe SLR resulted in 15 primary studies which confirm the impact of experience curve effect on software development in general but the experience curve effect in the adoption of SPLE got less attention. The analytical discussion of the cost models and the application of the refined SPLE cost models in the industrial context showed a clear impact of the experience curve effect on the time-to-market, cost of development and ROI in the SPLE adoption process.ConclusionsThe proposed analysis with the newly defined cost models for SPLE adoption provides a more precise analysis tool for the management, and as such helps to support a better decision making.     

Managing requirements inter-dependency for software product line derivation

Software Product Line Engineering (SPLE) can reduce software development costs, reduce time to market and improve product quality. A software product line is a set of software products sharing a set of common features but containing variation points. Successful SPLE requires making selection decisions at variation points effectively and efficiently. A significant challenge is how to identify, represent and manage the inter-dependency of selection decisions for requirements. We developed the concept of a meta-model for requirement decision models to bring formalism and consistency to the structure and to model inter-dependencies between requirement selection decisions. Here we present a meta-model for requirement selection decisions that includes inter-dependencies and we use a mobile phone worked example to illustrate our approach. To support our method, we developed two separate tools, V-Define (for domain decision model construction) and V-Resolve (for new product derivation). Finally the results of a metal processing product line case study using the tools are described.

Requirements engineering for software product lines: A systematic literature review

ContextSoftware product line engineering (SPLE) is a growing area showing promising results in research and practice. In order to foster its further development and acceptance in industry, it is necessary to assess the quality of the research so that proper evidence for adoption and validity are ensured. This holds in particular for requirements engineering (RE) within SPLE, where a growing number of approaches have been proposed.ObjectiveThis paper focuses on RE within SPLE and has the following goals: assess research quality, synthesize evidence to suggest important implications for practice, and identify research trends, open problems, and areas for improvement.MethodA systematic literature review was conducted with three research questions and assessed 49 studies, dated from 1990 to 2009.ResultsThe evidence for adoption of the methods is not mature, given the primary focus on toy examples. The proposed approaches still have serious limitations in terms of rigor, credibility, and validity of their findings. Additionally, most approaches still lack tool support addressing the heterogeneity and mostly textual nature of requirements formats as well as address only the proactive SPLE adoption strategy.ConclusionsFurther empirical studies should be performed with sufficient rigor to enhance the body of evidence in RE within SPLE. In this context, there is a clear need for conducting studies comparing alternative methods. In order to address scalability and popularization of the approaches, future research should be invested in tool support and in addressing combined SPLE adoption strategies.     

Combining service-orientation and software product line engineering: A systematic mapping study

ContextService-Orientation (SO) is a rapidly emerging paradigm for the design and development of adaptive and dynamic software systems. Software Product Line Engineering (SPLE) has also gained attention as a promising and successful software reuse development paradigm over the last decade and proven to provide effective solutions to deal with managing the growing complexity of software systems.ObjectiveThis study aims at characterizing and identifying the existing research on employing and leveraging SO and SPLE.MethodWe conducted a systematic mapping study to identify and analyze related literature. We identified 81 primary studies, dated from 2000–2011 and classified them with respect to research focus, types of research and contribution.ResultThe mapping synthesizes the available evidence about combining the synergy points and integration of SO and SPLE. The analysis shows that the majority of studies focus on service variability modeling and adaptive systems by employing SPLE principles and approaches.In particular, SPLE approaches, especially feature-oriented approaches for variability modeling, have been applied to the design and development of service-oriented systems. While SO is employed in software product line contexts for the realization of product lines to reconcile the flexibility, scalability and dynamism in product derivations thereby creating dynamic software product lines.ConclusionOur study summarizes and characterizes the SO and SPLE topics researchers have investigated over the past decade and identifies promising research directions as due to the synergy generated by integrating methods and techniques from these two areas.     

Evolutionary robust optimization for software product line scoping: An explorative study

Background: Software product line (SPL) scoping is an important phase when planning for product line adoption. An SPL scope specifies: (1) the extent of the domain supported by the product line, (2) portfolio of products in the product line and (3) list of assets to be developed for reuse across the family of products.Issue: SPL scope planning is usually based on estimates about the state of the market and the engineering capabilities of the development team. One challenge with these estimates is that there are inaccuracies due to uncertainty in the environment or accuracy of measurement. This may result in issues ranging from suboptimal plans to infeasible plans.Objective: To address the above, we propose to include uncertainty as part of the SPL scoping model. Plans developed in consideration of uncertainty would be more robust against possible fluctuations in estimates.Approach: In this paper, a method to incorporate uncertainty in scoping optimization and its application to generate robust solutions is proposed. We capture uncertainty as part of the formulation and model scoping optimization as a multi-objective problem with profit and stability as fitness functions. Profit stability and feasibility stability are considered to represent stability concerns.Results: Results show that, compared to other scope optimization approaches, both performance stability and feasibility stability are improved while maintaining near optimal performance for profit objective. Also, generated results consist of solutions with trade-offs between profit and stability, providing the decision maker with enhanced decision support.Conclusion: Multi-objective optimization with stability consideration for SPL scoping provides project managers with a robust and flexible way to address uncertainty in the process of SPL scoping.     

A method to optimize the scope of a software product platform based on end-user features

ContextDue to increased competition and the advent of mass customization, many software firms are utilizing product families – groups of related products derived from a product platform – to provide product variety in a cost-effective manner. The key to designing a successful software product family is the product platform, so it is important to determine the most appropriate product platform scope related to business objectives, for product line development.AimThis paper proposes a novel method to find the optimized scope of a software product platform based on end-user features.MethodThe proposed method, PPSMS (Product Platform Scoping Method for Software Product Lines), mathematically formulates the product platform scope selection as an optimization problem. The problem formulation targets identification of an optimized product platform scope that will maximize life cycle cost savings and the amount of commonality, while meeting the goals and needs of the envisioned customers’ segments. A simulated annealing based algorithm that can solve problems heuristically is then used to help the decision maker in selecting a scope for the product platform, by performing tradeoff analysis of the commonality and cost savings objectives.ResultsIn a case study, PPSMS helped in identifying 5 non-dominated solutions considered to be of highest preference for decision making, taking into account both cost savings and commonality objectives. A quantitative and qualitative analysis indicated that human experts perceived value in adopting the method in practice, and that it was effective in identifying appropriate product platform scope.     

VIVACE: A framework for the systematic evaluation of variability support in process-aware information systems

ContextThe increasing adoption of process-aware information systems (PAISs) such as workflow management systems, enterprise resource planning systems, or case management systems, together with the high variability in business processes (e.g., sales processes may vary depending on the respective products and countries), has resulted in large industrial process model repositories. To cope with this business process variability, the proper management of process variants along the entire process lifecycle becomes crucial.ObjectiveThe goal of this paper is to develop a fundamental understanding of business process variability. In particular, the paper will provide a framework for assessing and comparing process variability approaches and the support they provide for the different phases of the business process lifecycle (i.e., process analysis and design, configuration, enactment, diagnosis, and evolution).MethodWe conducted a systematic literature review (SLR) in order to discover how process variability is supported by existing approaches.ResultsThe SLR resulted in 63 primary studies which were deeply analyzed. Based on this analysis, we derived the VIVACE framework. VIVACE allows assessing the expressiveness of a process modeling language regarding the explicit specification of process variability. Furthermore, the support provided by a process-aware information system to properly deal with process model variants can be assessed with VIVACE as well.ConclusionsVIVACE provides an empirically-grounded framework for process engineers that enables them to evaluate existing process variability approaches as well as to select that variability approach meeting their requirements best. Finally, it helps process engineers in implementing PAISs supporting process variability along the entire process lifecycle.     

A systematic literature review on the usage of eye-tracking in software engineering

ContextEye-tracking is a mean to collect evidence regarding some participants’ cognitive processes. Eye-trackers monitor participants’ visual attention by collecting eye-movement data. These data are useful to get insights into participants’ cognitive processes during reasoning tasks.ObjectiveThe Evidence-based Software Engineering (EBSE) paradigm has been proposed in 2004 and, since then, has been used to provide detailed insights regarding different topics in software engineering research and practice. Systematic Literature Reviews (SLR) are also useful in the context of EBSE by bringing together all existing evidence of research and results about a particular topic. This SLR evaluates the current state of the art of using eye-trackers in software engineering and provides evidence on the uses and contributions of eye-trackers to empirical studies in software engineering.MethodWe perform a SLR covering eye-tracking studies in software engineering published from 1990 up to the end of 2014. To search all recognised resources, instead of applying manual search, we perform an extensive automated search using Engineering Village. We identify 36 relevant publications, including nine journal papers, two workshop papers, and 25 conference papers.ResultsThe software engineering community started using eye-trackers in the 1990s and they have become increasingly recognised as useful tools to conduct empirical studies from 2006. We observe that researchers use eye-trackers to study model comprehension, code comprehension, debugging, collaborative interaction, and traceability. Moreover, we find that studies use different metrics based on eye-movement data to obtain quantitative measures. We also report the limitations of current eye-tracking technology, which threaten the validity of previous studies, along with suggestions to mitigate these limitations.ConclusionHowever, not withstanding these limitations and threats, we conclude that the advent of new eye-trackers makes the use of these tools easier and less obtrusive and that the software engineering community could benefit more from this technology.     

A systematic mapping study of search-based software engineering for software product lines

ContextSearch-Based Software Engineering (SBSE) is an emerging discipline that focuses on the application of search-based optimization techniques to software engineering problems. Software Product Lines (SPLs) are families of related software systems whose members are distinguished by the set of features each one provides. SPL development practices have proven benefits such as improved software reuse, better customization, and faster time to market. A typical SPL usually involves a large number of systems and features, a fact that makes them attractive for the application of SBSE techniques which are able to tackle problems that involve large search spaces.ObjectiveThe main objective of our work is to identify the quantity and the type of research on the application of SBSE techniques to SPL problems. More concretely, the SBSE techniques that have been used and at what stage of the SPL life cycle, the type of case studies employed and their empirical analysis, and the fora where the research has been published.MethodA systematic mapping study was conducted with five research questions and assessed 77 publications from 2001, when the term SBSE was coined, until 2014.ResultsThe most common application of SBSE techniques found was testing followed by product configuration, with genetic algorithms and multi-objective evolutionary algorithms being the two most commonly used techniques. Our study identified the need to improve the robustness of the empirical evaluation of existing research, a lack of extensive and robust tool support, and multiple avenues worthy of further investigation.ConclusionsOur study attested the great synergy existing between both fields, corroborated the increasing and ongoing interest in research on the subject, and revealed challenging open research questions.     

A systematic review of evaluation of variability management approaches in software product lines

Context

Variability management (VM) is one of the most important activities of software product-line engineering (SPLE), which intends to develop software-intensive systems using platforms and mass customization. VM encompasses the activities of eliciting and representing variability in software artefacts, establishing and managing dependencies among different variabilities, and supporting the exploitation of the variabilities for building and evolving a family of software systems. Software product line (SPL) community has allocated huge amount of effort to develop various approaches to dealing with variability related challenges during the last two decade. Several dozens of VM approaches have been reported. However, there has been no systematic effort to study how the reported VM approaches have been evaluated.

Objective

The objectives of this research are to review the status of evaluation of reported VM approaches and to synthesize the available evidence about the effects of the reported approaches.

Method

We carried out a systematic literature review of the VM approaches in SPLE reported from 1990s until December 2007.

Results

We selected 97 papers according to our inclusion and exclusion criteria. The selected papers appeared in 56 publication venues. We found that only a small number of the reviewed approaches had been evaluated using rigorous scientific methods. A detailed investigation of the reviewed studies employing empirical research methods revealed significant quality deficiencies in various aspects of the used quality assessment criteria. The synthesis of the available evidence showed that all studies, except one, reported only positive effects.

Conclusion

The findings from this systematic review show that a large majority of the reported VM approaches have not been sufficiently evaluated using scientifically rigorous methods. The available evidence is sparse and the quality of the presented evidence is quite low. The findings highlight the areas in need of improvement, i.e., rigorous evaluation of VM approaches. However, the reported evidence is quite consistent across different studies. That means the proposed approaches may be very beneficial when they are applied properly in appropriate situations. Hence, it can be concluded that further investigations need to pay more attention to the contexts under which different approaches can be more beneficial.     

Using mapping studies as the basis for further research - A participant-observer case study

Context

We are strong advocates of evidence-based software engineering (EBSE) in general and systematic literature reviews (SLRs) in particular. We believe it is essential that the SLR methodology is used constructively to support software engineering research.

Objective

This study aims to assess the value of mapping studies which are a form of SLR that aims to identify and categorise the available research on a broad software engineering topic.

Method

We used a multi-case, participant-observer case study using five examples of studies that were based on preceding mapping studies. We also validated our results by contacting two other researchers who had undertaken studies based on preceding mapping studies and by assessing review comments related to our follow-on studies.

Results

Our original case study identified 11 unique benefits that can accrue from basing research on a preceding mapping study of which only two were case specific. We also identified nine problems associated with using preceding mapping studies of which two were case specific. These results were consistent with the information obtained from the validation activities. We did not find an example of an independent research group making use of a mapping study produced by other researchers.

Conclusion

Mapping studies can save time and effort for researchers and provide baselines to assist new research efforts. However, they must be of high quality in terms of completeness and rigour if they are to be a reliable basis for follow-on research.     

End user computing in development administration: Importance of decision support systems

Abstract

Effective diffusion of microcomputers, as distinct from quick adoption, is concerned with a demand for applications. It requires a new approach on the part of information system professionals and administrators. Microcomputers will fail to contribute to the extent they should to Development if not accompanied by appropriate institutional reforms. A Decision Support System (DSS) facilitates the increasingly interdependent nature of the process, by linking information flows and organizational changes with improved decision making. The steps in the evolution of a decision support system for district administrators are discussed as a possible methodology for implementation. The paper concludes with practical guidelines for those initiating the process of end user computing, so that costly mistakes can be avoided.     

The use of software product lines for business process management: A systematic literature review

ContextBusiness Process Management (BPM) is a potential domain in which Software Product Line (PL) can be successfully applied. Including the support of Service-oriented Architecture (SOA), BPM and PL may help companies achieve strategic alignment between business and IT.ObjectivePresenting the results of a study undertaken to seek and assess PL approaches for BPM through a Systematic Literature Review (SLR). Moreover, identifying the existence of dynamic PL approaches for BPM.MethodA SLR was conducted with four research questions formulated to evaluate PL approaches for BPM.Results63 papers were selected as primary studies according to the criteria established. From these primary studies, only 15 papers address the specific dynamic aspects in the context evaluated. Moreover, it was found that PLs only partially address the BPM lifecycle since the last business process phase is not a current concern on the found approaches.ConclusionsThe found PL approaches for BPM only cover partially the BPM lifecycle, not taking into account the last phase which restarts the lifecycle. Moreover, no wide dynamic PL proposal was found for BPM, but only the treatment of specific dynamic aspects. The results indicate that PL approaches for BPM are still at an early stage and gaining maturity.     

Supporting distributed product configuration by integrating heterogeneous variability modeling approaches

ContextIn industrial settings products are developed by more than one organization. Software vendors and suppliers commonly typically maintain their own product lines, which contribute to a larger (multi) product line or software ecosystem. It is unrealistic to assume that the participating organizations will agree on using a specific variability modeling technique—they will rather use different approaches and tools to manage the variability of their systems.ObjectiveWe aim to support product configuration in software ecosystems based on several variability models with different semantics that have been created using different notations.MethodWe present an integrative approach that provides a unified perspective to users configuring products in multi product line environments, regardless of the different modeling methods and tools used internally. We also present a technical infrastructure and a prototype implementation based on web services.ResultsWe show the feasibility of the approach and its implementation by using it with the three most widespread types of variability modeling approaches in the product line community, i.e., feature-based, OVM-style, and decision-oriented modeling. To demonstrate the feasibility and flexibility of our approach, we present an example derived from industrial experience in enterprise resource planning. We further applied the approach to support the configuration of privacy settings in the Android ecosystem based on multiple variability models. We also evaluated the performance of different model enactment strategies used in our approach.ConclusionsTools and techniques allowing stakeholders to handle variability in a uniform manner can considerably foster the initiation and growth of software ecosystems from the perspective of software reuse and configuration.     

Guidelines for conducting systematic mapping studies in software engineering: An update

ContextSystematic mapping studies are used to structure a research area, while systematic reviews are focused on gathering and synthesizing evidence. The most recent guidelines for systematic mapping are from 2008. Since that time, many suggestions have been made of how to improve systematic literature reviews (SLRs). There is a need to evaluate how researchers conduct the process of systematic mapping and identify how the guidelines should be updated based on the lessons learned from the existing systematic maps and SLR guidelines.ObjectiveTo identify how the systematic mapping process is conducted (including search, study selection, analysis and presentation of data, etc.); to identify improvement potentials in conducting the systematic mapping process and updating the guidelines accordingly.MethodWe conducted a systematic mapping study of systematic maps, considering some practices of systematic review guidelines as well (in particular in relation to defining the search and to conduct a quality assessment).ResultsIn a large number of studies multiple guidelines are used and combined, which leads to different ways in conducting mapping studies. The reason for combining guidelines was that they differed in the recommendations given.ConclusionThe most frequently followed guidelines are not sufficient alone. Hence, there was a need to provide an update of how to conduct systematic mapping studies. New guidelines have been proposed consolidating existing findings.     

Development of a decision support system for machining center selection

The machining centers are key resources for manufacturing companies in their dealing with their fierce competitive market environments. However, although selecting the most appropriate machining center is a very important decision for manufacturing companies, the availability of wide-range of types and models makes the selection process a complex and difficult task. In this study, a decision support system (DSS), namely MACSEL, is developed to help the decision makers in their machining center selection decisions. Several issues and applicability of the MACSEL is illustrated with case problems in the paper.Within the developed DSS, to select the feasible set of machining centers fifteen questions are placed in the elimination (pre-selection) module. The developed DSS uses fuzzy analytical hierarchy process (FAHP) or fuzzy technique for order preference by similarity to ideal solution (FTOPSIS), which are extended versions of multi-criteria decision making approaches, to rank the feasible machining centers. In the DSS, FAHP is used if a detailed pair-wise weighting of the hierarchically structured criteria is wanted. On the other hand, when a simpler separate weighting of each criterion is be considered as enough, FTOPSIS is used.     

Cognitive biases and decision support systems development: a design science approach

Abstract.  This paper presents design science research that aims to improve decision support systems (DSS) development in organizations. Evolutionary development has been central to DSS theory and practice for decades, but a significant problem for DSS analysts remains how to conceptualize the improvement of a decision task during evolutionary DSS development. The objective of a DSS project is to improve the decision process and outcome for a manager making an important decision. The DSS analyst needs to have a clear idea of the nature of the target decision task and a clear strategy of how to support the decision process. Existing psychological research was examined for help with the conceptualization problem, and the theory of cognitive bias is proposed as a candidate for this assistance. A taxonomy of 37 cognitive biases that codifies a complex area of psychological research is developed. The core of the project involves the construction of a design artefact – an evolutionary DSS development methodology that uses cognitive bias theory as a focusing construct, especially in its analysis cycles. The methodology is the major contribution of the project. The feasibility and effectiveness of the development methodology are evaluated in a participatory case study of a strategic DSS project where a managing director is supported in a decision about whether to close a division of a company.     

An empirical study on software defect prediction with a simplified metric set

ContextSoftware defect prediction plays a crucial role in estimating the most defect-prone components of software, and a large number of studies have pursued improving prediction accuracy within a project or across projects. However, the rules for making an appropriate decision between within- and cross-project defect prediction when available historical data are insufficient remain unclear.ObjectiveThe objective of this work is to validate the feasibility of the predictor built with a simplified metric set for software defect prediction in different scenarios, and to investigate practical guidelines for the choice of training data, classifier and metric subset of a given project.MethodFirst, based on six typical classifiers, three types of predictors using the size of software metric set were constructed in three scenarios. Then, we validated the acceptable performance of the predictor based on Top-k metrics in terms of statistical methods. Finally, we attempted to minimize the Top-k metric subset by removing redundant metrics, and we tested the stability of such a minimum metric subset with one-way ANOVA tests.ResultsThe study has been conducted on 34 releases of 10 open-source projects available at the PROMISE repository. The findings indicate that the predictors built with either Top-k metrics or the minimum metric subset can provide an acceptable result compared with benchmark predictors. The guideline for choosing a suitable simplified metric set in different scenarios is presented in Table 12.ConclusionThe experimental results indicate that (1) the choice of training data for defect prediction should depend on the specific requirement of accuracy; (2) the predictor built with a simplified metric set works well and is very useful in case limited resources are supplied; (3) simple classifiers (e.g., Naïve Bayes) also tend to perform well when using a simplified metric set for defect prediction; and (4) in several cases, the minimum metric subset can be identified to facilitate the procedure of general defect prediction with acceptable loss of prediction precision in practice.     

Using metrics in Agile and Lean Software Development – A systematic literature review of industrial studies

ContextSoftware industry has widely adopted Agile software development methods. Agile literature proposes a few key metrics but little is known of the actual metrics use in Agile teams.ObjectiveThe objective of this paper is to increase knowledge of the reasons for and effects of using metrics in industrial Agile development. We focus on the metrics that Agile teams use, rather than the ones used from outside by software engineering researchers. In addition, we analyse the influence of the used metrics.MethodThis paper presents a systematic literature review (SLR) on using metrics in industrial Agile software development. We identified 774 papers, which we reduced to 30 primary studies through our paper selection process.ResultsThe results indicate that the reasons for and the effects of using metrics are focused on the following areas: sprint planning, progress tracking, software quality measurement, fixing software process problems, and motivating people. Additionally, we show that although Agile teams use many metrics suggested in the Agile literature, they also use many custom metrics. Finally, the most influential metrics in the primary studies are Velocity and Effort estimate.ConclusionThe use of metrics in Agile software development is similar to Traditional software development. Projects and sprints need to be planned and tracked. Quality needs to be measured. Problems in the process need to be identified and fixed. Future work should focus on metrics that had high importance but low prevalence in our study, as they can offer the largest impact to the software industry.     

A Semiautomatic Experience-Based Tool for Solving Product Innovation Problem

ABSTRACT

In this paper, we present the idea of Smart Innovation Engineering (SIE) System and its implementation methodology. The SIE system is semiautomatic system that helps in carrying the process of product innovation. It collects the experiential knowledge from the formal decisional events. This experiential knowledge is collected from the group of similar products having some common functions and features. The SIE system behaves like a group of experts in its domain as it collects, captures, and stores the experiential knowledge from similar products as well as reuses this experiential knowledge that ultimately enhances the innovation process of manufactured goods. Moreover, with SIE in hand, entrepreneurs and manufacturing organizations will be able to take proper, enhanced decisions and most importantly at appropriate time. The system gains expertise each time a decision is taken and stored in the form of set of experience that can be used in future for similar queries. Implementation of the SIE system using Set of Experience Knowledge Structure and Decisional DNA for case study suggests that the SIE system is capable of capturing and reusing the innovation-related experiences of the manufactured products. The case study confirmed that the SIE system can be beneficial for entrepreneurs and manufacturing organizations for efficient decision making in the product innovation process.