Forty years of research on personality in software engineering: A mapping study

In this article, we present a systematic mapping study of research on personality in software engineering. The goal is to plot the landscape of current published empirical and theoretical studies that deal with the role of personality in software engineering. We applied the systematic review method to search and select published articles, and to extract and synthesize data from the selected articles that reported studies about personality. Our search retrieved more than 19,000 articles, from which we selected 90 articles published between 1970 and 2010. Nearly 72% of the studies were published after 2002 and 83% of the studies reported empirical research findings. Data extracted from the 90 studies showed that education and pair programming were the most recurring research topics, and that MBTI was the most used test. Research related to pair programming, education, team effectiveness, software process allocation, software engineer personality characteristics, and individual performance concentrated over 88% of the studies, while team process, behavior and preferences, and leadership performance were the topics with the smallest number of studies. We conclude that the number of articles has grown in the last few years, but contradictory evidence was found that might have been caused by differences in context, research method, and versions of the tests used in the studies. While this raises a warning for practitioners that wish to use personality tests in practice, it shows several opportunities for the research community to improve and extend findings in this field.     

Empirical Research in Software Engineering — A Literature Survey

Empirical research is playing a significant role in software engineering (SE), and it has been applied to evaluate software artifacts and technologies. There have been a great number of empirical research articles published recently. There is also a large research community in empirical software engineering (ESE). In this paper, we identify both the overall landscape and detailed implementations of ESE, and investigate frequently applied empirical methods, targeted research purposes, used data sources, and applied data processing approaches and tools in ESE. The aim is to identify new trends and obtain interesting observations of empirical software engineering across different sub-fields of software engineering. We conduct a mapping study on 538 selected articles from January 2013 to November 2017, with four research questions. We observe that the trend of applying empirical methods in software engineering is continuously increasing and the most commonly applied methods are experiment, case study and survey. Moreover, open source projects are the most frequently used data sources. We also observe that most of researchers have paid attention to the validity and the possibility to replicate their studies. These observations are carefully analyzed and presented as carefully designed diagrams. We also reveal shortcomings and demanded knowledge/strategies in ESE and propose recommendations for researchers.     

An analysis of the most cited articles in software engineering journals—1999

Citations and related work are crucial in any research to position the work and to build on the work of others. A high citation count is an indication of the influence of specific articles. The importance of citations means that it is interesting to analyze which articles are cited the most. Such an analysis has been conducted using the ISI Web of Science to identify the most cited software engineering journal articles published in 1999. The objective of the analysis is to identify and list the articles that have influenced others the most as measured by citation count. An understanding of which research is viewed as most valuable to build upon may provide valuable insights into what research to focus on now and in the future. Based on the analysis, a list of the 20 most cited articles is presented here. The intention of the analysis is twofold. First, to actually show the most cited articles, and second, to invite the authors of the most cited articles in 1999 to contribute to a special issue of Information and Software Technology. Five invited authors have accepted the invitation and their articles are appearing in this special issue. Moreover, the research topics and methods of the most cited articles in 1999 are compared with those from the most cited articles in 1994 to provide a picture of similarities and differences between the years.     

Investigations about replication of empirical studies in software engineering: A systematic mapping study

ContextTwo recent mapping studies which were intended to verify the current state of replication of empirical studies in Software Engineering (SE) identified two sets of studies: empirical studies actually reporting replications (published between 1994 and 2012) and a second group of studies that are concerned with definitions, classifications, processes, guidelines, and other research topics or themes about replication work in empirical software engineering research (published between 1996 and 2012).ObjectiveIn this current article, our goal is to analyze and discuss the contents of the second set of studies about replications to increase our understanding of the current state of the work on replication in empirical software engineering research.MethodWe applied the systematic literature review method to build a systematic mapping study, in which the primary studies were collected by two previous mapping studies covering the period 1996–2012 complemented by manual and automatic search procedures that collected articles published in 2013.ResultsWe analyzed 37 papers reporting studies about replication published in the last 17 years. These papers explore different topics related to concepts and classifications, presented guidelines, and discuss theoretical issues that are relevant for our understanding of replication in our field. We also investigated how these 37 papers have been cited in the 135 replication papers published between 1994 and 2012.ConclusionsReplication in SE still lacks a set of standardized concepts and terminology, which has a negative impact on the replication work in our field. To improve this situation, it is important that the SE research community engage on an effort to create and evaluate taxonomy, frameworks, guidelines, and methodologies to fully support the development of replications.     

Analysis of keyword networks in MIS research and implications for predicting knowledge evolution

New concepts and ideas build on older ones. This path dependence in knowledge evolution has promoted research to identify important knowledge elements, research trends, and opportunities by analyzing publication data. In our study, keyword networks formed from published academic articles were analyzed to examine how keywords are associated with each other and to identify important keywords and their change over time. Based on MIS publication data from 1999 to 2008, our analysis provided several notable findings. First, while the MIS field has changed rapidly, resulting in many new keywords, the connectivity among them is highly clustered. Second, the keyword networks show clear power-law distribution, which implies that the more popular a keyword, the more likely it is selected by new researchers and used in follow-on studies. In addition, a strong hierarchical structure is identified in the network. Third, the network-based perspective reveals interdisciplinary keywords which are different from popular ones and have the potential to lead research in the MIS field.     

On the nationality balance of authors and references in selected MIS journals

Citation analysis is an established technique in literature analysis. It can be used to reveal citation patterns and thus results stemming from the analysis, including co-citation patterns and groupings of researchers, even reveal paradigmatic schools within a certain field. In this article, twelve major MIS journals — six European and six American — are studied to find out whether there are any differences in author selection and referencing patterns. It was discovered that the American MIS community is much more self-sustaining than the European one, which relies heavily on work by American authors. This points either to the superiority of the American MIS community, or the under-appreciation of foreign research, or possibly both. On the European side, the result might be interpreted as a sign of low-quality research, lack of self-esteem and pride, or again both. Clear differences between the editorial policy of the journals can be discerned: some journals accept the American hegemony; some try to maintain a balance between contributions from different countries; and some favour European research.     

Empirical research in on-line trust: a review and critical assessment

Lack of trust is one of the most frequently cited reasons for consumers not purchasing from Internet vendors. During the last four years a number of empirical studies have investigated the role of trust in the specific context of e-commerce, focusing on different aspects of this multi-dimensional construct. However, empirical research in this area is beset by conflicting conceptualizations of the trust construct, inadequate understanding of the relationships between trust, its antecedents and consequents, and the frequent use of trust scales that are neither theoretically derived nor rigorously validated. The major objective of this paper is to provide an integrative review of the empirical literature on trust in e-commerce in order to allow cumulative analysis of results. The interpretation and comparison of different empirical studies on on-line trust first requires conceptual clarification. A set of trust constructs is proposed that reflects both institutional phenomena (system trust) and personal and interpersonal forms of trust (dispositional trust, trusting beliefs, trusting intentions and trust-related behaviours), thus facilitating a multi-level and multi-dimensional analysis of research problems related to trust in e-commerce.     

An analysis of the most cited articles in software engineering journals - 2000

Citations and related work are crucial in any research to position the work and to build on the work of others. A high citation count is an indication of the influence of specific articles. The importance of citations means that it is interesting to analyze which articles are cited the most. Such an analysis has been conducted using the ISI Web of Science to identify the most cited software engineering journal articles published in 2000. The objective of the analysis is to identify and list the articles that have influenced others the most as measured by citation count. An understanding of which research is viewed by the research community as most valuable to build upon may provide valuable insights into what research to focus on now and in the future. Based on the analysis, a list of the 20 most cited articles is presented here. The intention of the analysis is twofold. First, to identify the most cited articles, and second, to invite the authors of the most cited articles in 2000 to contribute to a special issue of Information and Software Technology. Five authors have accepted the invitation and their articles appear in this special issue. Moreover, an analysis of the most cited software engineering journal articles in the last 20 years is presented. The presentation includes both the most cited articles in absolute numbers and the most cited articles when looking at the average number of citations per year. The article describing the SPIN model checker by G.J. Holzmann published in 1997 is first on both these lists.     

What do human factors and ergonomics professionals value in research publications? Re-examining the research–practice gap

The research–practice gap is of concern in human factors/ergonomics (HF/E) as there is a belief that HF/E research may not be making an impact on practice in the ‘real world’. A potential issue is what researchers and practitioners perceive as important in HF/E journal articles as a primary means of conveying research findings to practitioners. This study examined the characteristics that make scientific journal articles appeal to HF/E researchers and practitioners using a web-based survey. HF/E researchers and practitioners were more similar than expected in judgements of important attributes and the selection of articles. Both practitioners and researchers considered practical significance to be more important than theoretical significance, in direct contrast to professionals from a related discipline – psychology. Well-written articles were appreciated across disciplines. The results signal a strong interest in practical applications in HF/E, but a relative lack of focus on development of theories that should be the basis for practical applications.     

What’s up with software metrics? - A preliminary mapping study

Background

Many papers are published on the topic of software metrics but it is difficult to assess the current status of metrics research.

Aim

This paper aims to identify trends in influential software metrics papers and assess the possibility of using secondary studies to integrate research results.

Method

Search facilities in the SCOPUS tool were used to identify the most cited papers in the years 2000-2005 inclusive. Less cited papers were also selected from 2005. The selected papers were classified according factors such as to main topic, goal and type (empirical or theoretical or mixed). Papers classified as “Evaluation studies” were assessed to investigate the extent to which results could be synthesized.

Results

Compared with less cited papers, the most cited papers were more frequently journal papers, and empirical validation or data analysis studies. However, there were problems with some empirical validation studies. For example, they sometimes attempted to evaluate theoretically invalid metrics and fail to appreciate the importance of the context in which data are collected.

Conclusions

This paper, together with other similar papers, confirms that there is a large body of research related to software metrics. However, software metrics researchers may need to refine their empirical methodology before they can answer useful empirical questions.     

An analysis of the most cited articles in software engineering journals – 2002

Citations and related work are crucial in any research to position the work and to build on the work of others. A high citation count is an indication of the influence of specific articles. The importance of citations means that it is interesting to analyze which articles are cited the most. Such an analysis has been conducted using the ISI Web of Science to identify the most cited software engineering journal articles published in 2002. The objective of the analysis is to identify and list the articles that have influenced others the most as measured by citation count. An understanding of which research is viewed by the research community as most valuable to build upon may provide valuable insights into what research to focus on now and in the future. Based on the analysis, a list of the 20 most cited articles is presented here. The intention of the analysis is twofold. First, to identify the most cited articles, and second, to invite the authors of the most cited articles in 2002 to contribute to a special section of Information and Software Technology. Six authors have accepted the invitation and their articles appear in this special section.     

Der Stand der Forschung zur Erfolgsmessung von Informationssystemen

Measuring information systems (IS) success is of great interest to both researchers and practitioners. This article examines multidimensional approaches to measuring IS success and explores the current state of IS success research through a literature review and by classifying articles published between 2003 and 2007. Based on a total of 41 academic journal and conference publications, the relevant research carried out is identified, while the research results are categorized, consolidated, and discussed. The results show that the dominant empirical research analyzes the individual impact of a certain type of information system by ascertaining users’ evaluation of it by means of surveys and then applying structural equation modeling. The DeLone and McLean Information Systems Success Model is the main theoretical basis of the reviewed empirical studies. This article provides researchers with a comprehensive review and structuring of IS success research. Furthermore, opportunities for additional development are identified and future research directions suggested.     

What does existing NeuroIS research focus on?

NeuroIS is a research field in which neuroscience theories and tools are used to better understand information systems phenomena. At present, NeuroIS is still an emerging field in information systems, and the number of available studies is limited. Because researchers who plan or execute NeuroIS research need to understand the status of the existing empirical research published in relevant journals, we have analyzed 78 empirical articles and put forward an integrative framework for understanding what existing NeuroIS research focuses on. Our framework is built upon stimulus–organism–response theory, which explains that stimulus factors can affect users’ psychological processes, which further lead to their responses. Then, we review the collected articles and summarize their findings to give more details of NeuroIS studies. Through this literature review, we identify several opportunities for future NeuroIS research in terms of influencing factors, measurement instruments, and subjects. We believe that our work will provide some meaningful insight for future NeuroIS research.     

A paradigmatic and methodological examination of information systems research from 1991 to 2001

Abstract. The field of information systems (IS) has evolved for more than three decades. Although many schools of thought have emerged and even become well established, few historical analyses of research paradigms and methodologies have been undertaken. One of the rare exceptions is Orlikowski & Baroudi (1991). Yet, the IS research community has evolved substantially since 1991 in many aspects. A variety of journal outlets have emerged and become well established. More attention has been paid to paradigmatic and methodological issues. Political and professional contexts have also changed noticeably. Therefore, it should be an opportune time for the field to ask: ‘What changes are manifested in journal publications?’‘Is the field making progress regarding pluralism in IS research?’‘How will the field's publications practices change in the future?’ The purpose of this paper is to investigate these questions and, in turn, reflect on the paradigmatic and methodological progress made since 1991. We examined 1893 articles published in eight major IS publication outlets between 1991 and 2001. Our findings suggest that the long‐term endeavours of interpretivist researchers might need to continue because the paradigmatic progress appears somewhat inconsequential; positivist research still dominates 81% of published empirical research. In particular, US journals, as opposed to European journals, tend to be more positivist, quantitative, cross‐sectional and survey oriented. With respect to research design, survey research is still the most widely used method (41%), although case studies have gained substantial recognition (36%). Further, the increase of qualitative research (30%), empirical studies (61%) and longitudinal cases (33%) at the expense of laboratory experiments (18%) might suggest that IS researchers have become more interested in obtaining scientific knowledge in real world settings. In summary, we suggest that the field has been dominated by the positivist paradigm, despite calls to the contrary. Indeed, if the field was to truly embrace pluralism, it would have to find ways to fundamentally change the publication practices of the journal system, including the current tenure and promotion system, which pose considerable obstacles for the acceptance of alternative paradigms.     

Replication of empirical studies in software engineering research: a systematic mapping study

In this article, we present a systematic mapping study of replications in software engineering. The goal is to plot the landscape of current published replications of empirical studies in software engineering research. We applied the systematic review method to search and select published articles, and to extract and synthesize data from the selected articles that reported replications. Our search retrieved more than 16,000 articles, from which we selected 96 articles, reporting 133 replications performed between 1994 and 2010, of 72 original studies. Nearly 70 % of the replications were published after 2004 and 70 % of these studies were internal replications. The topics of software requirements, software construction, and software quality concentrated over 55 % of the replications, while software design, configuration management, and software tools and methods were the topics with the smallest number of replications. We conclude that the number of replications has grown in the last few years, but the absolute number of replications is still small, in particular considering the breadth of topics in software engineering. We still need incentives to perform external replications, better standards to report empirical studies and their replications, and collaborative research agendas that could speed up development and publication of replications.     

On the use of construct reliability in MIS research: a meta-analysis

This paper reports results of a meta-analysis of construct reliability measures reported in MIS research. Based on 418 measures from 63 articles published in four major journals, it is observed that although scales developed more recently are no more reliable than those developed in the 1980s, the reliability of most scales reported in the literature is above generally accepted levels. However, for scales used in basic research, more than 40% of them had a reliability lower than the minimal acceptable level, at least 0.80. Scales using interviews for data collection; with more items, obtained from previous studies; and generated from both literature review and interview, were found to have higher reliability. Other research design characteristics, including sample size, type of subjects, scale type, scale format and number of scale points, were found to be insignificant in affecting reliability. Implications of the findings are discussed and guidelines for researchers on the design of their research methodologies with respect to having scales with the level of reliability suitable for the types of research at hand are provided.     

Machine Learning Techniques for Software Maintainability Prediction: Accuracy Analysis

Maintaining software once implemented on the end-user side is laborious and, over its lifetime, is most often considerably more expensive than the initial software development. The prediction of software maintainability has emerged as an important research topic to address industry expectations for reducing costs, in particular, maintenance costs. Researchers and practitioners have been working on proposing and identifying a variety of techniques ranging from statistical to machine learning (ML) for better prediction of software maintainability. This review has been carried out to analyze the empirical evidence on the accuracy of software product maintainability prediction (SPMP) using ML techniques. This paper analyzes and discusses the findings of 77 selected studies published from 2000 to 2018 according to the following criteria: maintainability prediction techniques, validation methods, accuracy criteria, overall accuracy of ML techniques, and the techniques offering the best performance. The review process followed the well-known systematic review process. The results show that ML techniques are frequently used in predicting maintainability. In particular, artificial neural network (ANN), support vector machine/regression (SVM/R), regression &; decision trees (DT), and fuzzy &; neuro fuzzy (FNF) techniques are more accurate in terms of PRED and MMRE. The N-fold and leave-one-out cross-validation methods, and the MMRE and PRED accuracy criteria are frequently used in empirical studies. In general, ML techniques outperformed non-machine learning techniques, e.g., regression analysis (RA) techniques, while FNF outperformed SVM/R, DT, and ANN in most experiments. However, while many techniques were reported superior, no specific one can be identified as the best.

     

Usability: what's it all about?

Software usability is a term which frequently appears in trade journals, industry project plans, and in research articles. This article discusses the qualities that make up usability and how these qualities can be measured. The practice of usability is discussed and problem areas are identified. The article notes the relevance of the papers in this edition to the current problems facing usability practitioners.     

Decision variables for selecting prototyping in information systems development: A Delphi study of MIS managers

The use of prototyping in the information systems development process is frequently advocated by both practitioners and academics; however, the decision variables that must be assessed when prototyping is considered have not been well defined. A literature review produced nineteen variables that potentially matter. Then a Delphi panel of MIS managers from Computerworld's Premier 100 firms, who are currently using prototyping, identified and ranked variables they consider when deciding whether to include prototyping in their systems development project. After three Delphi rounds, ten important decision variables emerged as most important, including one added during the process by the panel. Seven variables frequently cited in the literature were ranked low and hence are considered unimportant.