A multicriteria sorting approach based on data envelopment analysis for R&D project selection problem

In this paper, multiple criteria sorting methods based on data envelopment analysis (DEA) are developed to evaluate research and development (R&D) projects. The weight intervals of the criteria are obtained from Interval Analytic Hierarchy Process and employed as the assurance region constraints of models. Based on data envelopment analysis, two threshold estimation models, and five assignment models are developed for sorting. In addition to sorting, these models also provide ranking of the projects. The developed approach and the well-known sorting method UTADIS are applied to a real case study to analyze the R&D projects proposed to a grant program executed by a government funding agency in 2009. A five level R&D project selection criteria hierarchy and an assisting point allocation guide are defined to measure and quantify the performance of the projects. In the case study, the developed methods are observed to be more stable than UTADIS.     

R&D project evaluation: An integrated DEA and balanced scorecard approach

We present and demonstrate a multi-criteria approach for evaluating R&D projects in different stages of their life cycle. Our approach integrates the balanced scorecard (BSC) and data envelopment analysis (DEA) and develops an extended DEA model. The input and output measures for the integrated DEA–BSC model are grouped in “cards” which are associated with a “BSC for R&D projects”. The BSC is embedded in the DEA model through a hierarchical structure of constraints that reflect the BSC balance considerations. We illustrate the proposed approach with a case study involving an industrial research laboratory that selects and executes dozens of R&D projects every year.     

The performance evaluation of regional R&D investments in China: An application of DEA based on the first official China economic census data

R&D (Research and Development) activities represent the basic core of corporate science and technology activities, and play a crucial role in enhancing the ability of companies to achieve rapid and sustainable growth. In recent years, the total R&D investments in China have increased significantly and the proportion of the industrial investments in R&D activities relative to national R&D investments has increased rapidly. In order to investigate the effectiveness of these R&D investments, we utilize Data Envelopment Analysis (DEA) models to evaluate the relative efficiencies of 30 regional R&D investments using the First Official China Economic Census Data in 2004. Our investigation and study indicate the following: (1) Only six provinces are global technical efficient and the performance of regional R&D investments in China needs to improve dramatically. (2) Increasing returns to scale have not yet occurred in any province. Constant returns to scale have prevailed in most provinces in the Western region, and decreasing returns to scale have prevailed in most provinces in the Eastern and Central regions. (3) There are no direct relationships between global technical efficiency and the amount of R&D investment. The Western region has the highest average radial efficiency score, followed by the Eastern region, and then by the Central region. (4) The Eastern region has advantages in local technical efficiency, the Western region has advantages in scale efficiency, while the Central region has neither technical efficiency advantages nor scale efficiency advantages. Suggestions are proposed to improve efficiencies of regional R&D investments.     

Ranking efficient units in DEA

Super-efficiency data envelopment analysis (DEA) model can be used in ranking the performance of efficient decision making units (DMUs). Because of the infeasibility problem associated with the super-efficiency DEA model, ranking has been restricted to the model where constant returns to scale and proportional changes in all inputs or all outputs are assumed. In fact, when super-efficiency is used as an efficiency stability measure, infeasibility means the highest super-efficiency. However, if super-efficiency is interpreted as input saving or output surplus achieved by a specific efficient DMU, infeasibility does not necessarily mean the highest super-efficiency. In order to obtain a complete ranking of efficient DMUs when the two assumptions are relaxed, a modified super-efficiency DEA model is proposed to overcome the infeasibility problem and to correctly capture the super-efficiency represented by the input saving or the output surplus. The current paper suggests using both input- and output-oriented super-efficiency models to characterize the super-efficiency when infeasibility occurs. As a result, we can rank the efficient DMUs if infeasibility occurs. The approach is applied to 20 largest Japanese companies and 15 US cities, respectively.     

Theory of integer-valued data envelopment analysis under alternative returns to scale axioms

Kuosmanen and Kazemi Matin [Theory of integer valued data envelopment analysis. European Journal of Operational Research 2009;192:658–67] developed an axiomatic foundation for a data envelopment analysis (DEA) model that assumes subsets of input and output variables to be integer valued. In this paper we extend and generalize the axiomatic foundation for the integer DEA under variable, non-decreasing and non-increasing returns to scale. These model variants are achieved by introducing new axioms of natural convexity and natural augmentability. We also develop mixed integer linear programming (MILP) formulations for computing efficiency scores in these environments. An empirical example illustrates the approach.     

R&D,productivity, and market value: An empirical study from high-technology firms

Although prior research has addressed the influence of production activity and research and development (R&D) on productivity, it is not clear whether production and R&D affect the market value of a firm. This study proposes and verifies an R&D value chain framework to explore the relationship among productivity, R&D, and firm market values, as measured by Tobin's q theory. By doing so, we attempt to link new theoretical insights and empirical evidence on the effects of R&D efforts and basic production activities to the market valuations of high-technology firms. The value chain data envelopment analysis approach was proposed to estimate parallel-serial processes of basic operations and R&D efforts. This approach can be used to simultaneously estimate the profitability efficiency and marketability efficiency of high-technology firms. This area has rarely been studied, but it is particularly important for high-technology R&D policies and for further industrial development. Using the R&D value chain perspectives of model innovations and extensions proposed in several previous studies, we examined the appropriate levels of intermediate outputs. Production efficiency and R&D were combined to estimate the appropriate levels of intermediate outputs for high-technology firms. Based on the intermediate output analyses, we developed an R&D efforts decision matrix to explore and identify operational and R&D efficiency for high-technology firms. Our sample firms are displayed on a four-quadrant action grid that provides visual information on current short-term operational efficiency and decision making on long-term R&D strategic positions. The empirical findings from the R&D value chain model can provide information for policymakers and managers and suggest the adoption of various policies that place more emphasis on profitability and marketability strategies.     

An empirical analysis of the role of reference point in justice perception in R&D settings in Korea

This study examines the relationship of the respondent's reference point with regards to distributive and procedural justice in an R&D setting in Korea. The two characteristics: (1) the rewards difference between Korean public and private R&D institutes (the outcome level at the public R&D institutes is much less than that at the private ones), (2) and the similarity of procedures being used to determine R&D professional's rewards at both types of research institutes, provide an excellent real-life setting for studying the role of reference point. To avoid common method variance, we measured distributive and procedural justice separately. Consistent with the previous experiments, R&D professionals at the public R&D institutes indicated differentiated distributive justice levels according to R&D professionals' reference points. Results also showed that the presence of a social comparison other (suggested reference point) leads the R&D professionals at public R&D institutes to perceive procedural injustice. Finally, several theoretical and managerial implications were discussed.     

Evaluating the efficiency of dual-use technology development programs from the R&D and socio-economic perspectives

This study applies a network data envelopment analysis model called network slacks-based measure (NSBM) to examine the R&D efficiency and socio-economic efficiency of Taiwan's dual-use (military and general public) technology development programs. We also employ the additive super-efficiency model to distinguish programs at different stages and conduct an analysis on the strengths and weaknesses of internal resource utilization. The results of this study show that the R&D efficiency is substantially higher than the socio-economic efficiency. Among the dual-use technology development programs, material and chemical engineering programs demonstrate relatively higher performance. This study provides insights into the policy of resource utilization.     

Enhancing standard performance practices with DEA

Data envelopment analysis (DEA) can be interpreted as a natural generalization of elemental practices for efficiency and performance evaluation used in diverse areas, some of which have evolved independently of each other and of the traditional efficiency framework of Charnes et al. [Measuring the efficiency of decision making units. European Journal of Operational Research 1978;2(6):429–44]. The paper aims to show that some standard “real-world” procedures employed by practitioners in the fields of retailing, finance, and social science can be interpreted as unconscious applications of DEA, and that such analyses could be enhanced by the explicit use of DEA. A supplementary aim is to demonstrate that DEA can be derived from concrete examples and first principles. The choice of the examples covers the two typical returns to scale environments of constant returns to scale (CRS) and variable returns to scale (VRS). Motivating and deriving DEA as a generalization of established performance evaluation practices will help to explain and disseminate this methodology to practitioners and to those who find their methods familiar.     

Research fronts in data envelopment analysis

Research activities relating to data envelopment analysis (DEA) have grown at a fast rate recently. Exactly what activities have been carrying the research momentum forward is a question of particular interest to the research community. The purpose of this study is to find these research activities, or research fronts, in DEA. A research front refers to a coherent topic or issue addressed by a group of research articles in recent years. The large amount of DEA literature makes it difficult to use any traditional qualitative methodology to sort out the matter. Thus, this study applies a network clustering method to group the literature through a citation network established from the DEA literature over the period 2000 to 2014. The keywords of the articles in each discovered group help pinpoint its research focus. The four research fronts identified are “bootstrapping and two-stage analysis”, “undesirable factors”, “cross-efficiency and ranking”, and “network DEA, dynamic DEA, and SBM”. Each research front is then examined with key-route main path analysis to uncover the elements in its core. In addition to presenting the research fronts, this study also updates the main paths and author statistics of DEA development since its inception and compares them with those reported in a previous study.     

Super-efficiency based on a modified directional distance function

The problem of infeasibility arises in conventional radial super-efficiency data envelopment analysis (DEA) models under variable returns to scale (VRS). To tackle this issue, a Nerlove–Luenberger (N–L) measure of super-efficiency is developed based on a directional distance function. Although this N–L super-efficiency model does not suffer infeasibility problem as in the conventional radial super-efficiency DEA models, it can produce an infeasible solution in two special situations. The current paper proposes to modify the directional distance function by selecting proper feasible reference bundles so that the resulting N–L measure of super-efficiency is always feasible. As a result, our modified VRS super-efficiency model successfully addresses the infeasibility issues occurring either in conventional VRS models or the N–L super-efficiency model. Numerical examples are used to demonstrate our approach and compare results obtained from various super-efficiency measures.     

Within-cluster determinants of output efficiency of R&D in the space industry

Dynamic slacks-based data envelopment analysis is applied to measure output performance efficiency of space R&D active private firms. Fractional logit panel analysis is used to identify associations between this firm level R&D output efficiency and space industry level R&D specialisation, presence of anchor firms, and public R&D capacity within geographical clusters.Primary data collected for the entire population of space R&D active firms in Belgium for the period 2011–2015 reveals that an environment specialized in the focus firm's space activities exerts a positive influence on R&D output efficiency. The public space R&D capacity and the presence of space anchor firms within the cluster do not influence R&D output efficiency. Firm age, a mixture of space and non-space R&D, and larger amounts of public funding positively affect R&D output efficiency. These findings should be seen in a context of a space industry dominated by SMEs and in a five year time-span.     

Task Design,Team Context,and Psychological Safety: An Empirical Analysis of R&D Projects in High Technology Organizations

High technology organizations need to develop new products or processes that address the dual goals of exploration and exploitation. The competing viewpoints and the asymmetric nature of market returns associated with these goals in R&D projects can heighten stress levels among project team members and reduce their psychological safety. While current research calls for greater focus on task design for improving psychological safety, we know little about how team contextual factors affect this relationship. This study develops and tests a conceptual framework that examines the moderating role of R&D team contextual factors, namely, relative exploration and project‐organization metric alignment on the relationship between a key task design variable, namely, team autonomy, and psychological safety. Relative exploration captures the extent to which exploration goals are emphasized over exploitation goals in an R&D project, while project‐organization metric alignment measures the extent to which project metrics are aligned with broader organizational metrics. Furthermore, we examine the performance consequences of psychological safety in R&D projects. The empirical analysis is conducted using primary data collected from multiple informants across 110 R&D projects in 34 high technology business units. Our results indicate that relative exploration and project‐organization metric alignment have contrasting moderating effects. Furthermore, the effect of psychological safety on project performance is found to be indirect and mediated through team turnover. Implications of the study findings, limitations, and directions for future research are discussed.     

基于完全包络面的DEA Super效率评价方法

为了对决策单元进行公平、合理、完全的排序,提出一种考虑完全包络面的DEA Super效率评价方法.讨论包络面的选取与DEA效率值的关系,提出完全包络面的概念,并在等产出图上演示了最优前沿面、最劣前沿面与完全包络面的区别和关系;提出基于最劣前沿面的Super效率DEA模型,给出基于完全包络面的DEA Super效率评价排序模型,目标函数包含决策单元的最优效率和最劣效率两部分.该方法用于决策单元的排序,同时考虑最符合与最不符合决策单元自身偏好的权重体系;该方法用于决策单元的效率排序,能获得比较合理的排序结果,可以实际应用于各种决策单元排序中.     

DEA game cross-efficiency approach to Olympic rankings

A number of studies have used data envelopment analysis (DEA) to evaluate the performance of the countries in Olympic games. While competition exists among the countries in Olympic games/rankings, all these DEA studies do not model competition among peer decision making units (DMUs) or countries. These DEA studies find a set of weights/multipliers that keep the efficiency scores of all DMUs at or below unity. Although cross efficiency goes a further step by providing an efficiency measure in terms of the best multiplier bundle for the unit and all the other DMUs, it is not always unique. This paper presents a new and modified DEA game cross-efficiency model where each DMU is viewed as a competitor via non-cooperative game. For each competing DMU, a multiplier bundle is determined that optimizes the efficiency score for that DMU, with the additional constraint that the resulting score should be at or above that DMU 's estimated best performance. The problem, of course, arises that we will not know this best performance score for the DMU under evaluation until the best performances of all other DMUs are known. To combat this “chicken and egg” phenomenon, an iterative approach leading to the Nash equilibrium is presented. The current paper provides a modified variable returns to scale (VRS) model that yields non-negative cross-efficiency scores. The approach is applied to the last six Summer Olympic Games. Our results may indicate that our game cross-efficiency model implicitly incorporates the relative importance of gold, silver and bronze medals without the need for specifying the exact assurance regions.     

Two-level DEA approaches in research evaluation

It is well known that the discrimination power of data envelopment analysis (DEA) models will be much weakened if too many input or output indicators are used. It is a dilemma if decision makers wish to select comprehensive indicators, which often have some hierarchical structures, to present a relatively holistic evaluation using DEA. In this paper we show that it is possible to develop DEA models that utilize hierarchical structures of input–output data so that they are able to handle quite large numbers of inputs and outputs. We present two approaches in a pilot evaluation of 15 institutes for basic research in the Chinese Academy of Sciences using the DEA models.     

A fuzzy set approach for R&D portfolio selection using a real options valuation model

Making R&D portfolio decision is difficult, because long lead times of R&D and market and technology dynamics lead to unavailable and unreliable collected data for portfolio management. The objective of this research is to develop a fuzzy R&D portfolio selection model to hedge against the R&D uncertainty. Fuzzy set theory is applied to model uncertain and flexible project information. Since traditional project valuation methods often underestimate the risky project, a fuzzy compound-options model is used to evaluate the value of each R&D project. The R&D portfolio selection problem is formulated as a fuzzy zero–one integer programming model that can handle both uncertain and flexible parameters to determine the optimal project portfolio. A new transformation method based on qualitative possibility theory is developed to convert the fuzzy portfolio selection model into a crisp mathematical model from the risk-averse perspective. The transformed model can be solved by an optimization technique. An example is used to illustrate the proposed approach. We conclude that the proposed approach can assist decision makers in selecting suitable R&D portfolios, while there is a lack of reliable project information.     

Efficiency evaluations with context-dependent and measure-specific data envelopment approaches: An application in a World Bank supported project

We evaluate the efficiency of decision making units (DMUs) in a World Bank supported Social Risk Mitigation Project (SRMP) in Turkey through context-dependent and measure-specific data envelopment analysis (DEA) approaches. The results suggest that the efficiency evaluations with context-dependent and measure-specific DEA play various roles in an organization such as setting attainable targets to DMUs, setting long and short term targets to DMUs separately, grouping of DMUs, and improving internal competition between DMUs. Four main contributions of this study can be summarized as follows. Firstly, the study shows the applicability of context-dependent and measure-specific DEA methodologies in a World Bank supported large scale project to increase the effectiveness of the project. Secondly, it outlines some important managerial conclusions of context-dependent DEA clustering approach. Moreover, we propose an alternative approach for attractiveness scores computations in case of exogenous group formations. Finally, the study proposes and applies measure-specific version of context-dependent DEA approach.     

Benchmarking firm performance from a multiple-stakeholder perspective with an application to Chinese banking

We know very little about how a firm would compare against its peers when evaluated from a multiple-stakeholder perspective where the same variables are interpreted differently. Since most medium-to-large organizations acknowledge the multi-dimensional nature of their operations, finding out the performance evaluations of various stakeholders can inform managerial decision-making. Thus, the main motivation for this study is to capture the interactions among different perceptions on a common set of performance measures. Using data envelopment analysis (DEA), we work with an approach that allows a flexible designation of inputs and outputs based on varying perspectives of five key stakeholders in banking. The versatile approach demonstrates that different views from the stakeholder universe can be summarily captured in DEA scores. A numerical example on Chinese commercial banks identifies the compliant (efficient) banks versus rigid (inefficient) banks, as well as the amenable stakeholders (those evaluating banks as efficient) versus the recalcitrant stakeholders (those evaluating banks as inefficient). The aligned views held by management and shareholders as evidenced by significant correlation among performance rankings imply reduced agency costs. Similarly, shared perceptions between customers and employees may encourage management to examine how this important business interface can be improved.     

A three-stage hybrid approach for weight assignment in MADM

How to determine weights for attributes is one of the key issues in multiple attribute decision making (MADM). This paper aims to investigate a new approach for determining attribute weights based on a data envelopment analysis (DEA) model without explicit inputs (DEA-WEI) and minimax reference point optimisation. This new approach first considers a set of preliminary weights and the most favourite set of weights for each alternative or decision making unit (DMU) and then aggregates these weight sets to find the best compromise weights for attributes with the interests of all DMUs taken into account fairly and simultaneously. This approach is intended to support the solution of such MADM problems as performance assessment and policy analysis where (a) the preferences of decision makers (DMs) are either unclear and partial or difficult to acquire and (b) there is a need to consider the best "will" of each DMU. Two case studies are conducted to show the property of this new proposed approach and how to use it to determine weights for attributes in practice. The first case is about the assessment of research strengths of EU-28 member countries under certain measures and the second is for analysing the performances of Chinese Project 985 universities, where the weights of the attributes need to be assigned in a fair and unbiased manner.