Performance measurement in R&D: exploring the interplay between measurement objectives, dimensions of performance and contextual factors

Measuring research and development (R&D) performance has become a fundamental concern for R&D managers and executives in the last decades. As a result, the issue has been extensively debated in innovation and R&D management literature. The paper contributes to this growing body of knowledge, adopting a systemic and contextual perspective to look into the problem of measuring R&D performance. In particular, it explores the interplay between measurement objectives, performance dimensions and contextual factors in the design of a performance measurement system (PMS) for R&D activities. The paper relies on a multiple case study analysis that involved 15 Italian technology-intensive firms. The results indicate that firms measure R&D performance with different purposes, i.e. motivate researchers and engineers, monitor the progress of activities, evaluate the profitability of R&D projects, favour coordination and communication and stimulate organisational learning. These objectives are pursued in clusters, and the importance firms attach to each cluster is influenced by the context (type of R&D, industry belonging, size) in which measurement takes place. Furthermore, a firm's choice to measure R&D performance along a particular perspective (i.e. financial, customer, business processes or innovation and learning) is influenced by the classes of objectives (diagnostic, motivational or interactive) that are given higher priority. The implications of these results for R&D managers and scholars are discussed in the paper.     

Capital Budgeting Practices for R&D: A Survey and Analysis of Business Week's R&D Scoreboard

This paper discusses results of a survey of U.S. Corporations regarding the extent of the utilization of standard capital budgeting measures of investment worth to evaluate R&D projects. This study analyzes responses by three types of R&D activity: Basic, Applied and Development research. The results indicate that discounted cash flow methods are widely utilized in the evaluation of Development projects, and the analysis of Development projects is as sophisticated as the evaluation of non-R&D projects. The use of discounted cash flow analysis techniques for Applied projects is considerably less widespread than for Development projects. However, the utilization of Net Present Value and Internal Rate of Return increases with firm size. Basic research projects are much less likely to be subjected to a formal economic analysis and are generally thought of as being “strategic” investments. This research also indicates that post-audits of Applied and Development projects are based on economic measures, whereas Basic research projects focus on physical or operational goals. Lastly, this research indicates that most R&D projects are terminated because R&D priorities have changed rather than because they are not technically feasible.     

Integrating the R&D and Patent Functions: Implications for New Product Performance

This paper examines the impact of cross‐functional integration between the research and development (R&D) and the patent functions on new product development (NPD) performance. The attitudinal (collaboration) and the behavioral (contributions of the patent function to NPD) dimension of cross‐functional integration between the R&D and the patent functions are distinguished. It is also investigated if the level of innovativeness moderates the relationship between the attitudinal and the behavioral dimension of cross‐functional integration between the R&D and the patent department and NPD performance. The four hypotheses are tested based on a multi‐informant sample of 101 NPD projects which are nested within 72 technology‐based firms or strategic business units from multiple industries in Germany. The results show that the attitudinal and the behavioral dimensions of cross‐functional integration between the R&D and the patent functions have a significant and positive impact on NPD performance. This lends empirical support for the notion expressed in the literature that certain managerial capabilities are important for understanding the effect of patenting on appropriability outcomes such as value creation and performance. The level of cross‐functional integration between the patent and the R&D functions appears to be one of these critical patent management capabilities that affect the returns from investments into patents. There is support for the hypothesis that the context matters for the effect of cross‐functional integration between the R&D and the patent functions on NPD performance. In line with the initial hypothesis, the level of innovativeness positively moderates the impact of the behavioral dimension of cross‐functional integration between the R&D and the patent department on NPD performance. In contrast to the initial hypothesis, the findings reveal no moderating effect of the level of innovativeness on the link between the attitudinal dimension of cross‐functional integration between the R&D and the patent department and NPD performance. This implies that joint objectives and an open and trustful working relationship between the R&D and the patent functions are not sufficient for achieving higher NPD performance if firms aim to develop very innovative products. In the case of highly innovative products, the actual behavior, that is, the specific contributions of the patent department to the NPD project, matters. Overall, these findings have important implications for improving performance by means of effectively integrating the patent and the R&D functions during NPD.     

Achieving contextual ambidexterity in R&D organizations: a management control system approach

Research on how managers control R&D activities has tended to focus on the performance measurement systems used to exploit existing knowledge and capabilities. This focus has been at the expense of how broader forms of management control could be used to enable R&D contextual ambidexterity, the capacity to attain appropriate levels of exploitation and exploration behaviors in the same R&D organizational unit. In this paper, we develop a conceptual framework for understanding how different types of control system, guided by different R&D strategic goals, can be used to induce and balance both exploitation and exploration. We illustrate the elements of this framework and their relations using data from biotechnology firms, and then discuss how the framework provides a basis to empirically examine a number of important control relationships and phenomena.     

International R&D partnerships and intrafirm R&D–marketing–production integration of manufacturing firms in emerging economies

Although cross-functional integration is important for research and development (R&D), research about implications of cross-functional integration has been rather sparse. In new product development (NPD), no study to date has examined intrafirm as well as interfirm integration of key functions such as intrafirm R&D–marketing–production together with interfirm integration of host R&D–partner R&D. Such marketing and operations interface contributes to a better understanding of how operational and marketing activities impact on competitiveness and firm performance. This study collected data from 202 electronics manufacturing firms operating in an emerging economy, mainland China and Hong Kong with international R&D partnerships. The findings indicate that a high level of R&D integration between firms improved NPD performance when cross-functional integration is based on existing rather than new product configurations and key technologies. Interestingly, in high distance situations, cross-functional integration in the production validation stage generated NPD success. The findings show that high environmental uncertainties lead to a high level of host and partner firms R&D integration. However, product newness has no significant effects on R&D integration in any of the NPD stages.     

The R&D and marketing cooperation across new product development stages: An empirical study of Taiwan's IT industry

This study extends the new product development (NPD) process research to a new environmental context (Taiwan's IT industry) and a new business type (original design manufacturing, ODM). Taiwan's IT industry has achieved a very outstanding performance during the last two decades. The island's experience is quite valuable for those emerging countries that are struggling to transform themselves from producing low-value goods to making high-technology products. After analyzing the data collected from 153 research and development (R&D) and marketing managers in Taiwanese IT firms, this study finds that the higher the perceived importance of R&D-marketing cooperation is, the higher the attained level of R&D-marketing cooperation will be. Consequently, a better NPD performance can be achieved. This study additionally reports that a firm that has adopted a Defender innovation strategy attains a lower level of R&D-marketing cooperation, and has a poorer NPD performance than those firms that adopted either Prospector or Analyzer innovation strategies. Finally, environmental uncertainty has no significant impacts on the perceived importance and the attained level of R&D-marketing cooperation.     

Involving purchasing in new product development

Purchasing is evolving into a strategic business activity and thus also a potential contributor to the successful development of new products. However, the literature on the involvement of purchasing in new product development (NPD) is sorely lacking. We conducted an exploratory study to investigate purchasing's involvement in NPD, the drivers of this involvement and the influence on new product success. We conducted telephone interviews with purchasing and NPD managers from 43 firms. The results show that firms differ in the extent to which they involve purchasing in NPD and that higher involvement has a positive effect on NPD performance. R&D managers can use the results to design a more effective purchasing–R&D interface and increase the success of NPD.     

Assessing R&D quality in rehabilitation technology development. The case of Roessingh Research and Development

This paper presents a framework for describing and evaluating the links between the design of the R&D function, its context and its performance. As a first step towards the development of a quality assurance system, this framework was used to assess R&D quality performance at a privatized R&D unit connected to a rehabilitation centre (Roessingh Research and Development). The results suggest that the framework is a useful instrument for managers to identify and understand the strengths and weaknesses of the R&D function, both with respect to its present, operational (quality) performance as well as its strategic flexibility, i.e. its readiness to adapt to, anticipate, or even create future (quality) performance requirements. The paper concludes with managerial implications with regard to the choice of an appropriate quality assurance system for Roessingh R&D, thus illustrating briefly how the framework can be used to apply ISO/BS quality standards (which are aimed at product development) in more fundamental R&D.     

Exploring New Product Development Project Review Practices*

Most organizations use new product development (NPD) processes that consist of activities and review points. Activities basically solve problems and gather and produce information about the viability of successfully completing the project. Interspersed between the development activities are review points where project information is reviewed and a decision is made to either go on to the next stage of the process, stop it prior to completion, or hold it until more information is gathered and a better decision can be made. The review points are for controlling risk, prioritizing projects, and allocating resources, and the review team typically is cross‐disciplinary, comprising senior managers from marketing, finance, research and development (R&D), or manufacturing. Over the past four decades, research has greatly advanced knowledge with respect to NPD activities; however, much less is known about review practices. For this reason, the present paper reports findings of a study on NPD project review practices from 425 Product Development & Management Association (PDMA) members. The focus is on three decision points in the NPD process common across organizations (i.e., initial screen, prior to development and testing, and prior to commercialization). In this paper, the number of (1) review points used, (2) review criteria, (3) decision makers on review committees and the proficiency with which various evaluation criteria are used are compared across incremental and radical projects and across functional areas (i.e., marketing, technical, financial). Furthermore, the associations between these NPD review practices and new product performance are examined. Selected results show that more review points are used for radical NPD projects than incremental ones, and this is related to a relatively lower rate of survival for radical projects. The findings also show that the number of criteria used to evaluate NPD projects increases as NPD projects progress and that the number of review team members grows over the stages, too. Surprisingly, the results reveal that more criteria are used to evaluate incremental NPD projects than radical ones. As expected, managers appear to more proficiently use evaluation criteria when making project continuation/termination decisions for incremental projects; they use these criteria less proficiently during the development of radical projects, precisely when proficiency is most critical. At each review point, technical criteria were found to be the most frequently used type for incremental projects, and financial criteria were the most commonly used type for radical ones. Importantly, only review proficiency is significantly associated with performance; the number of review points, review team size, and number of review criteria are not associated with new product performance. Furthermore, only the coefficient for proficiently using marketing criteria was significantly related to new product program performance; the proficiency of using financial and technical information has no association with performance. Finally, across the three focal review points of the NPD process in this study, only the coefficient for proficiency at the first review point, (i.e., the initial screen) is significantly greater than zero. The results are discussed with respect to research and managerial practice, and future research directions are offered.     

建筑行业绩效与竞争力:国内外研究进展

绩效与竞争力已发展成一个比较成熟的研究领域,一系列绩效与竞争力分析框架被应用到建筑行业,如波特钻石模型、3-P竞争力模型、产业组织理论、SWOT分析等产业分析理论.建筑行业研究人员也着力于设计出一些特定的分析模型来探讨行业内问题,如H分析框架、3C模型及PMS分析框架等.建立定量绩效与竞争力测评体系已成为各国建筑行业管理的发展趋势,如美国、英国、韩国、葡萄牙及新加坡等发达国家.而研究的不同点则在于测评指标体系、指标权重与测评方法等方面.     

The Effects of Knowledge Depth and Scope on the Relationship between R&D Alliances and New Product Development

In this study, we extend the new product development (NPD) literature that proposes that firms' knowledge depth, defined as the reuse of well understood technical knowledge, and scope, defined as the use of newly acquired technical knowledge, and new knowledge accessed from R&D alliances all positively impact NPD. Building on the knowledge‐based view of the firm, we posit that the impact of firms' R&D alliances is limited when their internal knowledge depth and scope are adequate for NPD needs. We suggest that although firms form R&D alliances to gain the right to access external knowledge of R&D alliance partners, they are not obligated to invest in resources to integrate external knowledge from R&D alliances. We propose that they wait to see if their internal knowledge depth and scope prove sufficient for NPD. If the external knowledge proves to be unnecessary, firms choose not to invest the resources required to integrate this knowledge with their internal knowledge. Alternatively, we suggest an increased impact of R&D alliances on NPD when firms are more limited in their internal knowledge depth and scope. We propose that when knowledge depth and scope prove insufficient, firms make the additional investments required to integrate external knowledge from R&D alliances with their internal knowledge stock. This reasoning is consistent with real options theory as it has been applied in alliance research, where strategic alliances are characterized as real options. We find support for our hypotheses using panel data of 738 firm year observations for 143 U.S. biopharmaceutical firms operating in 2007. Our study contributes to the NPD literature and suggests new directions for future research.     

The Effects of Environmental Turbulence on New Product Development Strategy Planning

Managers need guidance on how to cope with turbulent environments in order to improve corporate performance. Research on environmental turbulence has suggested that firms adopt a less centralized, more organic structure in dynamic, uncertain environments. Little work has been done specifically, however, on how environmental turbulence affects strategy planning for new product development (NPD). In this article, we specify a baseline model with firm innovativeness, market orientation and top management risk taking as antecedents to NPD speed and corporate strategic planning; these in turn are modeled as antecedents to NPD program (not project) performance. Two conceptualizations of the role of environmental turbulence are examined: (1) that market turbulence and technological turbulence are additional direct antecedents to NPD program performance; and (2) that the baseline model is moderated by turbulence (that is, that the strengths of the paths differ depending on levels of turbulence). A cross-sectional survey methodology including four diverse industries [automotive, electronics, publishing, and manufacturing/research and development (R&D) laboratories] was used to test the hypotheses. The latter conceptualization is supported. In particular, the paths from innovativeness to strategic planning and from risk taking to NPD speed are significantly greater in highly turbulent environments. A set of managerial recommendations and implications are provided. First, managers must recognize the possible improvements in new product performance by actively including NPD personnel in corporate strategic planning and also by involving corporate planners in NPD activities. Second, managers also should recognize that turbulent environments heighten the need to make risky investments, and sometimes, risky decisions; risk-taking decisions ought to be encouraged in such environments.     

Enhancing the Creative Performance of New Product Teams: An Organizational Configurational Approach

Innovation is crucial to managing ever‐increasing environmental complexity. Creativity is the first stage of the innovation process and is particularly relevant in modern new product development (NPD) projects. In response to a call for further empirical research on collective creative performance combining individual and team levels in a comprehensive framework, this paper offers useful evidence for the design of NPD teams to foster creative performance. The results suggest that different sets of individual traits and collective processes combine and interact, enabling a similar level of creative performance from different configurations of individual and team “ingredients.” There are no consistently good‐quality or poor‐quality NPD teams or processes. However, equifinal configurations—based on team composition, and interpersonal, coordination, control, and diversity management processes—can be effective in producing creative products. Through a large‐scale study of 119 teams of students involved in an NPD activity, this paper contributes by expanding creativity and NPD team design literature, providing the basis for a “first right” approach to real‐world, in‐company research. It first proposes and tests the adoption of the configurational equifinality approach in the NPD team design domain, introducing the concept of complementarities among different types of “team ingredients,” both at the individual and team level. Second, it introduces different multidimensional measures of team creative performance, relevant to generalizing and comparing the research results. Third, it offers several guidelines for designing real‐world NPD teams through the combination of diversity and interpersonal management, as well as coordination and control processes, which have not been studied to any great extent but are at times controversial in creativity literature.     

The Role of Departmental Thought Worlds in Shaping Escalation of Commitment in New Product Development Projects

The debate over whether and how thought worlds of different departments (especially marketing and research and development [R&D]) affect managers' decision-making behavior in new product development (NPD) is ongoing. A key challenge of these decisions is to deal with deteriorating NPD projects, which are often subject to escalation of commitment (EoC), with many firms wasting billions of dollars by throwing good money after bad NPD projects. However, understanding departmental thought worlds and their role for EoC in NPD could help firms stop this profusion. Thus, this research provides answers to the question of how thought worlds affect managers' tendency toward EoC in NPD decision-making—both in general and under certain project characteristics. To do so, we conducted four studies based on real-life scenarios with 460 highly experienced NPD managers from marketing and R&D, thus ensuring high validity and reliability. Our research is the first to explore the impact of thought worlds on EoC, thereby detecting that the importance of managers' thought worlds for shaping EoC varies with the NPD project's characteristics. Thus, depending on the specific project situation, different types of managers may be more or less capable of making proper NPD decisions. Moreover, results show that belief updating serves as a respective key mediator. Doing so enriches the theory by showing that managers' thought worlds can substantially influence a major mechanism (i.e., belief updating) of coping with cognitive dissonance. Finally, post hoc tests reveal departmental differences in EoC behavior between marketing and R&D that vary with a project's characteristics. These results imply that firms need to carefully consider who is in charge of making decisions on NPD project continuance in different project situations.     

Utilizing the balanced scorecard for R&D performance measurement

R&D programs are critical for many firms to achieve and sustain competitive advantage. Yet, measuring R&D performance over time can be quite complex due to inherent uncertainty. The paper responds to calls in the R&D literature to explore integrated performance measurement systems that capture financial and nonfinancial performance. We integrate the Stage-Gate approach to R&D management with the Balanced Scorecard to present a framework to show how firms can link resource commitments to these activities and the firm's strategic objectives. In this paper, we provide specific examples of how firms can apply this integrated performance measurement system to the R&D function.     

Whether to Integrate R&D and Marketing: The Effect of Firm Competence

Integration of research and development (R&D) with marketing remains a frequent topic in the new product development (NPD) literature, largely because it represents a critical antecedent of new product performance (NPP). Two divergent opinions about this integration exist, such that those who contend that firms should pursue high levels of integration in every case provoke criticisms from those who propose that various NPD processes require different levels of integration. This paper proposes that the two perspectives can be reconciled by taking into account the fact that R&D and marketing are integrated mainly to combine critical knowledge (technological and market) that otherwise would be separate to achieve market success. Following Danneels's approach, we investigate how the effect of R&D–marketing integration on performance change across four types of NPD processes: pure exploitation, pure exploration, technological competence exploitation, and market competence exploitation. Data derived from a deep study of 11 NPD projects by five firms, analyzed through qualitative methods, highlight the necessity to vary the level of integration according to the type of competence to be developed during the NPD process. Our analysis suggests two main conclusions. First, the effect of integration depends strictly on the type of competence that the firm uses to develop and launch a new product. Second, integration does not have a unique effect on performance, but it is necessary to distinguish between market performance (e.g., sales and market share) and process performance (e.g., meeting the planned budget and time to market). In some projects, the effect of integration on the two types of performance is diametrically opposite. In particular, we propose that (1) higher performance will be associated with lower integration in pure exploitative projects; (2) in projects that exploit existing market knowledge, higher market performance will be associated with a higher integration, although these projects tend to offer poor process performance regardless of integration level; (3) in projects that exploit technical knowledge, higher performance will be associated with higher integration; and (4) higher integration will be associated with higher market performance but poorer process performance in pure explorative projects.     

Design principles for the development of measurement systems for research and development processes

Based on a comprehensive literature review and the activities of numerous case study companies, it is argued in this paper that performance measurement in R&D is a fundamental aspect to quality in R&D and to overall business performance. However, it is apparent from the case companies that many companies still struggle with the issue of R&D performance measurement. Excuses for not measuring are easily found, but there are also empirical examples and literature available with suggestions how it can be done. In this article this literature is reviewed and placed within the context of general performance control and contingency theory. Furthermore, the main measurement system design parameters are discussed and some basic system requirements are described as well as several design principles that can be useful for those who accept the challenge of establishing a meaningful measurement system.     

Absorptive Capacity and Source‐Recipient Complementarity in Designing New Products: An Empirically Derived Framework*

Numerous publications are dedicated to absorptive capacity and new product development (NPD). Most are centered on the recipient team, and very few consider the effects of the source team knowledge characteristics on the knowledge absorption and the NPD performance. This paper analyzes the type of the external knowledge sourced from outside the organization and the process through which it is used by the recipient firm and the effect on NPD performance. This is done through a specific type of source team knowledge, the design, and through the NPD process in industries (clothing and construction) where it plays a key role. NPD cases were analyzed and clustered in three categories of design absorption processes. From these categories, a conceptual framework of the source‐recipient knowledge complementarity and its impact on the NPD performance is proposed. The main result is that the complementarity between the recipient and the source knowledge is a critical aspect of the absorption process and therefore of the NPD performance. From a managerial perspective, this research highlights the role of design in the NPD process and how the combination of design knowledge with prior knowledge (marketing or technological) is related to NPD performance.