Bibliometric performance measures

Three different types of bibliometrics — literature bibliometrics, patent bibliometrics, and linkage bibliometric can all be used to address various government performance and results questions. Applications of these three bibliometric types will be described within the framework of Weinberg's internal and external criteria, whether the work being done is good science, efficiently and effectively done, and whether it is important science from a technological viewpoint. Within all bibliometrics the fundamental assumption is that the frequency with which a set of papers or patents is cited is a measure of the impact or influence of the set of papers. The literature bibliometric indicators are counts of publications and citations received in the scientific literature and various derived indicators including such phenomena as cross-sectoral citation, coauthorship and concentration within influential journals. One basic observation of literature bibliometrics, which carries over to patent bibliometrics, is that of highly skewed distributions — with a relatively small number of high-impact patents and papers, and large numbers of patents and papers of minimal impact. The key measure is whether an agency is producing or supporting highly cited papers and patents. The final set of data are in the area of linkage bibliometrics, looking at citations from patents to scientific papers. These are particularly relevant to the external criteria, in that it is quite obvious that institutions and supporting agencies whose papers are highly cited in patents are making measurable contributions to a nation's technological progress.     

The citation impact of social sciences and humanities upon patentable technology

This paper examines the citation impact of papers published in scientific-scholarly journals upon patentable technology, as reflected in examiner- or inventor-given references in granted patents. It analyses data created by SCImago Research Group, linking PATSTAT’s scientific non-patent references (SNPRs) to source documents indexed in Scopus. The frequency of patent citations to journal papers is calculated per discipline, year, institutional sector, journal subject category, and for “top” journals. PATSTAT/Scopus-based statistics are compared to those derived from Web of Science/USPTO linkage. A detailed assessment is presented of the technological impact of research publications in social sciences and humanities (SSH). Several subject fields perform well in terms of the number of citations from patents, especially Library and Information Science, Language and Linguistics, Education, and Law, but many of the most cited journals find themselves in the interface between SSH and biomedical or natural sciences. Analyses of the titles of citing patents and cited papers are presented that shed light upon the cognitive content of patent citations. It is proposed to develop more advanced indicators of citation impact of papers upon patents, and ways to combine citation counts with citation content and context analysis.

     

Can applied science be ‘good science’? Exploring the relationship between patent citations and citation impact in nanoscience

There is a rich literature on how science and technology are related to each other. Patent citation analysis is amongst the most frequently used to tool to track the strengths of links. In this paper we explore the relationship between patent citations and citation impact in nanoscience. Our observations indicate that patent-cited papers perform better in terms of standard bibliometric indicators than comparable publications that are not linked to technology in this way. More specifically, we found that articles cited in patents are more likely to be cited also by other papers. The share of highly cited papers is the most striking result. Instead of the average of 4% of all papers, 13.8% of the papers cited once or twice in patents fall into this category and even 23.5% of the papers more frequently cited in patents receive citation rates far above the standard. Our analyses further demonstrate the presence and the relevance of bandwagon effects driving the development of science and technology.     

Significant difference of dependence upon scientific knowledge among different technologies

Summary The authors have constructed an original database of the full text of the Japanese Patent Gazette published since 1994. The database includes not only the front page but also the body text of more than 880,000 granted Japanese patents. By reading the full texts of all 1,500 patent samples, we found that some inventors cite many academic papers in addition to earlier patents in the body texts of their Japanese patents. Using manually extracted academic paper citations and patent citations as “right” answers, we fine-tuned a search algorithm that automatically retrieves cited scientific papers and patents from the entire texts of all the Japanese patents in the database. An academic paper citation in a patent text indicates that the inventor used scientific knowledge in the cited paper when he/she invented the idea codified in the citing patent. The degree of science linkage, as measured by the number of research papers cited in patent documents, is particularly strong in biotechnology. Among other types of technology, those related to photographic-sensitized material, cryptography, optical computing, and speech recognition also show strong science linkage. This suggests that the degree of dependence on scientific knowledge differs from technology to technology and therefore, different ways of university-industry collaboration are necessary for different technology fields.     

Exploring the quality of environmental technology in Europe: evidence from patent citations

In this paper we carry out an empirical analysis to address some questions concerning the production and quality of technology in environmental sectors. The methodology involves patents as a measure of the generation of new knowledge, and patent citations as a proxy for the quality of a technological invention. The sample contains more than 12,000 environmental European patents from firms and government institutions from 1998 to 2004. From our econometric analysis, we found that environmental patents applied by individual inventors present on average less quality that those applied by institutional inventors. The size of family patent is relevant to explain forward patent citation. Furthermore, patents coming from abroad (out of Europe), in particular with US and Japan priority, are more cited on average than local patents (with European priority). Lastly, the specialization in environmental fields of a patent plays a negative role in determining the frequency of forward citation.     

Publication activity, citation impact and bi-directional links between publications and patents in biotechnology

The study focuses on publication activity, citation impact and citation links between publications and patents in biotechnology. The European Union (EU), US, Japan and China are the most important global players. However, the landscape is changing since the EU and the US are losing ground because of challenges from a group of emerging economies. National profiles differ between the two groups of main players and upcoming countries; the focus on red biotechnology in the US and Europe is contrasted by propensity for white and green technology in Asia. Furthermore, the subject profile of biotechnology papers citing patents and cited by patents as well as the relationship between patent citations and citation impact in scientific literature is explored. Papers that cite patents tend to reflect propensity towards white biotechnology while patent-cited publications have a higher relative share in red biotechnology. No significant difference concerning the citation impact of publications ??citing patents?? and ??not citing patents?? can be found. This is contrasted by the observation that patent-cited papers perform distinctly better in terms of standard bibliometric indicators than comparable publications that are not linked to technology in this direction.     

Technological Relevance of Science: An Assessment of Citation Linkages between Patents and Research Papers

Patent citations to the research literature offer a way for identifying and comparing contributions of scientific and technical knowledge to technological development. This case study applies this approach through a series of analyses of citations to Dutch research papers listed on Dutch-invented and foreign patents granted in the US during the years 1987–1996.First, we examined the general validity and utility of these data as input for quantitative analyses of science-technology interactions. The findings provide new empirical evidence in support of the general view that these citations reflect genuine links between science and technology. The results of the various analyses reveal several important features of industrially relevant Dutch science: (1) the international scientific impact of research papers that are also highly cited by patents, (2) the marked rise in citations to Dutch papers on foreign-invented patents; (3) the large share of author-inventor self-citations in Dutch-invented patents; (4) the growing relevance of the life sciences, (5) an increase in the importance of scientific co-operation. We also find significant differences between industrial sectors as well as major contributions of large science-based multinational enterprises, such as Philips, in domestic science-technology linkages.The paper concludes by discussing general benefits and limitations of this bibliometric approach for macro-level analysis of science bases in advanced industrialised countries like the Netherlands.     

Validation study: Patent citations as indicators of science and foreign dependence

The purpose of this study was to test whether the distribution of citations from issued U.S. patents could be used to measure the science dependence and the foreign dependence of patented technologies. The citations considered were front page references from U.S. patents citing to U.S. and foreign patents, to research papers and to other publications. Rankings based on the number of citations per patent to the scientific literature were compared to peer rankings of the science dependence of the technologies. Rankings based on the number of citations to foreign origin material, including foreign origin U.S. patents, foreign patents and foreign priority statements, were compared to peer rankings of the foreign dependence of the technological areas.For the analysis a total of 24 technologies were chosen. Twelve of these were judged in advance to be science dependent and twelve were judged in advance to be foreign dependent. A peer group of 19 high level R&D managers was asked to rank all 24 technologies in terms of both their science and their foreign dependence. The bibliometric rankings of the technologies, based on their citations, were then compared with the peer rankings of the technologies.Overall, a high degree of agreement was found between the experts' opinion as to the science and foreign dependence of the areas and the corresponding bibliometric rankings. For example, the eight technologies judged most science dependent by experts averaged 0.92 cites per patent to scientific journal papers, while the eight technologies judged least science dependent had only 0.05 references per patent to journal papers. Similarly, large and statistically significant differences were found in the number of cites to foreign origin material for the eight technologies judged most foreign dependent by the experts when compared with the eight technologies judged least foreign dependent by the experts. These findings provide support for the hypothesis that patent citation data can be used in technological indicators development, and in technological policy analysis. They imply that citation-based location and analysis of science and foreign dependent technologies is a valid research tool when applied to the U.S. patent system.     

Prediction of emerging technologies based on analysis of the US patent citation network

The network of patents connected by citations is an evolving graph, which provides a representation of the innovation process. A patent citing another implies that the cited patent reflects a piece of previously existing knowledge that the citing patent builds upon. A methodology presented here (1) identifies actual clusters of patents: i.e., technological branches, and (2) gives predictions about the temporal changes of the structure of the clusters. A predictor, called the citation vector, is defined for characterizing technological development to show how a patent cited by other patents belongs to various industrial fields. The clustering technique adopted is able to detect the new emerging recombinations, and predicts emerging new technology clusters. The predictive ability of our new method is illustrated on the example of USPTO subcategory 11, Agriculture, Food, Textiles. A cluster of patents is determined based on citation data up to 1991, which shows significant overlap of the class 442 formed at the beginning of 1997. These new tools of predictive analytics could support policy decision making processes in science and technology, and help formulate recommendations for action.     

What makes the first forward citation of a patent occur earlier?

Identifying the economic value of a patent is crucial to technology management. It also facilitates the commercialization and transactions of patents. As a proxy of patent value, forward citation counting is widely used, but it takes a long time for a patent to be sufficiently cited. In this context, we suggest the first citation lag, namely the time taken until the first citation, as a proxy of patent value, as it is positively correlated with the patent value and considers the fact that important patents with a high economic value tend to receive many citations in a short space of time. We explore the influential patent attributes related to the first citation lag to build a model to predict the patent value. By using the Cox proportional hazard model on green inventory patent data, we find that patents with a shorter technology cycle time and a larger number of IPC four-digit classes, claims, patent family, and backward citations are associated with the shorter first citation lag, while a patent’s science linkage has an inverse U-shaped relationship with the first citation lag. Further, patents having an unconventional technological core have a longer first citation lag and among patents having an unconventional technological core, the presence of a novel element makes the first citation earlier. Our study is expected to help patent evaluation in the early stage of technology transfer.     

Analyzing the value of technology based on the differences of patent citations between applicants and examiners

Patent applicants and examiners do not always have the same point of view when conducting a prior-art search. Although several studies have suggested differences between citations by applicants and examiners, the data and range of empirical studies are too incomplete to generalize the characteristics of relationships between citation types and the value of a technology or invention. To overcome this limitation, it is crucial to compare citations by applicants and by examiners in depth, with diverse perspectives and data, to determine the value of patent information for technological innovation. Thus, this paper suggests that the differences in the composition of technical information and patent quality in patent-level investigations as well as the locus of the knowledge source and knowledge recentness in knowledge-level investigations according to patent citation type (by applicants and examiners) reflect Pavitt’s perspective on the nature, impact, and source of technological innovation. We found that the quality of patents cited by applicants is higher than that of those by examiners in four industries, excluding a supplier-dominated industry. The citation types are related to the locus of the knowledge source in four industries, excluding the supplier-dominated industry. In particular, the patents cited by examiners tended to be more recently issued in all sectoral fields. This research contributes to confirming the technological value of patents based on the citation behaviors of applicants and examiners through empirical analysis. The results can be utilized to investigate signals or noise in technological innovation and improve processes or systems of patent examination. In addition, it can help applicants conduct more thorough prior-art searches by comprehending the examiner’s perspective toward citations to increase the probability of patent registration.     

Analysis of the relationship between citation frequency of patents and diversity of their backward citations for Japanese patents

Conventional patent citation analyses have focused mainly on the presence of citation relationships, the number of patents cited by the subject patent, and the number of times the subject patent is cited by others (i.e., the numbers of backward and forward citations of the subject patent). However, most of them have not focused on patent classifications. Assuming that a patent based on a variety of technological bases tends to be an important patent that is cited more often, this study examines and clarifies the relationship between the diversity of classifications assigned to backward citations and the number of forward citations for Japanese patents. The results show notable differences in the number of classifications assigned to backward citations between the often cited and less frequently cited groups. It is considered that the diversity of backward citations can be utilized in the evaluation criteria for grouping that roughly identifies the often cited patents or eliminates a large part of less frequently cited patents.     

Indicators for measuring the impact of scientific citations in patents

Scientific publications cited in patents are fundamental elements to assess the transfer of science to technology. Numerous studies evaluate the impact of references in patents and scientific publications and various measurement methods and indicators are proposed. This article reviewed the existing literature on the indicators used to date to determine their suitability and effectiveness to evaluate the impact of patent citations. For this purpose, we analyzed the characteristics of the studies examined and proposed a qualitative classification of indicators from both a technological (patents) and scientific-academic (scientific articles) perspective. Among the results we find that the use of scientometric indicators is primarily focused on analyzing their relevance for patents through the inclusion of scientific citations. Conversely, the same is not true from the academic point of view where gaps still persist in terms of what the impact is when scientific articles are cited in patents. Among the key conclusions is the diversity of proposals to measure scientific citations in patents, though these are conditioned on bibliographic standardization and metadata management of the patents themselves, making it possible to quantify aspects similar to the immediacy index, impact index, or h-index for authors/inventors, albeit from a technological dimension.     

Knowledge-transfer analysis based on co-citation clustering

Based on co-citation cluster analysis, we propose a knowledge-transfer analysis model for any technology field. In this model, patent data with backward citations to non-patent literature and forward citations by later patents would be analyzed. Co-citation clustering of the cited articles defines scientific knowledge sources, while that of the patents themselves defines technology fronts. According to the citation between the article and patent clusters, the landscape of knowledge-transfer including route and strength between scientific knowledge sources and technology fronts can be mapped out. The model has been applied to the field of transgenic rice. As a result of the analysis, ten scientific knowledge sources and eight technology fronts have emerged, and reasonable links between them have been established, which clearly show how knowledge has been transferred in this field.     

Large firms and the science-technology interface Patents,patent citations,and scientific output of multinational corporations in thin films

Firms operating in science-based technological fields reflect some of the complexities of the science-technology interaction. The present study attempts to investigate these interactions by analyzing patent citations, publication and patent outputs of multinational corporations (MNCs) in 'thin film' technology. In particular we explore different characteristics of knowledge production and knowledge utilization of these firms. The results indicate no correlation between intensity of research activity and patents produced by the MNCs. The relationship between scientific and technological knowledge generation as well as the linkage between science and technology appear to be firm-specific rather than dependent on a technological or industrial sector. The dispersion of journal sources for the majority of patent citations of scientific literature as well as for the majority of scientific outputs is narrow. Basic journals play an important role in patent citation as well as in addressing research of MNCs in thin-film technology.     

The growing number of patent citations to scientific papers: Changes in the world,nations, and fields

This study analyzes USPTO patents in the period 1998–2017. The number of science-related patents has increased twice as fast as the number of patents and scientific publications, and the number of cited papers per patent has almost doubled. These results vary substantially from one scientific and technological field to another. The proportion of the research papers cited by a patent has doubled. It refers to papers that are mostly published by the countries that have developed both scientific and technological capability and, surprisingly, are mainly used by inventors abroad. However, a weak relationship between the number of citations received from patents and papers reveals that the assessment of research performance needs some changes as the percentage of papers related to the innovations has grown over time.     

Citation rates to technologically important patents

The purpose of this study was to determine whether the average number of citations received by issued U.S. patents from subsequently issued U.S. patents is higher for patents associated with important technological advances than for a group of randomly selected patents. Analysis of examiners' citations to 100 selected patents showed that these selected patents, which underlay technically important products, were more than twice as frequently cited (significance level of 0.0001) as a randomly selected set of 102 control patents. This finding provides strong evidence for the hypothesis that patent citation data can be used in technological indicators development, and in technological policy analysis, since it implies that the location and analysis of groups of highly cited patents can provide a valid indicator of patent areas of technical importance.     

Science cited in patents: A geographic "flow" analysis of bibliographic citation patterns in patents

The interplay and cross-fertilization between science and technology, but also the specific role of science for technological development, have received ample attention in both the research and the policy communities. It is in this context that the concepts of absorptive capacity and knowledge spillovers play an important role. We operationalize the science-technology link by quantifying and modeling bibliographic references to the scientific literature as they occur in patents. This approach allows exploring the associative patterns between science creation (as emerging from the scientific literature) and technology development (as emerging from the patent literature). In the current paper, we focus on an analysis of the geographic distribution of the science citation patterns in patents, singling out two fields of (different) technological development, namely biotechnology and information technology. In both fields, the science citation flows from the European, Japanese and US science bases into USPTO and EPO-patents are explored and modeled. Intensive geographic citation flows between the regions are identified, pointing (amongst others) to the strength of both the US and the European science bases as sources for technological activity and creativity around the world.     

Patent Citations in a Novel Field of Technology — What Can They Tell about Interactions between Emerging Communities of Science and Technology?

This paper aims to contribute to a better understanding of patent citation analysis in general and its application to novel fields of science and technology in particular. It introduces into the subject-matter by discussing an empirical problem, the relationship of nano-publications and nano-patents as representations of nano-science and nano-technology. Drawing on a variety of sources, different interpretations of patent citations are presented. Then, the nature of patent citations is further investigated by comparing them to citations in the scientific literature. After characterizing the citation linkage as indicators of reciprocal relationships between science and technology, patent citations in nano-science and technology are analyzed in terms of interfield and organizational knowledge-flows.     

Multiple regression analysis of a patent’s citation frequency and quantitative characteristics: the case of Japanese patents

Although many studies have been conducted to clarify the factors that affect the citation frequency of “academic papers,” there are few studies where the citation frequency of “patents” has been predicted on the basis of statistical analysis, such as regression analysis. Assuming that a patent based on a variety of technological bases tends to be an important patent that is cited more often, this study examines the influence of the number of cited patents’ classifications and compares it with other factors, such as the numbers of inventors, classifications, pages, and claims. Multiple linear, logistic, and zero-inflated negative binomial regression analyses using these factors are performed. Significant positive correlations between the number of classifications of cited patents and the citation frequency are observed for all the models. Moreover, the multiple regression analyses demonstrate that the number of classifications of cited patents contributes more to the regression than do other factors. This implies that, if confounding between factors is taken into account, it is the diversity of classifications assigned to backward citations that more largely influences the number of forward citations.