Citation analysis using online databases: Feasibilities and shortcomings

Extensive citation analysis with the Science Citation Index (SCI) has become possible through expanded search capabilities introduced by STN International a few years ago. STN enhanced its retrieval language with some important features, originally developed for statistical analysis of patents. Most important are an expanded select command and several functions to list the search results. The publications to be evaluated may be selected either in the SCI, or in a number of other bibliographic databases offered by the host. With the help of these features, the basic methods to appropriately measure the impact of scientific activities are demonstrated. Furthermore, possible shortcomings as well as the risks when interpreting the results of such studies are discussed. This revised version was published online in June 2006 with corrections to the Cover Date.     

International and domestic co-publishing and their citation impact in different disciplines

This paper studies disciplinary differences in citation impacts of different types of co-publishing. The citation impacts of international, domestic inter-organizational and domestic intra-organizational co-publications, and single-authored publications, are compared. In particular, we examine the extent to which the number of authors explains the potential differences in citation impacts when compared to the influence of different types of international and domestic collaborations. The analysis is based on Finland’s publications in Thomson Reuters Web of Science database in 1990–2008. Finland is a small country, thus, it has fewer opportunities to find collaborators inside own country when compared to larger countries. Finland’s science policy has underlined internationalization and research collaboration as key means to increase the quality and impact of Finnish research. This study indicates that both international and domestic co-publishing have steadily increased during the past two decades in all disciplinary groups. International co-publications gain on average more citations than domestic co-publications. In natural sciences and engineering, co-authorship explains only a small proportion of variability in publications’ citation rates. When the effect of the number of authors is taken into account there are no big differences in citation impacts between international and domestic co-publications. However, international co-publications by ten authors or more gather significantly more citations than other publications. In humanities, the difference in citation impacts between co-authored publications in relation to single-authored publications is significant. However, international co-publications are not on average more highly cited in relation to domestic co-publications in humanities.     

Studying leadership and subdisciplinary structure of scientific disciplines

A new method for the analysis of leadership and subdisciplinary structure of a scientific discipline is discussed. The database consists of lists of participants in international scientific meetings. Disciplinary leaders are identified by means of their frequency of participation. The subdisciplinary structure is mapped by means of cluster analysis of meetings with respect to degree of similarily. The method possesses strengths not shared by citation analysis: in addition to scientists frequently cited in the literature for their contribution to cognitive research programs, it also identifies administrative discipline builders. The method may also represent better the cognitive interests of scientists.Paper presented at the Fourth International Conference on Bibliometrics, Informetrics and Scientometrics in Berlin (Germany), September 11–15, 1993.     

Science in Scandinavia: A Bibliometric Approach

The development of publication activity and citation impact in Scandinavian countries is studied for the 1980–1997 period. Besides the analysis of trends in publication and citation patterns and of national publication profiles, an attempt is made to find statistical evidences of the relation between international co-authorship and both research profile and citation impact in the Nordic countries. A coherent Scandinavian cluster has been found, and the Nordic countries have strong co-authorship links with highly developed countries in West Europe and North America. It was found that international co-authorship, in general, results in publications with higher citation rates than purely domestic papers. International collaboration has, however, not the same influence on publication profiles and citation impact of each analysed countries.     

To include or not: the role of review papers in citation-based analysis

Citation information helps researchers observe the evolution of the knowledge. In scientific publications, a review paper discusses a professional field and thus tends to have more citations than general papers do. This study investigated whether specific characteristics of review papers induce different results in citation-based analysis. From the Scopus database, we collected scientific publications in a specific research field, e-tourism, to examine the role of review papers in citation-based analysis. The dataset includes 1421 publications covering the period from the 1988–2015. Empirical statistics show that review papers’ specific citation patterns influence citation analysis. First, in the main path analysis, the result expresses review papers’ integrative role in linking papers from diverse perspectives toward a clear mainstream topic. Second, in a well-defined research context, review papers introduce bias in citation-based clustering analysis because the specific high citation pattern in review papers obfuscates the grouping process. When using citation information in analysis, scholars must consider the purpose of the study and treat review papers distinctly to avoid bias when using certain analysis methods and datasets.     

Citations to the “Introduction to informetrics” indexed by WOS,Scopus and Google Scholar

Google Scholar and Scopus are recent rivals to Web of Science. In this paper we examined these three citation databases through the citations of the book “Introduction to informetrics” by Leo Egghe and Ronald Rousseau. Scopus citations are comparable to Web of Science citations when limiting the citation period to 1996 and onwards (the citation coverage of Scopus)—each database covered about 90% of the citations located by the other. Google Scholar missed about 30% of the citations covered by Scopus and Web of Science (90 citations), but another 108 citations located by Google Scholar were not covered either by Scopus or by Web of Science. Google Scholar performed considerably better than reported in previous studies, however Google Scholar is not very “user-friendly” as a bibliometric data collection tool at this point in time. Such “microscopic” analysis of the citing documents retrieved by each of the citation databases allows us a deeper understanding of the similarities and the differences between the databases.     

Literature retrieval based on citation context

While the citation context of a reference may provide detailed and direct information about the nature of a citation, few studies have specifically addressed the role of this information in retrieving relevant documents from the literature primarily due to the lack of full text databases. In this paper, we design a retrieval system based on full texts in the PubMed Central database. We constructed two modules in the retrieval system. One is a reference retrieval module based on citation contexts. Another is a citation context retrieval module for searching the citation contexts of a specific paper. The results of comparisons show that the reference retrieval module performed better than Google Scholar and PubMed database in terms of finding proper references based on topic words extracted from citation context. It also performed very well on searching highly cited papers and classic papers. The citation context retrieval module visualizes the topics of citation contexts as tag clouds and classifies citation contexts based on cue words in citation contexts.     

Citation analysis of Ted Nelson’s works and his influence on hypertext concept

This study investigates Ted Nelson’s works and the influence of his hypertext concept through citation analysis, including citation counting, characteristics of citing articles on language, document type, citing year, discipline, and citation content. The selection of the Nelson’s works was based on searching Library Literature & Information Science, Library and Information Science Abstracts, Google and Yahoo search engines. The citation data were compiled from the database of Web of Science. The results of the study reveal that hypertext has directly great impact on information retrieval and world wide web; therefore, the concept has had profound influence on information, library and computer science disciplines. Moreover, the influence of Nelson’s works spreads to other disciplines variously, especially on education, literature, business and economics, engineering, sociology, psychology, etc. The citation context analysis of citing articles on information and library science reveals that (1) definition, orientation and general introduction of hypertext; (2) relation of Vannevar Bush and Ted Nelson in terms of hypertext; (3) Nelson’s Xanadu system and its component of hypertext; (4) the application of hypertext in information science and library science are four most citing purpose.     

Transnational citation, technological diversity and small world in global nanotechnology patenting

Despite the extensive studies conducted in the field of nanotechnology based on US patent data, the choice of a single database may impede a wider view of this technology frontier. Based on patent data from the Derwent Innovation Index database that covers the data of 41 major patent offices, we review the development of nanotechnology patenting from the dimensions of patenting authority and technological classification. We find that a small number of countries dominating the technology have similar technological diversity in terms of nanotechnology patents. After the discussion and summary of the citation modes and citation rate curve, we construct the patent citation networks at the patent document level and discuss the distinctive transnational citation patterns. We then use Search Path Count Method to extract the technological trajectory, where we find very high selectiveness. In the final section of this paper, we discover the small world phenomenon in the citation networks, which is widely investigated in undirected networks such as co-authorship networks, but rarely touched in citation networks due to the limitations of the presumptions. We propose the reachable path length and citation clustering in the revised small world model for acyclic directed networks and provide the realistic meaning of the new measures.     

Reconsidering the gold open access citation advantage postulate in a multidisciplinary context: an analysis of the subject categories in the Web of Science database 2009–2014

Since Lawrence in 2001 proposed the open access (OA) citation advantage, the potential benefit of OA in relation to citation impact has been discussed in depth. The methodology to test this postulate ranges from comparing the impact factors of OA journals versus traditional ones, to comparing citations of OA versus non-OA articles published in the same non-OA journals. However, conclusions are not entirely consistent among fields, and two possible explications have been suggested in those fields where a citation advantage has been observed for OA: the early view and the selection bias postulates. In this study, a longitudinal and multidisciplinary analysis of gold OA citation advantage is developed. All research articles in all journals for all subject categories in the multidisciplinary database Web of Science are considered. A total of 1,138,392 articles—60,566 (5.3%) OA articles and 1,077,826 (94.7%) non-OA articles—published in 2009 are analysed. The citation window considered goes from 2009 to 2014, and data are aggregated for the 249 disciplines (subject categories). At journal level, we also study the evolution of journal impact factors for OA and non-OA journals in those disciplines whose OA prevalence is higher (top 36 subject categories). As the main conclusion, there is no generalizable gold OA citation advantage, neither at article nor at journal level.     

Web of Science with the Conference Proceedings Citation Indexes: the case of computer science

Google Scholar and Scopus are recent rivals to Web of Science. In this paper we examined these three citation databases through the citations of the book “Introduction to informetrics” by Leo Egghe and Ronald Rousseau. Scopus citations are comparable to Web of Science citations when limiting the citation period to 1996 and onwards (the citation coverage of Scopus)—each database covered about 90% of the citations located by the other. Google Scholar missed about 30% of the citations covered by Scopus and Web of Science (90 citations), but another 108 citations located by Google Scholar were not covered either by Scopus or by Web of Science. Google Scholar performed considerably better than reported in previous studies, however Google Scholar is not very “user-friendly” as a bibliometric data collection tool at this point in time. Such “microscopic” analysis of the citing documents retrieved by each of the citation databases allows us a deeper understanding of the similarities and the differences between the databases.     

Publication types,citation rates and evaluation

In order to resolve questions frequently raised in the context of research evaluation about the citation rates of journal publications in relation to other types of publications, the total research output of substantial institutions or systems has to be brought under bibliographic control. That precondition has rarely been met: there are few published studies of the total range of publications of major research institutions, including books, book chapters, technical reports and published conference proceedings. The Research Evaluation and Policy Project (REPP) at the Australian National University (ANU) has established a database covering all the publications from the Institute of Advanced Studies (IAS), a fulltime research institution at the ANU, and has examined in detail citations in the journal literature accruing to all types of publications. The database contains a significant number of publications, nearly 30 000 items, and covers the sciences and the social sciences and humanities. This data enables us to examine whether the citation record of research publications appearing in journals indexed by the Institute for Scientific Information (ISI) is a useable surrogate for the citation record within ISI journals of all model of publication. We contend that, if certain preconditions are met, the choice of citation rate is not critical.     

The journal download immediacy index (DII): experiences using a Chinese full-text database

Relationships between the journal download immediacy index (DII) and some citation indicators are studied. The Chinese full-text database CNKI is used for data collection. Results suggest that the DII can be considered as an independent indicator, but that it also has predictive value for other indicators, such as a journal’s h-index. In case a journal cannot yet have an impact factor—because its citation history within the database is too short—the DII can be used for a preliminary evaluation. The article provides results related to the CNKI database as a whole and additionally, some detailed information about agricultural and forestry journals.     

Assessing the effect of the United States’ “citation advantage” on other countries’ scientific impact as measured in the Web of Science (WoS) database

Eugene Garfield, always insisted that citation analysis “can be used wisely or abused” and that it is “up to the scientific community to prevent abuse of the SCI by devoting the necessary attention to its proper and judicious exploitation” (Garfield in Nat 227:669–671, 1970). Dedicated to his memory, this paper aim to assess the significance of a parameter that is seldom taken into account in evaluation studies: the existence of a USA comparative citation (visibility) advantage built in the database and thus affecting countries that collaborate more with the USA than with other countries. We analyze how this USA citation advantage affects the measure of the scientific impact (usually measured through citations received) of major countries. The main conclusion coming out of this study is that, given the strong presence of the USA in the WoS database, the comparative rankings tend, by construction, to give a citation advantage to countries having the closest relation to that country.     

Analyzing the citation characteristics of books: edited books,book series and publisher types in the book citation index

This paper presents a first approach to analyzing the factors that determine the citation characteristics of books. For this we use the Thomson Reuters’ book citation index, a novel multidisciplinary database launched in 2011 which offers bibliometric data on books. We analyze three possible factors which are considered to affect the citation impact of books: the presence of editors, the inclusion in series and the type of publisher. Also, we focus on highly cited books to see if these factors may affect them as well. We considered as highly cited books, those in the top 5 % of those most highly cited in the database. We define these three aspects and present results for four major scientific areas in order to identify differences by area (science, engineering and technology, social sciences and arts and humanities). Finally, we report differences for edited books and publisher type, however books included in series showed higher impact in two areas.     

Mapping High-Temperature Superconductors—A Scientometric Approach

This study has been carried out to analyze the research field of high-temperature superconductivity and to demonstrate the potential of modern databases and search systems for generating meta-information. The alkaline earth (A2) rare earth (RE) cuprate high-temperature superconductors as a typical inorganic compound family and the corresponding literature were analyzed by scientometric methods. The time dependent overall number of articles and patents and of the publications related to specific compound subsets and subject categories are given. The data reveal a significant decrease of basic research activity in this research field. The A2 RE cuprate species covered by the CAS compound file were analyzed with respect to the occurrence of specific elements in order to visualize known and unknown substances and to identify characteristic patterns. The quaternary and quinternary cuprates were selected and the number of compound species as a function of specific combinations of A2 and RE elements is given. The Cu/O and RE/A2 ratios of the quaternary cuprate species as a function of A2 and RE atoms are shown. In addition, the research landscape of the MgB₂ related publications was established using STN AnaVist, an analysis tool recently developed by STN International.     

Influence of publication languages on the citation rate of scientific articles: A case study of East German journals

In order to quantify the influence of publication languages on the rate of citation of scientific articles, such East German journals from the Science Citation Index database were selected which publish relevant shares of contributions in several languages, especially in English and German. For a fixed period of time (1988) the selective citation impact of both English- and German-language articles was calculated. The results of our investigation reveal a non-uniform picture: In some cases English-language papers exhibit a significantly higher citations-per-paper average than German-language articles, but in a few cases German-language publications achieve a higher mean citation rate. For the half of selected journals there does not exist a statistically significant difference of citation frequencies of publications in both languages. Possible causes of these phenomena (editorial practice of journals, native countries of authors) are considered.     

The effect of database dirty data on h-index calculation

As all databases, the bibliometric ones (e.g. Scopus, Web of Knowledge and Google Scholar) are not exempt from errors, such as missing or wrong records, which may obviously affect publication/citation statistics and—more in general—the resulting bibliometric indicators. This paper tries to answer to the question “What is the effect of database uncertainty on the evaluation of the h-index?”, breaking the paradigm of deterministic database analysis and treating responses to database queries as random variables. Precisely an informetric model of the h-index is used to quantify the variability of this indicator with respect to the variability stemming from errors in database records. Some preliminary results are presented and discussed.     

Double effort = Double impact? A critical view at international co-authorship in chemistry

An attempt is made to find statistical evidences of the relation between international co-authorship and citation impact. It was found that international co-authorship, in average, results inpublications with higher citation rates than purely domestic papers. No correlation has beenfound, however, between the strength of co-authorship links and the relative citation eminence ofthe resulting publications. International co-authorship links in chemistry, as represented by thewell-known Salton's measure, displayed a characteristic pattern reflecting geopolitical, historical,linguistic, etc. relations among countries. A new indicator, representing also the asymmetry ofco-authorship links was used to reveal main "attractive" and "repulsive" centres of co-operation.