A preliminary test of Google Scholar as a source for citation data: a longitudinal study of Nobel prize winners

Most governmental research assessment exercises do not use citation data for the Social Sciences and Humanities as Web of Science or Scopus coverage in these disciplines is considered to be insufficient. We therefore assess to what extent Google Scholar can be used as an alternative source of citation data. In order to provide a credible alternative, Google Scholar needs to be stable over time, display comprehensive coverage, and provide non-biased comparisons across disciplines. This article assesses these conditions through a longitudinal study of 20 Nobel Prize winners in Chemistry, Economics, Medicine and Physics. Our results indicate that Google Scholar displays considerable stability over time. However, coverage for disciplines that have traditionally been poorly represented in Google Scholar (Chemistry and Physics) is increasing rapidly. Google Scholar’s coverage is also comprehensive; all of the 800 most cited publications by our Nobelists can be located in Google Scholar, although in four cases there are some problems with the results. Finally, we argue that Google Scholar might provide a less biased comparison across disciplines than the Web of Science. The use of Google Scholar might therefore redress the traditionally disadvantaged position of the Social Sciences in citation analysis.     

Citations to the “Introduction to informetrics” indexed by WOS,Scopus and Google Scholar

Google Scholar and Scopus are recent rivals to Web of Science. In this paper we examined these three citation databases through the citations of the book “Introduction to informetrics” by Leo Egghe and Ronald Rousseau. Scopus citations are comparable to Web of Science citations when limiting the citation period to 1996 and onwards (the citation coverage of Scopus)—each database covered about 90% of the citations located by the other. Google Scholar missed about 30% of the citations covered by Scopus and Web of Science (90 citations), but another 108 citations located by Google Scholar were not covered either by Scopus or by Web of Science. Google Scholar performed considerably better than reported in previous studies, however Google Scholar is not very “user-friendly” as a bibliometric data collection tool at this point in time. Such “microscopic” analysis of the citing documents retrieved by each of the citation databases allows us a deeper understanding of the similarities and the differences between the databases.     

Web of Science with the Conference Proceedings Citation Indexes: the case of computer science

Google Scholar and Scopus are recent rivals to Web of Science. In this paper we examined these three citation databases through the citations of the book “Introduction to informetrics” by Leo Egghe and Ronald Rousseau. Scopus citations are comparable to Web of Science citations when limiting the citation period to 1996 and onwards (the citation coverage of Scopus)—each database covered about 90% of the citations located by the other. Google Scholar missed about 30% of the citations covered by Scopus and Web of Science (90 citations), but another 108 citations located by Google Scholar were not covered either by Scopus or by Web of Science. Google Scholar performed considerably better than reported in previous studies, however Google Scholar is not very “user-friendly” as a bibliometric data collection tool at this point in time. Such “microscopic” analysis of the citing documents retrieved by each of the citation databases allows us a deeper understanding of the similarities and the differences between the databases.     

Which h-index? — A comparison of WoS,Scopus and Google Scholar

This paper compares the h-indices of a list of highly-cited Israeli researchers based on citations counts retrieved from the Web of Science, Scopus and Google Scholar respectively. In several case the results obtained through Google Scholar are considerably different from the results based on the Web of Science and Scopus. Data cleansing is discussed extensively.     

Benchmarking Google Scholar with the New Zealand PBRF research assessment exercise

Google Scholar was used to generate citation counts to the web-based research output of New Zealand Universities. Total citations and hits from Google Scholar correlated with the research output as measured by the official New Zealand Performance-Based Research Fund (PBRF) exercise. The article discusses the use of Google Scholar as a cybermetric tool and methodology issues in obtaining citation counts for institutions. Google Scholar is compared with other tools that provide web citation data: Web of Science, SCOPUS, and the Wolverhampton Cybermetric Crawler.     

Robust h-index

Scientometrics - The h-index is the most used measurement of impact for researchers. Sites such as Web of Science, Google Scholar, Microsoft Academic, and Scopus leverage it to show and compare the...     

Can Microsoft Academic be used for citation analysis of preprint archives? The case of the Social Science Research Network

Preprint archives play an important scholarly communication role within some fields. The impact of archives and individual preprints are difficult to analyse because online repositories are not indexed by the Web of Science or Scopus. In response, this article assesses whether the new Microsoft Academic can be used for citation analysis of preprint archives, focusing on the Social Science Research Network (SSRN). Although Microsoft Academic seems to index SSRN comprehensively, it groups a small fraction of SSRN papers into an easily retrievable set that has variations in character over time, making any field normalisation or citation comparisons untrustworthy. A brief parallel analysis of arXiv suggests that similar results would occur for other online repositories. Systematic analyses of preprint archives are nevertheless possible with Microsoft Academic when complete lists of archive publications are available from other sources because of its promising coverage and citation results.     

Galton 2011 revisited: a bibliometric journey in the footprints of a universal genius

Commemorating the 100th death anniversary of Francis Galton, this paper is a bibliometric impact analysis of the works of this outstanding scientist and predecessor of scientometrics. Citation analysis was done in Web of Science, Scopus and Google Scholar (Publish or Perish) in order to retrieve the most cited books and journal articles. Additionally references were identified where Galton was rather mentioned than cited in order to analyze the phenomenon of obliteration by incorporation. Finally occurrence counts of Galton’s works in obituaries, Festschrift, the website Galton.org, major encyclopaedias and biographical indexes were compared to citation counts. As an outcome Galton’s works are increasingly cited or mentioned. Obliteration (use of eponyms) applies to one-third of Galton’s works and seems to be typical for fields like mathematics or statistics, whereas citations are more common in psychology. The most cited books and journal articles are also the most mentioned with remarkable correlation. Overall citation analysis and occurrence counting are complementary useful methods for the impact analysis of the works of “giants”.     

The expansion of Google Scholar versus Web of Science: a longitudinal study

Web of Science (WoS) and Google Scholar (GS) are prominent citation services with distinct indexing mechanisms. Comprehensive knowledge about the growth patterns of these two citation services is lacking. We analyzed the development of citation counts in WoS and GS for two classic articles and 56 articles from diverse research fields, making a distinction between retroactive growth (i.e., the relative difference between citation counts up to mid-2005 measured in mid-2005 and citation counts up to mid-2005 measured in April 2013) and actual growth (i.e., the relative difference between citation counts up to mid-2005 measured in April 2013 and citation counts up to April 2013 measured in April 2013). One of the classic articles was used for a citation-by-citation analysis. Results showed that GS has substantially grown in a retroactive manner (median of 170 % across articles), especially for articles that initially had low citations counts in GS as compared to WoS. Retroactive growth of WoS was small, with a median of 2 % across articles. Actual growth percentages were moderately higher for GS than for WoS (medians of 54 vs. 41 %). The citation-by-citation analysis showed that the percentage of citations being unique in WoS was lower for more recent citations (6.8 % for citations from 1995 and later vs. 41 % for citations from before 1995), whereas the opposite was noted for GS (57 vs. 33 %). It is concluded that, since its inception, GS has shown substantial expansion, and that the majority of recent works indexed in WoS are now also retrievable via GS. A discussion is provided on quantity versus quality of citations, threats for WoS, weaknesses of GS, and implications for literature research and research evaluation.     

Correlations between bibliometrics and peer evaluation for all disciplines: the evaluation of Brazilian scientists

This paper correlates the peer evaluations performed in late 2009 by the disciplinary committees of CNPq (a Brazilian funding agency) with some standard bibliometric measures for 55 scientific areas. We compared the decisions to increase, maintain or decrease a scientist’s research scholarship funded by CNPq. We analyzed these decisions for 2,663 Brazilian scientists and computed their correlations (Spearman rho) with 21 different measures, among them: total production, production in the last 5 years, production indexed in Web of Science and Scopus, total citations received (according to WOS, Scopus, and Google Scholar), h-index and m-quotient (according to the three citation services). The highest correlations for each area range from 0.95 to 0.29, although there are areas with no significantly positive correlation with any of the metrics.     

A new database of the references on international clinical practice guidelines: a facility for the evaluation of clinical research

Scientometrics - Although there are now several bibliographic databases of research publications, such as Google Scholar, Pubmed, Scopus, and the Web of Science (WoS), and some also include counts...     

Citation analysis of Ph.D. theses with data from Scopus and Google Books

This study investigates the potential of citation analysis of Ph.D. theses to obtain valid and useful early career performance indicators at the level of university departments. For German theses from 1996 to 2018 the suitability of citation data from Scopus and Google Books is studied and found to be sufficient to obtain quantitative estimates of early career researchers’ performance at departmental level in terms of scientific recognition and use of their dissertations as reflected in citations. Scopus and Google Books citations complement each other and have little overlap. Individual theses’ citation counts are much higher for those awarded a dissertation award than others. Departmental level estimates of citation impact agree reasonably well with panel committee peer review ratings of early career researcher support.

     

Use of Scopus and Google Scholar to measure social sciences production in four major Spanish universities

A large part of Social Sciences and the Humanities do not adapt to international proceedings used in English for scientific output on databases such as the Web of Science and Scopus. The aim of this paper is to show the different results obtained in scientific work by comparing Social Sciences researchers with those of other sciences in four Spanish universities. The first finding is that some Social Sciences researchers are somewhat internationalised. However, the majority of individuals who are prestigious in their local academic-scientific community do not even appear on the information sources mentioned above.     

Impact of open access on citation of scholarly publications in the field of civil engineering

The development of science is accompanied by growth of scholarly publications, primarily in the form of articles in peer-reviewed journals. Scientific work is often evaluated through the number of scientific publications in international journals and their citations. This article discusses the impact of open access (OA) on the number of citations for an institution from the field of civil engineering. We analyzed articles, published in 2007 in 14 international journals with impact factor, which are included in the Journal Citation Reports subject category “Civil Engineering”. The influence of open access on the number of citations was analyzed. The aim of our research was to determine if open access articles from the field of civil engineering receive more citations than non-open access articles. Based on the value of impact factor and ranking in quartiles, we also looked at the influence of the rank of journals on the number of citations, separately for OA and Non OA articles, in databases Web of Science (WOS), Scopus and Google Scholar. For 2,026 studied articles we found out that 22 % of them were published as OA articles. They received 29 % of all citations in the observed period. We can conclude by the significance level 5 % or less that in the databases WOS and Scopus the articles from top ranked journals (first quartile) achieved more citations than Non OA articles. This argument can be confirmed for some other journals from second quartile as well, while for the journals ranked into the third quartile it can’t be confirmed. This could be confirmed only partly for journals from the second quartile, and would not be confirmed for journals ranked into the third quartile. This shows that open access is not a sufficient condition for citation, but increases the number of citations for articles published in journals with high impact.     

Contributions of Turkish academicians supervising PhD dissertations and their universities to economics: an evaluation of the 1990–2011 period

In this study, doctoral dissertations conducted at the Turkish Universities for the period of 1990–2011 and the scientifically indexed publications of the professors supervising these dissertations were investigated. In the evaluations, the publications scores of the Google Scholar, Web of Science, and Scopus as well as the citations belonging to the publications in these indexes were included. During the relevant period, 617 professors supervised all the 1,906 doctoral dissertations in the field of economics. The first three universities with the highest number of doctoral dissertations were determined to be Istanbul University, Marmara University, and Dokuz Eylül University whereas the first three universities with the highest scientific indexes were Ihsan Do?ramaci Bilkent University, Middle East Technical University, and Bo?aziçi University. The academicians of the last three universities also outperformed those at the other universities regarding the scientific publications they produced and the citations they received. Overall, there is a low correlation between the dissertations conducted and scientific citations. Finally, academicians with publications and/or citations in Web of Science or Scopus were researched further via logistic regressions. The results indicate that there is positive and meaningful relation with PhD degrees earned abroad and number of publication in the Web of Science.     

Ranking of departments and researchers within a university using two different databases: Web of Science versus Scopus

In this work, we compare the difference in the number of citations compiled with Scopus as opposed to the Web of Science (WoS) with the aim of analysing the agreement among the citation rankings generated by these databases. For this, we analysed the area of Health Sciences of the University of Navarra (Spain), composed of a total of 50 departments and 864 researchers. The total number of published works reflected in the WoS during the period 1999–2005 was 2299. For each work, the number of citations in both databases was recorded. The results indicate that the works received 14.7% more citations in Scopus than in WoS. In the departments, the difference was greater in the clinical ones than in the basic ones. In the case of the rankings of citations, it was found that both databases generate similar results. The Spearman and Kendall-Tau coefficients were higher than 0.9. It was concluded that the difference in the number of citations found did not correspond to the difference of coverage of WoS and Scopus.     

The citation impact of social sciences and humanities upon patentable technology

This paper examines the citation impact of papers published in scientific-scholarly journals upon patentable technology, as reflected in examiner- or inventor-given references in granted patents. It analyses data created by SCImago Research Group, linking PATSTAT’s scientific non-patent references (SNPRs) to source documents indexed in Scopus. The frequency of patent citations to journal papers is calculated per discipline, year, institutional sector, journal subject category, and for “top” journals. PATSTAT/Scopus-based statistics are compared to those derived from Web of Science/USPTO linkage. A detailed assessment is presented of the technological impact of research publications in social sciences and humanities (SSH). Several subject fields perform well in terms of the number of citations from patents, especially Library and Information Science, Language and Linguistics, Education, and Law, but many of the most cited journals find themselves in the interface between SSH and biomedical or natural sciences. Analyses of the titles of citing patents and cited papers are presented that shed light upon the cognitive content of patent citations. It is proposed to develop more advanced indicators of citation impact of papers upon patents, and ways to combine citation counts with citation content and context analysis.

     

The impact of electronic theses and dissertations: a study of the institutional repository of a university in South Africa

This paper investigates the academic and societal impacts of the engineering electronic theses and dissertations (ETDs) at the North-West University. The citation counts obtained through Google Scholar (GS) and the altmetrics data obtained from Dspace on the North-West University institutional repository are used to determine these impacts. Further, the conversion rates of ETDs from an educational output into an academic research output are determined using data from GS and the North-West University institutional repository. The results show that the 612 theses and dissertations attracted 931 citations which translates to 1.52 citations per thesis on average. A total of 41.2% theses and dissertations received at least one citation and the conversion rate into academic research output was 16.8%. The conversion rates in the last 3 years that are part of this research is 27.3% which suggests an improvement. An average of 323 PDF views were received per thesis. The citation and PDF views data show that the Engineering ETDs have commendable academic and societal impacts. Nonetheless, the academic impacts show scant correlation with the societal impacts although some positive trends are noticeable when looking at the origin of PDF views and citations by country.     

Changing publication patterns in the Social Sciences and Humanities, 2000–2009

An analysis of the changing publication patterns in the Social Sciences and Humanities (SSH) in the period 2000?C2009 is presented on the basis of the VABB-SHW, a full coverage database of peer reviewed publication output in SSH developed for the region of Flanders, Belgium. Data collection took place as part of the Flemish performance-based funding system for university research. The development of the database is described and an overview of its contents presented. In terms of coverage of publications by the Web of Science we observe considerable differences across disciplines in the SSH. The overall growth rate in number of publications is over 62.1%, but varies across disciplines between 7.5 and 172.9%. Publication output grew faster in the Social Sciences than in the Humanities. A steady increase in the number and the proportion of publications in English is observed, going hand in hand with a decline in publishing in Dutch and other languages. However, no overall shift away from book publishing is observed. In the Humanities, the share of book publications even seems to be increasing. The study shows that additional full coverage regional databases are needed to be able to characterise publication output in the SSH.