The impact of electronic theses and dissertations: a study of the institutional repository of a university in South Africa

This paper investigates the academic and societal impacts of the engineering electronic theses and dissertations (ETDs) at the North-West University. The citation counts obtained through Google Scholar (GS) and the altmetrics data obtained from Dspace on the North-West University institutional repository are used to determine these impacts. Further, the conversion rates of ETDs from an educational output into an academic research output are determined using data from GS and the North-West University institutional repository. The results show that the 612 theses and dissertations attracted 931 citations which translates to 1.52 citations per thesis on average. A total of 41.2% theses and dissertations received at least one citation and the conversion rate into academic research output was 16.8%. The conversion rates in the last 3 years that are part of this research is 27.3% which suggests an improvement. An average of 323 PDF views were received per thesis. The citation and PDF views data show that the Engineering ETDs have commendable academic and societal impacts. Nonetheless, the academic impacts show scant correlation with the societal impacts although some positive trends are noticeable when looking at the origin of PDF views and citations by country.     

Global Bioregions: Knowledge Domains, Capabilities and Innovation System Networks

In this paper, regional capabilities and knowledge domain theses are proposed and global biotechnology dynamics are analysed. Detailed research on Europe's leading bioeconomy, the UK, is presented. Global network analysis is performed based on research into collaborations between 'star' scientists and their institutes in bioregions at a global scale. This is with regard to joint publication of bioscientific articles in US and EU Science Citation Index representative and leading cited journals. New evidence lies in identifying the hierarchical structure and main network axes in the global bioscientific research system. The results show that the strongest bioregions are in North America, particularly around Boston, San Diego and San Francisco. For collaboration, using this measure, the UK is revealed as a strong European research base, as is Sweden. New bioregions are found rising in Asia, but many 'global cities' fail to appear in the bioregional ranking.     

A citation-analysis of economic research institutes

The citation analysis of the research output of the German economic research institutes presented here is based on publications in peer-reviewed journals listed in the Social Science Citation Index for the 2000–2009 period. The novel feature of the paper is that a count data model quantifies the determinants of citation success and simulates their citation potential. Among the determinants of the number of cites the quality of the publication outlet exhibits a strong positive effect. The same effect has the number of the published pages, but journals with size limits also yield more cites. Field journals get less citations in comparison to general journals. Controlling for journal quality, the number of co-authors of a paper has no effect, but it is positive when co-authors are located outside the own institution. We find that the potential citations predicted by our best model lead to different rankings across the institutes than current citations indicating structural change.     

Using Google Scholar institutional level data to evaluate the quality of university research

In recent years, the extent of formal research evaluation, at all levels from the individual to the multiversity has increased dramatically. At the institutional level, there are world university rankings based on an ad hoc combination of different indicators. There are also national exercises, such as those in the UK and Australia that evaluate research outputs and environment through peer review panels. These are extremely costly and time consuming. This paper evaluates the possibility of using Google Scholar (GS) institutional level data to evaluate university research in a relatively automatic way. Several citation-based metrics are collected from GS for all 130 UK universities. These are used to evaluate performance and produce university rankings which are then compared with various rankings based on the 2014 UK Research Excellence Framework (REF). The rankings are shown to be credible and to avoid some of the obvious problems of the REF ranking, as well as being highly efficient and cost effective. We also investigate the possibility of normalizing the results for the university subject mix since science subjects generally produce significantly more citations than social science or humanities.     

The expansion of Google Scholar versus Web of Science: a longitudinal study

Web of Science (WoS) and Google Scholar (GS) are prominent citation services with distinct indexing mechanisms. Comprehensive knowledge about the growth patterns of these two citation services is lacking. We analyzed the development of citation counts in WoS and GS for two classic articles and 56 articles from diverse research fields, making a distinction between retroactive growth (i.e., the relative difference between citation counts up to mid-2005 measured in mid-2005 and citation counts up to mid-2005 measured in April 2013) and actual growth (i.e., the relative difference between citation counts up to mid-2005 measured in April 2013 and citation counts up to April 2013 measured in April 2013). One of the classic articles was used for a citation-by-citation analysis. Results showed that GS has substantially grown in a retroactive manner (median of 170 % across articles), especially for articles that initially had low citations counts in GS as compared to WoS. Retroactive growth of WoS was small, with a median of 2 % across articles. Actual growth percentages were moderately higher for GS than for WoS (medians of 54 vs. 41 %). The citation-by-citation analysis showed that the percentage of citations being unique in WoS was lower for more recent citations (6.8 % for citations from 1995 and later vs. 41 % for citations from before 1995), whereas the opposite was noted for GS (57 vs. 33 %). It is concluded that, since its inception, GS has shown substantial expansion, and that the majority of recent works indexed in WoS are now also retrievable via GS. A discussion is provided on quantity versus quality of citations, threats for WoS, weaknesses of GS, and implications for literature research and research evaluation.     

Patterns of annual citation of highly cited articles and the prediction of their citation ranking: A comparison across subjects

High citation is associated with research quality and consequently findings on highly cited articles are useful to increase understanding of the factors that produce high quality research. This study explores highly cited articles in six subjects, focusing on late citation and peak citation years. Longitudinal citation patterns were found to be highly varied and, on average, different from the remaining articles in each subject. For four of the six subjects, there is a correlation of over 0.42 between the percentage of early citations and total citation ranking but more highly ranked articles had a lower percentage of early citations. Surprisingly, for highly cited articles in all six subjects the prediction of citation ranking of from the sum of citations during their first six years was less accurate than prediction using the sum of the citations for only the fifth and sixth year.     

Citations versus limitations of citations: beyond Hirsch index

It appears popular, particularly among science administrators, to use citations and various citation measures for ranking scientists, as if such exercises would reflect the scientific potential of the persons considered. In recent time the Hirsch index h in particular has obtained visibility in this respect in view of its simplicity. We consider a possible extension of the concept of selective citations, which in fact is innate to the h index, and propose a simple generalization, indices H and Q, which to a degree supplement the information accompanying the evaluation of h. The H index keeps record of the “history” of citations and the quotient Q = H/h is a measure for the quality of a scientist based on the history of his/her citations.     

Profiling institutes: Identifying high research performance and social relevance in the social and behavioral sciences

This study focuses on the identification of high output research and high impact research in the social and behavioral sciences. A second objective is to monitor developments in research that is related to societal needs and problems. For each topic, we identify institutes and authors that have contributed a considerable number of SSCI articles and/or several (relatively) highly cited articles on a topic. To identify papers with a (relatively) high impact, the present study used two citation thresholds, each based upon a combination of a statistically determined minimum number of ‘external’ citations (by others than the authors of a paper), and a statistical comparison with world average citation levels. Topics were categorised according to social relevance. Detailed profiles of both large and small institutes are obtained, showing strengths and weaknesses in research performance that tend to be obscured by standard indicators focusing on ‘average’ research performance. Socially relevant topics tend to be concentrated in multidisciplinary clusters, whereas clusters dominated by one or two disciplines contain more basis research. The results provide a first insight in the extent to which institutes and authors address socially relevant topics. Use of the results by policy bodies seems dependent upon which organisational level of research they tend to address.     

Correlation between quality and quantity in scientific production: A case study of Iranian organizations from 1997 to 2006

In order to prevent the formation of a gap between the quality and quantity in Iranian scientific publications, this study makes an effort to analyze Iranian scientific publications indexed on the ISI Web of Science database using quantitative and qualitative scientometrics criteria over a ten year period. As a first step, all Iranian institutes were divided into three categories; universities, research institutes and other organizations. Then they were compared according to quantitative and qualitative criteria. Second, the correlation between the quality and quantity of the publications was measured. The research findings indicated that, according to qualitative criteria (citation, citation impact and percentage of cited documents) there are no meaningful differences among the three groups, while regarding quantitative criterion(number of papers), universities rank higher than the other two groups. The results also indicated that there is a positive and meaningful correlation among qualitative and quantitative criteria in the scholarly scientific publications conducted by Iranian organizations. In other words, in Iranian organizations the quality of publications increases as their quantity increases. The comparison of magnitude of correlation between these two criteria in the three categories reveals the fact that the correlation between number of papers and citations criterion in research institutes is stronger than the other two groups.     

Knowledge management research status in China from 2006 to 2010: based on analysis of the degree theses

Knowledge management has attracted an increasing number of researchers since the concept was born. Its research scope is expanding constantly and its research depth is strengthening. Also, in our country, there are experts and scholars in different fields carrying out researches into knowledge management theory and practice from their own point of view. In order to understand the present situation and the trend of knowledge management, this paper collected degree theses about knowledge management from 2006 to 2010 from Chinese Selected Doctoral Dissertations and Master’s Theses Full-Text Databases (CDMD) to analyze. A total of 173 Ph.D. theses and 1,243 Master theses were found by retrieving “knowledge management” with title or keywords. Statistical analysis data shows: the number of the knowledge management degree theses has decreased since 2008; disciplines of degree theses distribute widely and they mainly concentrate on management science, technology and education field; the number of the research institutions has increased, however, the main research institutions are still prominent relatively; knowledge management research hot spots based on the lexical frequency distribution are wide and deep; knowledge management research hot spots based on the analysis of the key words are clear.     

Citation analysis of Ph.D. theses with data from Scopus and Google Books

This study investigates the potential of citation analysis of Ph.D. theses to obtain valid and useful early career performance indicators at the level of university departments. For German theses from 1996 to 2018 the suitability of citation data from Scopus and Google Books is studied and found to be sufficient to obtain quantitative estimates of early career researchers’ performance at departmental level in terms of scientific recognition and use of their dissertations as reflected in citations. Scopus and Google Books citations complement each other and have little overlap. Individual theses’ citation counts are much higher for those awarded a dissertation award than others. Departmental level estimates of citation impact agree reasonably well with panel committee peer review ratings of early career researcher support.

     

Methodology for the evaluation of scientific journals: Aggregated Citations of Cited Articles

The criteria for the evaluation of scientific journals have changed from characteristics of its contents to citations of articles. Among many problems associated with citation-based evaluation methods are that it is applicable only to a limited number of journals, preferential selection of citable documents, differential values to citations, time duration for assessment, etc. The proposed index, Aggregated Citations of Cited Articles (ACCA), is calculated based on citations data, derived from only of cited articles, and therefore can be validated from standard database. While giving more importance to citations, the number of cited articles published in a journal also has some influence in the new index. The calculated values are consistent with time and can be used to back-track the status of a journal in its past and for continued evaluation. The new Index ensures neutrality, qualitative and quantitative hierarchy and consistency in the estimation of journal ranking.     

A weightage formula to rerank periodicals in the field of microbiology

Ranking lists of periodicals in different scientific disciplines are generally based on the total number of citations derived from a group of source journals with equal weightage given to all citations irrespective of chronology. As a result, in all these ranking lists, the position occupied by the post-war journals, may not be an accurate index of their demand or usefulness to research workers in the concerned field. This may happen because the quantum of published literature in the new journals is limited by the period for which they have been in existence, which is in many cases less than 20 years. Moreover, many of the post-war journals appearing in all these recent ranking lists have started publication even after 1960s, and their enumerated citations cover in fact a variable period of less than 20 years unlike in the case of pre-war journals. To avoid such discriminating against new journals which necessarily have fewer citation credits Sengupta suggested earlier an off-setting weightage formula which has been applied in this paper to rerank microbiology periodicals. A revised ranking list in the field of microbiology has been recommended in preference to the earlier one compiled by Sengupta. Results and findings, after application of the weightage formula, have been presented and discussed in this paper.     

Ranking Israel’s economists

One of the more important measures of a scholar’s research impact is the number of times that the scholar’s work is cited by other researchers as a source of knowledge. This paper conducts a first of its kind examination on Israel’s academic economists and economics departments, ranking them according to the number of citations on their work. It also provides a vista into one of the primary reasons given by junior Israeli economists for an unparalleled brain drain from the country—discrepancies between research impact and promotion. The type of examination carried out in this paper can now be easily replicated in other fields and in other countries utilizing freely available citations data and compilation software that have been made readily accessible in recent years.     

Information-based evaluation of the quality of doctoral theses

A method of information-based evaluation of the quality of doctoral theses has been worked out. It is based on a multidimensional classification system which includes: a list of attributes to characterize the given theses and their authors; singling out the most significant attributes; calculating a complex criterion showing the quality of a thesis on the basis of a series of significant attributes; ranging the theses according to this criterion. This method was used to evaluate 36 doctoral theses in chemistry according to 41 attributes considered. Four main attributes were singled out. Based on them a complex quality criterion which we termed the originality index was calculated. The values of the originality index of these theses differed by an order of magnitude. Two attributes affecting the index of originality were singled out: the place where the thesis was prepared, and the fact whether the author had any papers published in non-Academy journals (journals not published by the USSR Academy of Sciences).     

The matthew index—Concentration patterns and Matthew core journals

In this paper we extend our studies to the micro-structure of the Matthew effect for countries (MEC). The MEC allows the ranking of countries by their Matthew Index. The rank distribution of countries, observable only at a macro-level, has its roots in re-distribution processes of citations in every journal of the database. These re-distributed citations we call Matthew citations. Data for 44 countries and 2712 journals (based on theScience Citation Index) are analyzed. The strength of the contribution of the individual journals to the MEC (their number of Matthew citations) is skewly distributed. Due to this high concentration of the MEC we are able to define a new type of journal the Matthew core journal: 145 Matthew core journals account for 50% of the MEC. These journals carry a high potential of gaining a surplus of citations over what is expected and the risk of losing a high number of citations as well.     

Citation analysis and peer ranking of Australian social science journals

Citation analyses were performed for Australian social science journals to determine the differences between data drawn from Web of Science and Scopus. These data were compared with the tier rankings assigned by disciplinary groups to the journals for the purposes of a new research assessment model, Excellence in Research for Australia (ERA), due to be implemented in 2010. In addition, citation-based indicators including an extended journal impact factor, the h-index, and a modified journal diffusion factor, were calculated to assess whether subsequent analyses influence the ranking of journals. The findings suggest that the Scopus database provides higher number of citations for more of the journals. However, there appears to be very little association between the assigned tier ranking of journals and their rank derived from citations data. The implications for Australian social science researchers are discussed in relation to the use of citation analysis in the ERA.     

Are leaders really leading? Journals that are first in Web of Science subject categories in the context of their groups

This study focuses on journals that lead their Web of Science (WoS) subject category ranking when the usual 2-year window for the Journal Impact Factor (JIF2) is used as the ranking variable, and examines evidence that contradicts their top-ranked position in the context of their group. The source data were obtained from all 177 WoS subject categories in the Science Edition 2015 Journal Citation Reports (JCR). I compared journals in each WoS subject category with leaders in terms of JIF2, number of citable items and number of citations that contribute to the JIF2. Rankings were calculated with alternative metrics (for example, the Journal Impact Factor without self-citations and the eigenfactor), and the minimum reduction in the number of citations that would displace the top-ranked journal from its leading position was also calculated. In addition, the stability of rankings over time, the number of WoS subject categories in which journals are leaders, the publishers that own leading journals, and the percentages of research articles (as opposed to review articles) published in different journals were also studied. In general, leading journals are not necessarily the top-ranked in terms of citations received or the number of citable items they publish. In addition, most leaders maintained their position when other metrics were used instead the JIF2, although rankings based on the eigenfactor were at variance with this finding. The distribution of publishers was highly skewed, with a linear relationship between the cumulative number of publishers owning the top-ranked journal and the cumulative number of WoS subject categories. In only 85 subject categories (48%) the percentage of research articles (not reviews) in the number of citable items was greater than the mean percentage for the subject category. In 31 instances, leaders did not publish any research articles, but only reviews.     

Research,technology and development evaluation; developments in Japan

The reseach, technology and development (RTD) evaluation in terms of science and technology policy has come to be important in stimulating research activities and in continuously keeping the vitality and the higher quality of research in RTD institutions. There are two criteria on the RTD evaluation, i.e, in-house evaluation from the stand point of RTD management and independent macroscopic evaluation for the decision making of companies and/or policy making for science and technology policy.The most important point for RTD evaluation in the former criteria is in the mission itself. RTD in universities, public research institutes, and enterprises have different objectives and characteristics. Therefore, the mission and methodology of RTD evaluations should be different, by categorized type and objectives of research institutions, and be developed inhouse. Results of RTD evaluations should be fed back to researchers or engineers and disclosed principally if the mission was to stimulate knowledge creation through RTD activities.The in-house RTD evaluation can be classified in general into three categories: prior evaluation, mid-term review, and ex-post facto review. The methodologies to evaluate RTD in each phase of the RTD process are different, even among those institutes categorized into the same type such as national and regional research institutes. In this paper, two cases of RTD evaluation a) in Riken, which was founded in 1917 as a private research foundation and later reorganized as a semi-public research corporation of the Science and Technology-Agency, b) in regional public research institutes.RTD evaluation from the view point of policy assessment of governmental science and technology policy is discussed through analysis of data obtained by the survey of research activities in regional public research institutes. It can be concluded that developments and introduction of RTD evaluation as a new management system in these institutes is improving the research environment and advancing the quality of research. The differences of RTD evaluation between a Center of Excellence (COE) such as Riken and local technology centers, will be compared and the policy implication of RTD evaluation will also be discussed in terms of promotion of science and technology.     

Citation frequency: A biased measure of research impact significantly influenced by the geographical origin of research articles

Context. The use of citation frequency and impact factor as measures of research quality and journal prestige is being criticized. Citation frequency is augmented by self-citation and for most journals the majority of citations originate from a minority of papers. We hypothesized that citation frequency is also associated with the geographical origin of the research publication. Objective. We determined whether citations originate more frequently from institutes that are located in the same country as the authors of the cited publication than would be expected by chance. Design. We screened citations referring to 1200 cardiovascular publications in the 7 years following their publication. For the 1200 citation recipient publications we documented the country where the research originated (9 countries/regions) and the total number of received citations. For a selection of 8864 citation donor papers we registered the country/region where the citing paper originated. Results. Self-citation was common in cardiovascular journals (n = 1534, 17.8%). After exclusion of self-citation, however, the number of citations that originated from the same country as the author of the citation recipient was found to be on average 31.6% higher than would be expected by chance (p<0.01 for all countries/regions). In absolute numbers, nation oriented citation bias was most pronounced in the USA, the country with the largest research output (p<0.001). Conclusion. Citation frequency was significantly augmented by nation oriented citation bias. This nation oriented citation behaviour seems to mainly influence the cumulative citation number for papers originating from the countries with a larger research output.