Publication counting methods for a national research evaluation exercise

In this paper, we investigate the effects of using four methods of publication counting (complete, whole, fractional, square root fractional) and limiting the number of publications (at researcher and institution levels) on the results of a national research evaluation exercise across fields using Polish data. We use bibliographic information on 0.58 million publications from the 2013–2016 period. Our analysis reveals that the largest effects are in those fields within which a variety publication and cooperation patterns can be observed (e.g. in Physical sciences or History and archeology). We argue that selecting the publication counting method for national evaluation purposes needs to take into account the current situation in the given country in terms of the excellence of research outcomes, level of internal, external and international collaboration, and publication patterns in the various fields of sciences. Our findings show that the social sciences and humanities are not significantly influenced by the different publication counting methods and limiting the number of publications included in the evaluation, as publication patterns in these fields are quite different from those observed in the so-called hard sciences. When discussing the goals of any national research evaluation system, we should be aware that the ways of achieving these goals are closely related to the publication counting method, which can serve as incentives for certain publication practices.     

How to consider fractional counting and field normalization in the statistical modeling of bibliometric data: A multilevel Poisson regression approach

The numerical-algorithmic procedures of fractional counting and field normalization are often mentioned as indispensable requirements for bibliometric analyses. Against the background of the increasing importance of statistics in bibliometrics, a multilevel Poisson regression model (level 1: publication, level 2: author) shows possible ways to consider fractional counting and field normalization in a statistical model (fractional counting I). However, due to the assumption of duplicate publications in the data set, the approach is not quite optimal. Therefore, a more advanced approach, a multilevel multiple membership model, is proposed that no longer provides for duplicates (fractional counting II). It is assumed that the citation impact can essentially be attributed to time-stable dispositions of researchers as authors who contribute with different fractions to the success of a publication’s citation. The two approaches are applied to bibliometric data for 254 scientists working in social science methodology. A major advantage of fractional counting II is that the results no longer depend on the type of fractional counting (e.g., equal weighting). Differences between authors in rankings are reproduced more clearly than on the basis of percentiles. In addition, the strong importance of field normalization is demonstrated; 60% of the citation variance is explained by field normalization.     

The influences of counting methods on university rankings based on paper count and citation count

In an age of intensifying scientific collaboration, the counting of papers by multiple authors has become an important methodological issue in scientometric based research evaluation. Especially, how counting methods influence institutional level research evaluation has not been studied in existing literatures. In this study, we selected the top 300 universities in physics in the 2011 HEEACT Ranking as our study subjects. We compared the university rankings generated from four different counting methods (i.e. whole counting, straight counting using first author, straight counting using corresponding author, and fractional counting) to show how paper counts and citation counts and the subsequent university ranks were affected by counting method selection. The counting was based on the 1988–2008 physics papers records indexed in ISI WoS. We also observed how paper and citation counts were inflated by whole counting. The results show that counting methods affected the universities in the middle range more than those in the upper or lower ranges. Citation counts were also more affected than paper counts. The correlation between the rankings generated from whole counting and those from the other methods were low or negative in the middle ranges. Based on the findings, this study concluded that straight counting and fractional counting were better choices for paper count and citation count in the institutional level research evaluation.     

科学计量研究中全计数法与分数计数法研究综述

[目的/意义]对科学计量研究中计数方法的相关概念进行界定,构建计数方法分类体系,梳理比较计数方法的特征和差异,分析现存问题并提出未来改进的方向和选择计数方法的建议。[方法/过程] 首先概括计数方法的组成要素和使用流程,从信誉值分配的角度提出计数方法分类的两个要素,将计数方法分为全计数法与分数计数法两大类,并对各方法进行概述;以全计数与分数计数法的等权算法--full counts与fractional counts为例,从论文指标、引文指标、网络指标3个视角,比较计数方法的差异。[结果/结论] 文章对于全计数与分数计数方法的优劣势、计数单元与计数对象的一致性、信誉值分配规则合理性、网络影响力测度4个方面的问题进行了思考,指出在未来上述4个方面进一步研究的方向。     

中国档案学的科学精神

科学精神是科学家为实现科学活动的目标而应遵循的价值规范体系.中国档案学研究需要科学精神.然而,中国档案学思想先天"发育不良",中国档案学研究后天"营养过剩",需要从研究方法、研究态度、研究兴趣、研究领域等层面予以改造.     

The many faces of mobility: Using bibliometric data to measure the movement of scientists

This paper presents a methodological framework for developing scientific mobility indicators based on bibliometric data. We identify nearly 16 million individual authors from publications covered in the Web of Science for the 2008–2015 period. Based on the information provided across individuals’ publication records, we propose a general classification for analyzing scientific mobility using institutional affiliation changes. We distinguish between migrants--authors who have ruptures with their country of origin--and travelers--authors who gain additional affiliations while maintaining affiliation with their country of origin. We find that 3.7% of researchers who have published at least one paper over the period are mobile. Travelers represent 72.7% of all mobile scholars, but migrants have higher scientific impact. We apply this classification at the country level, expanding the classification to incorporate the directionality of scientists’ mobility (i.e., incoming and outgoing). We provide a brief analysis to highlight the utility of the proposed taxonomy to study scholarly mobility and discuss the implications for science policy.     

Are Researchers Reading the Journals they Publish in? A Case Study of Icahn School of Medicine

In this study we sought to find whether there is a correlation between the journals that scientists read to those that they publish in. For the analysis, we selected the journals that include at least 10 articles authored or coauthored by Icahn School of Medicine at Mount Sinai scientists and journals that were used at least 10,000 times in 2015. For each of the journals we also collected the Journal Impact Factor value for 2015. Our results show that Icahn School of Medicine at Mount Sinai scientists do not necessarily publish in and read the same journals at the same rate. The data also shows that scientists tend to read journals with higher impact factor.     

Introducing recalibrated academic performance indicators in the evaluation of individuals’ research performance: A case study from Eastern Europe

In Hungary, the highest and most prestigious scientific qualification is considered to be the Doctor of Science (DSc) title being awarded by the Hungarian Academy of Sciences. The academic performance indicators of the DSc title are of high importance in the evaluation of individuals’ research performance not only when a researcher applies for obtaining a DSc title, but also during promotions and appointments at universities, and in the case of the evaluation of applications for scientific titles and degrees, and the assessment of applications for funding. In the Section of Earth Sciences encompassing nine related disciplines, rather than carrying out a straightforward bibliometric analysis, the performance indicators were designed as a result of a consensual agreement between leading academicians, each of whom represented a particular discipline. Therefore, the minimum values of the indicators, required to be fulfilled if one is applying for a DSc title, do not adequately reflect the actual discipline-specific performance of researchers. This problem may generate tension between researchers during the evaluation process. The main goal of this paper is to recalibrate the minimum values of four major performance indicators by taking the actual discipline-specific distance ratios into account. In addition, each minimum value will be defined by employing integer and fractional counting methods as well. The research outcome of this study can provide impetus for the Section of Earth Sciences (and eventually other sections of the Hungarian Academy of Sciences) to optimize the minimum values of the DSc title performance indicators by taking the specifics of each discipline into account. Because academic performance indicators are also employed in other Eastern European countries in the evaluation of individuals’ research performance, the methods used in that paper can be placed into a wider geographical context.     

Statistical relationships between corresponding authorship,international co-authorship and citation impact of national research systems

This paper presents a statistical analysis of the relationship between three science indicators applied in earlier bibliometric studies, namely research leadership based on corresponding authorship, international collaboration using international co-authorship data, and field-normalized citation impact. Indicators at the level of countries are extracted from the SIR database created by SCImago Research Group from publication records indexed for Elsevier’s Scopus. The relationship between authorship and citation-based indicators is found to be complex, as it reflects a country’s phase of scientific development and the coverage policy of the database. Moreover, one should distinguish a genuine leadership effect from a purely statistical effect due to fractional counting. Further analyses at the level of institutions and qualitative validation studies are recommended.     

论科技编辑工作的创造性价值

在科技编辑工作的每一个环节中都体现出编辑劳动的创造性价值。编辑家在选题组稿时,择优弃劣,弘扬创造性,引导着科技创造活动的新潮流;在修改稿件时,将自身的创造性融入稿件中,将稿件转变为文献,使信息智化;在总体设计时,构造科技书刊整体,产生创造性价值增量,增大其创造性;在编辑产品流通时,对科学技术中的创造性成果发挥着运载传播、转化促生和承前启后的重要作用。     

A Quantitative Analysis of Research Output of Jawaharlal Nehru University,New Delhi,India

This study provides an overview of the research output of Jawaharlal Nehru University, New Delhi, India, in terms of number of publications contributed by faculty members and researchers. The study analyzes bibliometric indicators of published research of Jawaharlal Nehru University as indexed in the Web of Science. There were 5,007 research publications from Jawaharlal Nehru University in the four decades from 1971–2010 (70s; 80s; 90s; and 2000s). The study noted encouraging trends in collaborative research work with multiple-authored research papers. International collaboration in research was also observed among the faculty members and researchers. English was the predominant language of communication. The social scientists and humanists of Jawaharlal Nehru University publish books which are not indexed in the Web of Science. Similarly, research work published in regional languages or published in national level journals are seldom included in the Web of Science. It suggests that research needs to be done by collecting data from the Indian Citation Index and books published by the researchers in order to get an accurate and consolidated picture of the research output of the university.     

Bibliographic coupling networks reveal the advantage of diversification in scientific projects

Scientific research project serves as a fundamental part of supporting research progress, the data of which can reflect the research behavior of scientists. We use data from the acknowledgements of papers published by the American Physical Society (APS) and focus on the patterns of the successfully established scientific research projects. We find that the number of articles published by each project and the number of projects acknowledged by each article both obey the power-law distribution. After constructing each project’s bibliographic coupling network, it can be found that there is sublinear relationship between project’s core research fields and project scale. We explore factors affecting projects’ outcome and find the advantage of diversification in scientific projects, i.e., the dispersion of research fields can lead to higher ratio of papers in top journals and higher originality of papers. Meanwhile, evolution of research fields indicates that a typical project tend to study the core topics first, and then explore other possible research directions.     

A preliminary investigation on diffusion through a layered system

We introduce layered systems such as the citations–citing authors–citing institutes–citing countries one. Diffusion of scientific ideas flows through such layered systems. Our contribution contains three main topics: a fractional counting system for the number of different units in a layer; the fractional number of items of the same type, i.e. in the same layer, over which ideas have been diffused; and the evenness of diffusion over different layers. In this way we construct a coherent system to measure the extent to which scientific ideas are diffused.     

The representation and professional success of scientific diasporas of the Pushchino Scientific Center abroad

This paper analysis the composition and success of scientific diasporas from three research institutes of the Pushchino Scientific Center of the Russian Academy of Sciences that were formed as a result of the outflow of personnel in the 1990–2014 period. It is shown that the current cooperation of our scientists with representatives of scientific diasporas abroad is very weak. In this regard, it is noted that it is necessary to monitor scientific diasporas at the local level, which will help in establishing bilateral scientific relations. It is assumed that this monitoring can be offered to scientific and academic libraries as a new functions.     

Authorship order

Considering the fact that authorship order plays such a significant role as a basis for scientific merit, this paper looks into the practices of authorship order from a research ethical perspective. We conclude that there is a wide variety of practices and no common understanding of what the different authorship positions signify. Authorship guidelines do not provide much help. We recognize that, regardless of what system for valuing authorship positions is used, it will be misleading and unfair in most applications because relative contributions vary in ways that are not captured by fixed value assignments to authorship positions. In theory, assigning percentage figures reflecting the relative contributions of the authors would solve that problem, but we argue that such a scheme is not likely to work in practice. It can also be questioned whether relative, rather than absolute, contributions should be the basis for scientific merit. Contributorship is discussed as an alternative, but is recognized to be insufficient both in communicating absolute and relative contributions, as standardly used. However, there may be a way forward with contributorship, but then, the level of detail needs to increase considerably and its application be standardized.     

科技期刊设立客座编辑需解决的2个关键问题——以《重庆医科大学学报》的实践为例

设立客座编辑能吸引优质稿源,提高期刊学术影响力,引领学科前沿,服务行业发展.《重庆医科大学学报》从2015年开始设立客座编辑,出版特色专辑,取得了一定成效,但也暴露出了一些问题.本文以设立客座编辑需解决的2个关键问题为例,分享《重庆医科大学学报》的实践经验:采用亲情邀约、专家推荐、学者自荐的方式寻找"合适的"客座编辑;采取科学的分工搭配,明确的责、权、利关系,有效的激励措施,便捷的配套设施调动客座编辑的积极性,以期为其他期刊提供参考.     

Biblioranking fundamental physics

Counting of number of papers, of citations and the h-index are the simplest bibliometric indices of the impact of research. We discuss some improvements. First, we replace citations with individual citations, fractionally shared among co-authors, to take into account that different papers and different fields have largely different average number of co-authors and of references. Next, we improve on citation counting applying the PageRank algorithm to citations among papers. Being time-ordered, this reduces to a weighted counting of citation descendants that we call PaperRank. We compute a related AuthorRank applying the PageRank algorithm to citations among authors. These metrics quantify the impact of an author or paper taking into account the impact of those authors that cite it. Finally, we show how self- and circular-citations can be eliminated by defining a closed market of Citation-coins. We apply these metrics to the InSpire database that covers fundamental physics, presenting results for papers, authors, journals, institutes, towns, countries for all-time and in recent time periods.