An empirical review of anonymity effects in peer assessment,peer feedback,peer review,peer evaluation and peer grading

Abstract

Peer assessment has proven to have positive learning outcomes. Importantly, peer assessment is a social process and some claim that the use of anonymity might have advantages. However, the findings have not always been in the same direction. Our aims were: (a) to review the effects of using anonymity in peer assessment on performance, peer feedback content, peer grading accuracy, social effects and students’ perspective on peer assessment; and (b) to investigate the effects of four moderating variables (educational level, peer grading, assessment aids, direction of anonymity) in relation to anonymity. A literature search was conducted including five different terms related to peer assessment (e.g., peer feedback) and anonymity. Fourteen studies that used a control group or a within group design were found. The narrative review revealed that anonymous peer assessment seems to provide advantages for students’ perceptions about the learning value of peer assessment, delivering more critical peer feedback, increased self-perceived social effects, a slight tendency for more performance, especially in higher education and with less peer assessment aids. Some conclusions are that: (a) when implementing anonymity in peer assessment the instructional context and goals need to be considered, (b) existent empirical research is still limited, and (c) future research should employ stronger and more complex research designs.     

Does peer assessment promote student learning? A meta-analysis

Abstract

In recent years, there has been an increasing use of peer assessment in classrooms and other learning settings. Despite the prevailing view that peer assessment has a positive effect on learning across empirical studies, the results reported are mixed. In this meta-analysis, we synthesised findings based on 134 effect sizes from 58 studies. Compared to students who do not participate in peer assessment, those who participate in peer assessment show a .291 standard deviation unit increase in their performance. Further, we performed a meta-regression analysis to examine the factors that are likely to influence the peer assessment effect. The most critical factor is rater training. When students receive rater training, the effect size of peer assessment is substantially larger than when students do not receive such training. Computer-mediated peer assessment is also associated with greater learning gains than the paper-based peer assessment. A few other variables (such as rating format, rating criteria and frequency of peer assessment) also show noticeable, although not statistically significant, effects. The results of the meta-analysis can be considered by researchers and teachers as a basis for determining how to make effective use of peer assessment as a learning tool.     

The role of technology-facilitated peer assessment and supporting strategies: a meta-analysis

Abstract

Technology-facilitated peer assessment is gaining increasing attention. However, evidence for the contribution of technology-facilitated peer assessment to learning achievements has not been investigated. The present meta-analysis integrated findings on the effects of technology-facilitated peer assessment based on two main elements: (1) technology-facilitated peer assessment, (2) the use of extra supporting strategies in technology-facilitated peer assessment. A total of 37 empirical studies published from 1999 to 2018 were selected and analysed. Results indicated that technology-facilitated peer assessment had a significant and medium effect on learning achievements with an overall mean effect size of 0.576. The use of extra supporting strategies in technology-facilitated peer assessment also produced a positive and medium effect on students’ learning achievements with an overall mean effect size of 0.543. Different moderator variables, such as task types, assessment modes, training for assessors, durations, grouping types and assessment methods were related to different effect sizes. The results together with the implications for both practice and research are discussed.     

Learning theories and assessment methodologies – an engineering educational perspective

This paper attempts to critically review theories of learning from the perspective of engineering education in order to align relevant assessment methods with each respective learning theory, considering theoretical aspects and practical observations and reflections. The role of formative assessment, taxonomies, peer learning and educational policy as regards promoting the learning of engineering is discussed. It is suggested that an integrated learning method in which cognitive levels, social factors and teamwork and behaviouristic elements are integrated will optimise the learning process on an engineering course. Moreover, assessment of learning should not be isolated from views of teaching and the learning methods employed by the university teacher.     

The assessment cycle: a model for learning through peer assessment

This paper advances a model describing how peer assessment supports self-assessment. Although prior research demonstrates that peer assessment promotes self-assessment, the connection between these two activities is underspecified. This model, the assessment cycle, draws from theories of self-assessment to elaborate how learning takes place through peer assessment. The model is applied to three activity structures described in the literature to analyse their potential to support learning by promoting self-assessment. Broadly speaking, the model can be used to understand learning that takes place in a variety of peer assessment activities: marking/grading, analysis, feedback, conferencing and revision. This approach contrasts most studies on peer assessment, which have focused on calibration of instructor and peer grades, rather than learning opportunities.     

网络同伴互评活动设计与应用研究

同伴互评作为一种形成性评价策略,已经成为现阶段网络学习评价领域研究的热点。本研究将同伴互评策略应用于高校混合式教学之中,设计同伴互评学习活动。研究整体采用行动研究法设计、修改同伴互评活动,并分析学习者对活动的改进意见。研究一方面分析互评活动对在线学习效果的影响以及活动对于两类学习风格学习者影响的差异;另一方面深入分析学习者对于活动的态度及改进意见。主要研究结论认为：（1）研究设计的同伴互评活动能够促进学习者深入理解学习内容,并提升最终成果的质量。（2）相对综合型学习风格的学习者而言,同伴互评更加有助于提升序列型学习者对知识运用的水平。最后,研究总结了同伴互评活动设计的改进方案及后续设计要点。     

A Social-Cognitive Framework for Pedagogical Agents as Learning Companions

Teaching and learning are highly social activities. Seminal psychologists such as Vygotsky, Piaget, and Bandura have theorized that social interaction is a key mechanism in the process of learning and development. In particular, the benefits of peer interaction for learning and motivation in classrooms have been broadly demonstrated through empirical studies. Hence, it would be valuable if computer-based environments could support a mechanism for a peer interaction. Though no claim of peer equivalence is made, pedagogical agents as learning companions (PALs)—animated digital characters functioning to simulate human-peer-like interaction—might provide an opportunity to simulate such social interaction in computer-based learning. In this article we ground the instructional potential of PALs in several social-cognitive theories, including distributed cognition, social interaction, and Bandura’s social-cognitive theory. We discuss how specific concepts of the theories might support various instructional functions of PALs, acknowledging concepts that PALs cannot address. Based on the theoretical perspectives, we suggest key constituents for designing PALs that in human-peer interactions have proven significant. Finally, we review the current status of PAL research with respect to these constituents and suggest where further empirical research is necessary.     

Peer-to-peer Teaching in Higher Education: A Critical Literature Review

The aim of my critical literature review is to identify studies where students are engaged as partners in teaching in higher education and to analyze how tutors and tutees benefit from peer teaching. Thirty studies were included for review. Thirteen countries are represented and two thirds of the studies conducted in the United States of America or the United Kingdom. There is a significant representation of studies from natural- and physical science. The dominating pedagogical belief and theory is social constructivism. The most frequent study design is the use of quasi-experimental pre- and post-testing. University teachers do not comprise the view of peer teaching necessarily resulting in greater academic achievement gains or deep learning. University teachers identify and esteem other pedagogical benefits such as improving students’: critical thinking, learning autonomy, motivation, collaborative and communicative skills. The main finding of this review is the clarification that the training of generic skills benefits from peer teaching.     

Assessment of prior learning in higher education: a review from a validity perspective

The process of giving official acknowledgment to formal, informal and non‐formal prior learning is commonly labelled as assessment, accreditation or recognition of prior learning (APL), representing a practice that is expanding in higher education in many countries. This paper focuses specifically on the assessment part of APL, which undoubtedly is central to the whole process, through a review of research in this area and an analysis of the reviewed studies from a validity perspective. The research reviewed (published 1990–2007) is categorised into empirical as well as more theoretically oriented publications, with a quantitative dominance of the latter. According to the validity analysis, a majority of the studies conducted in this area relate to the evidential basis of test interpretation and use, primarily providing theoretical rationales and theories for a variety of practices. The consequential basis of test interpretation and use has not been studied to any larger extent, resulting in a lack of both theoretical and empirical studies dealing with this aspect of validity.     

How does student peer review influence perceptions,engagement and academic outcomes? A case study

Involving students in peer review has many pedagogical benefits, but few studies have explicitly investigated relationships between the content of peer reviews, student perceptions and assessment outcomes. We conducted a case study of peer review within a third-year undergraduate subject at a research-intensive Australian university, in which we examined: (1) students’ perceptions of the peer review process before and after peer review, (2) content of the peer reviews and what kinds of feedback were adopted and (3) the effect of participation in peer review on performance (grades) in the assessment task. Students overwhelmingly perceived peer review to be beneficial, and the opportunity to participate in peer review resulted in a significant improvement in the quality of work submitted for assessment. Students who benefited most from peer review were those of below-median performance, and the magnitude of benefit was related to the degree to which students engaged with the peer review process. Our study confirms that participation in peer review can lead to important improvements in performance and learning outcomes.