Feedback,feedforward: evaluating the effectiveness of an oral peer review exercise amongst postgraduate students

Assessment for learning approaches, such as peer review exercises may improve student performance in summative assessments and increase their satisfaction with assessment practices. We conducted a mixed methods study to evaluate the effectiveness of an oral peer review exercise among post-graduate students. We examined: (1) final assessment grades among students who did and did not take part in the peer review exercise; (2) student perceptions of the impact of the peer review exercise; and (3) student understanding of, and satisfaction with, this new assessment practice. We found that students who took part in the exercise had a significantly higher mean grade in a subsequent summative oral presentation assessment than students who did not take part in the exercise. Students gained a better understanding of assessment and marking criteria and expressed increased confidence and decreased anxiety about completing the subsequent summative assessment. Assessment for learning improves academic attainment and the learning experience in postgraduate students.     

Formative peer assessment in a CSCL environment: a case study

In this case study our aim was to gain more insight in the possibilities of qualitative formative peer assessment in a computer supported collaborative learning (CSCL) environment. An approach was chosen in which peer assessment was operationalized in assessment assignments and assessment tools that were embedded in the course material. The course concerned a higher education case‐based virtual seminar, in which students were asked to conduct research and write a report in small multidisciplinary teams. The assessment assignments contained the discussion of assessment criteria, the assessment of a group report of a fellow group and writing an assessment report. A list of feedback rules was one of the assessment tools. A qualitative oriented study was conducted, focusing on the attitude of students towards peer assessment and practical use of peer assessment assignments and tools. Results showed that students’ attitude towards peer assessment was positive and that assessment assignments had added value. However, not all students fulfilled all assessment assignments. Recommendations for implementation of peer assessment in CSCL environments as well as suggestions for future research are discussed.     

Spiral and Project-Based Learning with Peer Assessment in a Computer Science Project Management Course

Different learning methods such as project-based learning, spiral learning and peer assessment have been implemented in science disciplines with different outcomes. This paper presents a proposal for a project management course in the context of a computer science degree. Our proposal combines three well-known methods: project-based learning, spiral learning and peer assessment. Namely, the course is articulated during a semester through the structured (progressive and incremental) development of a sequence of four projects, whose duration, scope and difficulty of management increase as the student gains theoretical and instrumental knowledge related to planning, monitoring and controlling projects. Moreover, the proposal is complemented using peer assessment. The proposal has already been implemented and validated for the last 3 years in two different universities. In the first year, project-based learning and spiral learning methods were combined. Such a combination was also employed in the other 2 years; but additionally, students had the opportunity to assess projects developed by university partners and by students of the other university. A total of 154 students have participated in the study. We obtain a gain in the quality of the subsequently projects derived from the spiral project-based learning. Moreover, this gain is significantly bigger when peer assessment is introduced. In addition, high-performance students take advantage of peer assessment from the first moment, whereas the improvement in poor-performance students is delayed.     

Peer assessment in a project-based engineering course: comparing between on-campus and online learning environments

As the number of participants in online distance learning courses increases, peer assessment is becoming a popular strategy for evaluating open assignments and for breaking the social isolation surrounding distance education. Yet, the quality and characteristics of peer assessment in massive online courses has received little attention. Hence, this study was set to examine peer feedback quality and grading accuracy in a project-based course. The study applied a sequential exploratory mixed methods design. It included 339 participants who studied the same engineering course, but in three different modes: on-campus (n = 77), small private online course (n = 110), and massive open online course (MOOC) (n = 152). Content analysis of feedback comments identified four categories: reinforcement, statement, verification and elaboration, arranged in an ascending scale of cognitive ability. The findings indicated that the MOOC participants provided more feedback comments and volunteered to assess more projects than their counterparts did. However, the on-campus students provided higher quality feedback and their peer grading was better correlated with the grades assigned by the teaching assistants.     

Grudges and gratitude: the social-affective impacts of peer assessment

Abstract

Students’ dissatisfaction with peer assessment has been widely documented. While most relevant literature places focus on the cognitive (content and uptake of feedback) or structural (feedback design) dimensions, students’ emotions in peer assessment have received scant attention. This study investigates the social-affective impacts of peer assessment by analysing students’ appeal letters addressed to their tutors, reflective posts in the online discussion forum and responses to a survey. A thematic analysis of data indicated three main aspects of students’ (dis)satisfaction: content, scores and process of peer assessment. The most negative emotion that students expressed was related to ‘disrespectful’ behaviour and attitudes of peer reviewers, whereas the feeling of appreciation was triggered by the helpful feedback attributes which were perceived as reflecting reviewers’ respect to others’ works. Students generally held mixed feelings toward peer assessment, valuing learning in the process of providing and receiving feedback but showing resistance to using peer assessment for summative purposes. The findings highlight the significance of respect in peer assessment and argue that the perceived lack of mutual respect seems to underpin the nature of students’ dissatisfaction. This study carries implications for nurturing students’ respectful attitudes and behaviour in and through peer assessment.     

'In Most Classes You Sit Around Very Quietly at a Table and get Lectured at…': Debates,assessment and student learning

This paper details a collaborative action research enquiry undertaken while both authors worked at the University of Glasgow. It explores the use of class debates as a teaching method in an International Management Honours course as the framing context for developing students' capacity to assess their own and each other's learning through the debates. In addition, issues of assessment for grading purposes are signalled and explored with the students. The collaborating partners in the study were a Management Studies lecturer and a Higher Education Studies lecturer, who worked together on the framework for the debates, reflected on the unfolding process together, and collected and analysed evidence. This case suggests that the debates enabled students to develop a critical view of the topics under discussion and to acquire a number of 'transferable skills', for example, team work. On peer grading, students were ambivalent. While self-and peer assessment appears to work well for formative purposes, summative peer assessment may not be welcomed by students.     

The 'Mini-Viva' as a Tool to Enhance Assessment for Learning

This paper reports on an action research project which aimed to promote assessment for learning within a summative assignment. In an attempt to provide enhanced feedback to students, the process of their group assignment involved lecturer feedback at various stages. A particular focus was on a 'mini-viva', whereby students explained and justified to the lecturer selected aspects of their assignment after it was submitted but before a mark was awarded. Oral and written evaluation data were collected from students, two peer observers and from colleagues taking part in an action research team. Some implications for 'good assessment practices' are discussed.     

In search of quality criteria in peer assessment practices

With the increasing popularity of peer assessment as an assessment tool, questions may arise about its measurement quality. Among such questions, the extent peer assessment practices adhere to standards of measurement. It has been claimed that new forms of assessment, require new criteria to judge their validity and reliability, since they aim for specific goals and address different aspects of learning. But although new criteria have been formulated, little is known about how they are deployed and operate in actual peer assessment practices. This study intends to explicate the quality criteria relevant to the context of peer assessment. We conducted a survey in vocational education on peer assessment quality criteria applied in classrooms. Fifty-six teachers using peer assessment rated several quality criteria as relevant to their teaching practice. The findings suggest that peer assessment practices entail many of the quality criteria recognized in measurement and evaluation, although in an embedded way: the generic quality criteria are tuned or adapted to the peer assessment setting. The findings also show that peer assessment is very much in the hands of the teacher, who organizes and orchestrates the appraisal process in which students receive clear guidelines to appraise one another's work. Our results indicate that quality criteria hardly differ from the function peer assessment serves in classrooms, either within a summative or formative evaluation context. By gaining insight on the specific criteria of peer assessment and their precise purposes, we believe it becomes possible to help teachers improve its measurement quality.     

The Importance of Marking Criteria in the Use of Peer Assessment

This paper reports a study which implemented and evaluated a method of peer assessment, as a formative and summative assessment procedure. Pairs of first‐year undergraduate biology students were asked to complete a poster assignment on a specific aspect of nerve physiology. This paper contains details of a method which allows student peer and tutor marking of work against the individual marking criteria to be evaluated. The results show that a comparison between the tutor and the student peer mark may be misleading as a guide to the validity of peer assessment. The importance of considering the individual sections of the marking criteria is illustrated. It was found that when the individual criteria were analysed the number of students marking the same as the tutor ranged from 31% to 62%. It also became clear that specific areas of the marking criteria were prone to over and under‐marking. Analysis of student feedback forms showed that students not only liked carrying out peer assessment, but felt the benefits in terms of developing facets of their learning process and heightening their awareness of their work. These results are discussed in the light of other studies.     

Teachers' assessment of students' research skills

Today, teachers are expected to develop complex skills, such as research skills, in their students while implementing new views on learning and teaching and using authentic assessment strategies. About these new assessment strategies there is much debate and teachers are vulnerable in using them. We studied upper secondary education natural and social science teachers' practices using two surveys and two rounds of expert panel judgement on teacher‐submitted assessment‐related material and information. Our study shows that there are grounds for concern regarding the clarity of teachers' assessment criteria, the consistency between teachers' goals, assignments, and criteria, and the validity and acceptability of teachers' assessment practices. The extent to which it is justifiable to judge teachers' assessment practice by professional quality criteria is discussed, and suggestions are given as to the main quality criteria for formative and summative assessment and as to ways in which teachers could improve their assessment practices.     

大学英语自主学习形成性评估实证研究

自主学习要求学习者监控学习进程和评价学习效果,传统的终结性评估不适用于自主学习。构建了适用于自主学习的形成性评估模式,该模式由"学习档案袋评价""教师评价"及"学生自评和他评"构成。经过在两个教学班级一学期的教学实验,结果表明:通过对学生自主学习过程的监控和管理,形成性评估激发了学生语言学习的动机和兴趣,促进了学生学习策略形成,有效培养了学生自主学习能力。     

Explicitness of criteria in peer assessment processes for first-year engineering students

This article will outline the motives for the implementation of peer assessment as put into action at the first year of the Civil Engineering course of the University of Minho. The implementation of new assessment methods was a consequence of the successful implementation of peer assessment at other engineering courses. During the semester, three assessment moments took place, in which students assessed the work of their peers. Assessment criteria were defined in a negotiation process between students and teacher. The students subsequently graded the work of their colleagues and had to provide a clear justification for each given grade. The aim of this method is to involve students in the subject, enhance their motivation and deepen their learning. Advantages and disadvantages of peer assessment are discussed. The results that have been obtained so far draw attention to the nature of assessment criteria and the corresponding explicitness.     

Explaining students' appraisal of lectures and student-activating teaching: perceived context and student characteristics

During lectures, some students are continuously focused and attentive, whereas others tend to be bored, jittery, or inattentive. The same might happen when students are given student-activating assignments. Some students simply love one type of instruction, whereas others tend to resent it. Moreover, it is not the context itself, but the context as it is perceived by the student that affects learning. This study sets out to discover how students view instruction, based on their experiences with a particular setting, and aims to investigate the effects of “context-by-student” characteristics, such as students' course experiences, and the effects of “student-by-context” features, such as approaches to learning. The research has a quasi-experimental design and compares a lecture-based setting with student-activating teaching methods (i.e. self-discovery learning by means of authentic tasks with the help of a scaffolding approach); followed by one of four assessment methods: a multiple-choice test, a case-based examination, a peer assessment or a portfolio assessment. Data (N = 579) were collected by means of validated questionnaires, comprising the ASSIST and LSQ instruments and the course experience questionnaire. Results demonstrate that students' appraisal of the assessment method, their course experiences and the teaching methods that are dominant in the setting contribute positively to students' appraisal of instruction. Interestingly, this conclusion applies to both the lecture-based and the student-activating groups and to both conventional assessment methods and new modes of assessment. Moreover, deep approaches to learning, monitoring studying and/or organized studying, as well as preferences for courses that support understanding, positively affect students' appraisal of student-activating teaching methods. In short, both student-by-context and context-by-student features prove to be strong predictors of students' appraisal of instruction and, as a consequence, student learning.     

Response of students to statement bank feedback: the impact of assessment literacy on performances in summative tasks

Efficiency gains arising from the use of electronic marking tools that allow tutors to select comments from a statement bank are well documented, but how students use this type of feedback remains under explored. Natural science students (N = 161) were emailed feedback reports on a spreadsheet assessment that included an invitation to reply placed at different positions. Outcomes suggest that students either read feedback completely, or not at all. Although mean marks for repliers (M = 75.5%, N = 39) and non-repliers (M = 57.2%, N = 68) were significantly different (p < .01), these two groups possessed equivalent attendance records and similar submission rates and performances in a contemporaneous formatively assessed laboratory report. Notably, average marks for a follow-up summative laboratory report, using the same assessment criteria as the formative task, were 10% higher for students who replied to the original invite. It is concluded that the repliers represent a group of assessment literate students, and that statement bank feedback can foster learning: a simple ‘fire’ analogy for feedback is advanced that advocates high-quality information on progress (fuel) and a curricular atmosphere conducive to learning (oxygen). However, only if students are assessment literate (ignition) will feedback illuminate.     

How convincing is alternative assessment for use in higher education?

The current preference for alternative assessment has been partly stimulated by recent evidence on learning which points to the importance of students’ active engagement in the learning process. While alternative assessment may well fulfil the aims of formative assessment, its value in the summative assessment required in higher education is more problematic. If alternative assessment devices are to be used for summative purposes, the validity of alternative assessment has to be considered. The paper argues that task specification and marking consistency in alternative ­assessment can make comparability of performance difficult to effect, thereby leaving alternative assessment a less than convincing form for use in higher education.     

学习性评价:涵义、方法及原理

学习性评价是以"促进学生发展、教师提高和改进教学实践"为目的的内部非正式评价,也是有效教学的一个有机组成部分.学习性评价的主要方法可以概括为:(1)改进课堂提问方式,增加回答"等待时间";(2)提高作业反馈质量,促进学生深入反思;(3)加强同伴与自我评价,形成自主与责任意识;(4)两种评价结合并用,优化评价教育功能.在科学课堂上实施学习性评价,首先改变了科学教师的教学方式,其次促进了学生学习方式的转变,提高课堂互动的效果和教学质量,从而有效地促进师生的共同发展.     

Student attitudes that predict participation in peer assessment

Peer assessment has been widely applied to actively engage students in learning to write. However, sometimes students resist peer assessment. This study explores reviewers’ attitudes and other underlying factors that influence students’ participation in online peer assessment. Participants were 234 Chinese undergraduates from two different academic backgrounds: engineering majors (n = 168) and English majors (n = 66). Gender, academic background and prior experience with peer assessment were all related to participation levels. Moreover, factor analyses revealed three attitudinal factors: (1) positive attitude (a general endorsement of the benefits of peer assessment), (2) interpersonal negative (concerns about the negative effects on interpersonal relationships), and (3) procedural negative (doubts about the procedural rationality of peer assessment). Among the attitudinal factors, procedural negative was negatively associated with participation, as expected. Interestingly, interpersonal negative was associated with greater participation, and positive attitude was associated with lower participation, in part because students worked hard on each review rather than doing many reviews superficially. Implications for instruction are discussed.     

Using Assessment to Induct Students and Staff into the PBL Tutorial Process

The introduction of a new problem-based Graduate Medical Course with over 230 students and more than 200 staff required a systematic process for induction into the PBL philosophy and tutorial process. The 4SAT (4 Step Assessment Task) was designed for use as an instrument for formative and summative assessment of reasoning ability, elements of the PBL tutorial (group) process and individual learning outcomes. The instrument was developed through evaluative cycles based upon staff and student feedback, reflection and peer review. The 4SAT has face validity with the learning process and emphasises reasoning and integration of knowledge. An unexpected out come of its development was its use as a powerful tool to induct staff and students into the PBL group processes, scaffold development of hypothetico-deductive reasoning and provide guidance on the depth and breadth of learning expected in the course.     

Students’ experience of peer assessment in a professional course

This paper reports a case study that investigated the practice of peer assessment in a professional course, with particular emphasis on students’ experience. It was found that peer assessment processes were beneficial to students’ learning and development as professionals. The case study pointed to several conditions for effective implementation of peer assessment, which also relate to assessment in general. These conditions include: (a) adequate and appropriate preparation for the use of peer assessment; (b) alignment of assessment, learning objectives and the broader purpose of the course (e.g. preparation as professionals); (c) the availability of assistance from a teacher throughout the peer assessment process; and (d) constructive discussions following peer assessment, sensitively handled by a teacher.