Pedagogical Considerations for Examining Rater Variability in Rater‐Mediated Assessments: A Three‐Model Framework

Rater‐mediated assessments are a common methodology for measuring persons, investigating rater behavior, and/or defining latent constructs. The purpose of this article is to provide a pedagogical framework for examining rater variability in the context of rater‐mediated assessments using three distinct models. The first model is the observation model, which includes ecological/environmental considerations for the evaluation system. The second model is the measurement model, which includes the transformation of observed, rater response data to linear measures using a measurement model with specific requirements of rater‐invariant measurement in order to examine raters’ construct‐relevant variability stemming from the evaluative system. The third model is the interaction model, which includes an interaction parameter to allow for the investigation into raters’ systematic, construct‐irrelevant variability stemming from the evaluative system. Implications for measurement outcomes and validity are discussed.     

Sense of Belonging and Self-Regulated Learning: Testing Achievement Goals as Mediators

We examined two aspects of college students' (N = 385) sense of belonging and its relations with three indicators of self-regulated learning. We also tested the mediating role of achievement goals in these relations. One aspect, sense of belonging to school, functioned as a significant predictor of self-reported metacognitive and academic time management strategies. In comparison, a second aspect, sense of belonging to peer groups, was a significant predictor of self-reported peer learning strategies. Findings from the mediation analyses indicated that sense of belonging to school was related with mastery goals, whereas sense of belonging to peer groups was related with performance goals. Further, mastery goals mediated the relations between sense of belonging to school and metacognitive and academic time management strategies.     

The introduction and refinement of the assessment of digitally recorded audio presentations

Abstract

This case study critically evaluates benefits and challenges of a form of assessment included in a final year undergraduate Religious Studies Open University module, which combines a written essay task with a digital audio recording of a short oral presentation. Based on the analysis of student and tutor feedback and sample assignments, this study critically examines how teaching and learning practices linked to this novel form of assessment have been iteratively developed in light of the project findings over a period of two years. It concludes that while this form of assessment poses a number of challenges, it can create valuable opportunities for the development of transferable twenty-first-century graduate employability skills as well as deep, effective learning experiences, particularly – though not exclusively – in distance learning settings.     

Effective Early Literacy Skill Development for Young Spanish-Speaking English Language Learners: An Experimental Study of Two Methods

Ninety-four Spanish-speaking preschoolers ( M age = 54.51 months, SD  = 4.72; 43 girls) were randomly assigned to receive the High/Scope Curriculum (control n =  32) or the Literacy Express Preschool Curriculum in English-only ( n =  31) or initially in Spanish transitioning to English ( n =  31). Children's emergent literacy skills were assessed before and after the intervention in Spanish and English. Children in the English-only and transitional groups made significant gains in their emergent literacy skills in both Spanish and English compared to the control group, The English-only and transitional models were equally effective for English language outcomes, but for Spanish-language outcomes, only the transitional model was effective. The results suggest that a targeted early literacy intervention can improve Spanish-speaking preschoolers' preliteracy skills.     

Amplifying organisational discourses to the public: Media narratives of Teach For Australia, 2008–2020

Alternative teacher education programmes have emerged in many countries as a new approach to recruiting, educating and placing teachers in underperforming schools. The media plays an important role in framing perceptions of these programmes and their teachers, including in Australia, but this has not been the subject of significant research to date. This study examines how one alternative teacher education programme—Teach For Australia (TFAus)—has been positioned and framed within the news media since its inception. It critically analyses the portrayal of TFAus and concomitant educational discourses to explore their connection to larger issues surrounding teacher education and teaching quality. Drawing on an analysis of 122 print/online media articles, we identify narratives related to prestige, benevolence and the ‘alternative’ nature of the programme, alongside a narrative critical of TFAus. Despite this critical narrative, in investigating and opening up a dialogue on the perceptions and depictions of TFAus, we posit that the programme—although controversial in nature—has generally benefited from print media coverage, advancing its reputation as a major contributor to education reform and champion of educational equity.     

Learning to solve ill-defined statistics problems: does self-explanation quality mediate the worked example effect?

Instructional Science - Extensive research has established that successful learning from an example is conditional on an important learning activity: self-explanation. Moreover, a model for...     

The future is present in the past: A meta-analysis on the longitudinal associations of parent–adolescent relationships with peer and romantic relationships

Positive peer and romantic relationships are crucial for adolescents' positive adjustment and relationships with parents lay the foundation for these relationships. This longitudinal meta-analysis examined how parent–adolescent relationships continue into later peer and romantic relationships. Included longitudinal studies (k = 54 involving peer relationships, k = 38 involving romantic relationships) contained demographically diverse samples from predominantly Western cultural contexts. Multilevel meta-regressions indicated that supportive and negative parent–adolescent relationships were associated with supportive and negative future peer and romantic relationships. Meta-analytic structural equation modeling (k = 54) indicated that supportive parent–adolescent relationships unidirectionally predicted supportive and negative peer relationships, while negative parent–adolescent relationships were bidirectionally associated with supportive and negative peer relationships. Maintaining mutually supportive relationships with parents may help adolescents to develop positive social relationships.     

DEVELOPMENT, EVALUATION, AND VALIDATION OF A PAPER-AND-PENCIL TEST FOR MEASURING TWO COMPONENTS OF BIOLOGY TEACHERS’ PEDAGOGICAL CONTENT KNOWLEDGE CONCERNING THE “CARDIOVASCULAR SYSTEM”

One main focus of teacher education research concentrates on teachers’ pedagogical content knowledge (PCK). It has been shown that teachers’ PCK correlates with teaching effectiveness as well as with students’ achievement gains. Teachers’ PCK should be analyzed as one of the main important components to evaluate professional development programs. On this account, it is necessary to develop standardized measures of biology teachers’ topic-specific PCK that are labor-efficient. This paper presents a study on the development, evaluation, and validation of a paper-and-pencil test to measure biology teachers’ declarative PCK on the topic of blood and the human cardiovascular system. The development of the test was based, among other considerations, on a review of research literature on PCK and an analysis of 50 videotaped biology lessons. The final test instrument was comprised of 15 items distributed across 2 scales. The findings of the main study—with 93 preservice and in-service biology teachers and 12 biologists—confirmed that this measure of biology teachers’ declarative PCK was reliable, objective, and valid. In-service biology teachers scored higher on the test than preservice teachers (effect size Cohen’s d, 0.65) on one hand and, also, than biologists (Cohen’s d, 1.00) on the other hand. Future versions of this test should explore enlarging the scales and measuring procedural aspects of PCK.