Teaching Scientists to Communicate: Evidence-based assessment for undergraduate science education

Communication skills are one of five nationally recognised learning outcomes for an Australian Bachelor of Science (BSc) degree. Previous evidence indicates that communication skills taught in Australian undergraduate science degrees are not developed sufficiently to meet the requirements of the modern-day workplace—a problem faced in the UK and USA also. Curriculum development in this area, however, hinges on first evaluating how communication skills are taught currently as a base from which to make effective changes. This study aimed to quantify the current standard of communication education within BSc degrees at Australian research-intensive universities. A detailed evidential baseline for not only what but also how communication skills are being taught was established. We quantified which communication skills were taught and assessed explicitly, implicitly, or were absent in a range of undergraduate science assessment tasks (n?=?35) from four research-intensive Australian universities. Results indicate that 10 of the 12 core science communication skills used for evaluation were absent from more than 50% of assessment tasks and 77.14% of all assessment tasks taught less than 5 core communication skills explicitly. The design of assessment tasks significantly affected whether communication skills were taught explicitly. Prominent trends were that communication skills in tasks aimed at non-scientific audiences were taught more explicitly than in tasks aimed at scientific audiences, and the majority of group and multimedia tasks taught communication elements more explicitly than individual, or written and oral tasks. Implications for science communication in the BSc and further research are discussed.     

When academics integrate research skill development in the curriculum

This study considered outcomes when 27 academics explicitly developed and assessed student research skills in 28 regular (non-research methods) semester-length courses. These courses ranged from small (n?=?17) to medium-large (n = 222) and included those from first year to masters in business, engineering, health science, humanities and science, across five universities in three Australian cities. The two-year study used three data sets to determine the outcomes of development and assessment initiatives: student pre- (n?=?779) and post-questionnaires (n?=?601), interviews with students (n?=?46) one year after completing a course that developed research skills and interviews with academics (n?=?17) involved in developing and assessing student research skills. These multiple sources provided evidence that students developed a variety of discipline-specific research skills and that these skills were useful for subsequent studies and especially for employment. Academics indicated that the process of making explicit the development of student research skills led to enhancement of their teaching, helping the academics to clarify major course purposes as well as enabling them to provide more substantial feedback to students than in the past. Academics also indicated that this teaching process changed their understanding of disciplinary research and, for some, even suggested new directions in their research.     

An investigation into the English language experiences of Indian international students studying in New Zealand universities

ABSTRACT

Language proficiency is highlighted in the literature on the adjustment and security of international students. India is the second top source country for mobile students globally, but few studies have focused explicitly on students from India. The purpose of this study was to examine the English language experiences of Indian students at New Zealand universities. This mixed-method study utilised a questionnaire (n?=?109) and interviews (n?=?15). Collectively the participants had high levels of confidence and proficiency in English language and adjusted more rapidly than many international students to the new academic environment. Major differences experienced by the participants between India and New Zealand related to unfamiliar forms of academic writing and tasks, and on the role of learning support services. Formal high value assessment was for some the first indication of a problem with writing. While most students adjusted through error-based learning others struggled with argumentation and avoiding plagiarism. These findings have relevance for local and international students. We recommend the use of, and research on, early low-stakes assessment of writing and on effective support services for students transitioning into new academic contexts.     

Social moderation,assessment and assuring standards for accounting graduates

Evidencing student achievement of standards is a growing imperative worldwide. Key stakeholders (including current and prospective students, government, regulators and employers) want confidence that threshold learning standards in an accounting degree have been assured. Australia’s new higher education regulatory environment requires that student achievements are benchmarked against intended programme learning outcomes, guided by published disciplinary standards and a national qualifications framework, and against other higher education providers. Here, we report on a process involving academics from 10 universities, aided by professional practitioners, to establish and equip assessors to reliably assure threshold learning standards in accounting that are nationally comparable. Importantly, we are learning more about how standards are interpreted. Based on the premise that meaning is constructed from tacit experiences, social interactions and intentional reflection on explicit information, we report outcomes of three multi-part calibration interventions, situated around judgements of the quality of the written communication skills exhibited in student work and their related assessment tasks. Qualitative data from 30 participants in the calibration process suggest that they perceive that the process both assists them both in developing a shared understanding of the accounting threshold learning standards and in the redesign of assessment tasks to more validly assess the threshold learning standards.     

Modelling water systems in an introductory undergraduate course: Students’ use and evaluation of data-driven,computer-based models

ABSTRACT

Introductory undergraduate courses present an opportunity to use disciplinary concepts in solving authentic problems. Making complex natural systems accessible to students through computer-based models allows them to practice making evidence-based predictions and communicate understanding. Despite the importance of modelling tools in formal classrooms, gaps exist in our understanding of how post-secondary students engage in computer-based modelling. Introductory courses, particularly in the hydrosciences, typically do not use these tools. This mixed methods study examines students’ model-based reasoning about a water-related issue over two years in response to a flipped course model. Students in an introductory water course learned basic hydrologic content and used a computer-based water model to complete a project. Data came from a pre-/post-course assessment, student assignments, and student interviews. Results of quantitative and qualitative data analyses show that students in the revised version of the course (Year 2, n?=?53) increased their understanding of core hydrology concepts and performed better on their evaluation of a computer-based water model, than students in the initial course (Year 1, n?=?38). We tentatively attribute these observed changes to increased active learning opportunities surrounding computer-based modelling of water systems. Findings contribute to science literacy development, undergraduate science learning environment design, and undergraduate scientific modelling.     

International experience and graduate employability: stakeholder perceptions on the connection

This paper reports the findings of an Australian qualitative study (N = 45) concerned with the way that employers, academics and students perceived connections between international experience and graduate employability. Drawing on the literature, the authors argue that increasing globalisation and internationalisation has heightened the need for graduates with the ability to operate in culturally diverse contexts. Universities have focussed upon exchange as part of internationalisation to prepare students for work but there is still limited literature on the nature of the relationship between international experience, more broadly and graduate employability. The findings suggest that all stakeholders identify clear connections between international experience and employability given outcomes associated with the forging of networks, opportunities for experiential learning, language acquisition and the development of soft skills related to cultural understandings, personal characteristics and ways of thinking.     

Creating first-year assessment support: lecturer perspectives and student access*

While online courses provide opportunities for a range of assessment tools and resources, little is known about how lecturers might best provide assessment cues to support diverse cohorts. This study explored how lecturers communicate messages about assessment expectations to first-year students in two courses at an Australian regional university (Nursing, n = 323; Engineering, n = 153), and how students accessed the assessment resources. A comparative case study methodology was used. Data included interviews with course coordinators to identify the assessment cues provided, course online content related to assessment and data recording student access of assessment resources. Diverse messaging systems were observed and verbal and written cues were abundant and embedded throughout both courses; however, not all students accessed all cues. What remains unclear is how students might bring together these cues in a productive way. Further research must explore the extent to which students can be better supported to improve their learning.     

Policies on and practices of cultural inclusivity in learning management systems: perspectives of Indigenous holistic pedagogies

Online learning has become a conventional term and practice in Australian higher education, yet cultural inclusivity for Indigenous (Indigenous for the purposes of this paper refers to Australian Aboriginal and Torres Strait Islander peoples) students is insufficiently reflected in learning management system (LMS) policies and design. This study aims to explore culturally inclusive learning entrenched in Australian university policies on and practices of LMS by applying Indigenous holistic pedagogical values in LMS design. Based on a literature review, we articulate four dimensions: communication, collaboration, community and interculturality for culturally inclusive learning in an online learning environment. By using the dimensions, we critically review policies (n?=?10) and LMS sites (n?=?50). In this review, we argue that there are contrasts of individually heterogeneous and collectively homogeneous approaches, self-focused and community-driven pedagogy, and task-oriented and relational learning. Significantly, the review results indicate that Indigenous holistic pedagogies have a metaphysical strength to be the ontological foundation for cultural inclusivity.     

Examining the use of multiple writing and discourse tasks in 5th grade mathematics

Abstract

The purpose of this study is to examine how fifth grade students were impacted by the infusion of multiple writing tasks in mathematics. In this study, writing tasks provided opportunities for students to communicate prior knowledge, share ideas to construct and justify arguments, for reflection, and assessment. In this deductive qualitative study, students’ work samples were analyzed. Findings indicated that students grew in their understanding of mathematics and ability to self-reflect and self-evaluate through multiple opportunities to write for a variety of purposes. The opportunities for constructing mathematical understanding with activities that included writing and discourse also fostered learning between peers. The findings suggest a variety of opportunities to write and engage in mathematics discourse encouraged reflection, evaluation, and learning. Implications for future research include the need to examine the impact of these activities on students’ mathematics understanding as measured by assessments or an analysis of student work samples.     

Context-based assessment: creating opportunities for resonance between classroom fields and societal fields

There is on-going international interest in the relationships between assessment instruments, students’ understanding of science concepts and context-based curriculum approaches. This study extends earlier research showing that students can develop connections between contexts and concepts – called fluid transitions – when studying context-based courses. We provide an in-depth investigation of one student’s experiences with multiple contextual assessment instruments that were associated with a context-based course. We analyzed the student’s responses to context-based assessment instruments to determine the extent to which contextual tests, reports of field investigations, and extended experimental investigations afforded her opportunities to make connections between contexts and concepts. A system of categorizing student responses was developed that can inform other educators when analyzing student responses to contextual assessment. We also refine the theoretical construct of fluid transitions that informed the study initially. Implications for curriculum and assessment design are provided in light of the findings.     

Graduate learning outcomes in science: variation in perceptions of single- and dual-degree students

The development of transferrable skillsets, articulated in statements of graduate learning outcomes, is emphasised in undergraduate science degree programmes. Science students enrolled in dual (double) degrees comprise a significant minority of Australian science undergraduates. Research comparing perceptions of single and dual degree students on their science learning outcomes has rarely been explored. The Science Students Skills Inventory was used to compare the perceptions of single (n = 640) and dual (n = 266) degree undergraduate science students. The instrument explored science graduate learning outcomes across six indicators: importance; the extent to which outcomes were included; the extent to which they were assessed; improvement; confidence; and likely future use. Analysis of findings, employing the planned–enacted–experienced curricula framework, offers insight into potential avenues towards coherence of the experienced curriculum by arguing the need for shared perceptions of graduate learning outcomes for single and dual degree science students. The key contribution of this study is a shift towards progressive curriculum development that draws on both single and dual degree student perspectives to achieve graduate learning outcomes. Recommendations include: whole-of-programmes curricular pathways premised on progressive development of learning outcomes that are inclusive of dual degree students, explicit interdisciplinary learning opportunities, and adoption of dual/single status as a demographic variable reported in future research.     

A comparative study on student perceptions of their learning outcomes in undergraduate science degree programmes with differing curriculum models

ABSTRACT

This study investigated students’ perceptions of their graduate learning outcomes including content knowledge, communication, writing, teamwork, quantitative skills, and ethical thinking in two Australian universities. One university has a traditional discipline-orientated curriculum and the other, an interdisciplinary curriculum in the entry semester of first year. The Science Students Skills Inventory asked students (n?=?613) in first and final years to rate their perceptions of the importance of developing graduate learning outcomes within the programme; how much they improved their graduate learning outcomes throughout their undergraduate science programme; how much they saw learning outcomes included in the programme; and how confident they were about their learning outcomes. A framework of progressive curriculum development was adopted to interpret results. Students in the discipline-oriented degree programme reported higher perceptions of scientific content knowledge and ethical thinking while students from the interdisciplinary curriculum indicated higher perceptions of oral communication and teamwork. Implications for curriculum development include ensuring progressive development from first to third years, a need for enhanced focus on scientific ethics, and career opportunities from first year onwards.     

Opportunities for Inquiry Science in Montessori Classrooms: Learning from a Culture of Interest, Communication, and Explanation

Although classroom inquiry is the primary pedagogy of science education, it has often been difficult to implement within conventional classroom cultures. This study turned to the alternatively structured Montessori learning environment to better understand the ways in which it fosters the essential elements of classroom inquiry, as defined by prominent policy documents. Specifically, we examined the opportunities present in Montessori classrooms for students to develop an interest in the natural world, generate explanations in science, and communicate about science. Using ethnographic research methods in four Montessori classrooms at the primary and elementary levels, this research captured a range of scientific learning opportunities. The study found that the Montessori learning environment provided opportunities for students to develop enduring interests in scientific topics and communicate about science in various ways. The data also indicated that explanation was largely teacher-driven in the Montessori classroom culture. This study offers lessons for both conventional and Montessori classrooms and suggests further research that bridges educational contexts.     

Multi‐level Assessment of Scientific Content Knowledge Gains Associated with Socioscientific Issues‐based Instruction

This study explored the impact of using a socioscientific issue (SSI) based curriculum on developing science content knowledge. Using a multi‐level assessment design, student content knowledge gains were measured before and after implementation of a three‐week unit on global warming (a prominent SSI) that explored both the relevant science content and the controversy surrounding global warming. Measures of student content knowledge were made using a standards‐aligned content knowledge exam (distal assessment) and a curriculum‐aligned exam (proximal assessment). Data were collected from 108 students enrolled from two schools. Quantitative analysis of the distal assessment indicated that student post‐test scores were statistically significantly different than their pre‐test scores (F = 15.31, p<0.001). Qualitative analyses of student responses from the proximal assessment indicated that students, on average, expressed more accurate, more detailed, and more sophisticated understandings of global warming, the greenhouse effect, and the controversy and challenges associated with these issues following the three‐week unit. Combined results from the proximal and distal assessments explored in this study offer important evidence in supporting the efficacy of using SSI as contexts for science education. In addition to a discussion of the components of an SSI‐based curriculum, this study provides support for the use of SSI as a context for learning science content.     

Students’ perceptions of undertaking workplace tasks within a foundation degree – health and social care

Foundation degrees (FDs) involve the fusion of academic and vocational paths in higher education (HE) qualifications; the challenge for academics and employers is the credible assessment of the student’s workplace learning. Focusing to the workplace enables participants to learn from their daily routines encountered at work. The challenge is to help them demonstrate learning from experience within the workplace, by developing their ability to think at a higher level. A vital aspect of using workplace learning in partnership with employers/sector skills councils is that the learning is valid to the role. In response, the programme development group, which was composed of academics and employers, felt that workplace tasks (WPTs) would allow students to demonstrate role development through experiential learning at work. Therefore, this article uses a pilot case study (n = 16), using a focus group methodology, to discuss the assessment of the WPTs within a FD and the students’ perception towards this form of assessment. Findings show that participants value WPTs, they motivate participants to learn and improve their self‐esteem, however, some groups need specialist skills training where workplace competence is required.     

How to encourage a lifelong learner? The complex relation between learning strategies and assessment in a medical curriculum

Abstract

To foster lifelong learning skills, we need new didactic approaches with aligned assessment methods. Therefore, we investigated whether the outcomes of a project assignment show a different relation to learning strategies than a longitudinal knowledge-based assessment. We studied learning strategies of first year students of medicine and biomedical sciences (n?=?248) and performed hierarchical regression analyses for the learning strategies and grades of the longitudinal knowledge-based test and project assignment. Scores of students, measured with the Motivated Strategies for Learning Questionnaire (Likert scale 1–7), were relatively low for critical thinking (3.53), compared to rehearsal (4.40), elaboration (4.82), organisation (4.69) and metacognitive self-regulation (4.33). Knowledge based tests showed a significant relation to elaboration (p?<?0.01). For the project-based assessment, we did not find a significant relation to any learning strategy (p?=?0.074). Explained variance of the grades was low for all learning strategies (R² < 0.043). Different types of assessment did not discriminate between students with high or low scores on learning strategies associated with lifelong learning. An explanation is that the curriculum is not aligned with assessment, or students do not benefit in terms of grades. We conclude that, if assessment is to drive lifelong learning skills, this is not self-evident.     

The differential effects of task complexity on domain-specific and peer assessment skills

In this study the relationship between domain-specific skills and peer assessment skills as a function of task complexity is investigated. We hypothesised that peer assessment skills were superposed on domain-specific skills and will therefore suffer more when higher cognitive load is induced by increased task complexity. In a mixed factorial design with the between-subjects factor task complexity (simple, n?=?51; complex, n?=?59) and within-subjects factor task type (domain-specific, peer assessment), secondary school students studied four integrated study tasks, requiring them to learn a domain-specific skill (i.e. identifying the six steps of scientific research) and to learn how to assess a fictitious peer performing the same skill. Additionally, the students performed two domain-specific test tasks and two peer assessment test tasks. The interaction effect found on test performance supports our hypothesis. Implications for the teaching and learning of peer assessment skills are discussed.