Supporting practitioners in school-based evaluations of autism spectrum disorders

Students with autism spectrum disorder (ASD) have vastly different educational needs. Although some students with ASD may perform well across subjects within the general education classroom, other students with ASD may need more individualized support outside of the classroom. Historically, ASD assessments in schools have primarily focused on the measurement of cornerstone behaviors of ASD such as deficits in social communication and restricted or repetitive behaviors while underestimating the focus of academic skills even though both factors are required in the evaluation process and for an eligibility classification. The purpose of this article is to provide an overview of academic assessment practices in the schools by highlighting best practices in ASD evaluation, outlining ASD academic profiles and how to identify academic skill deficits, and reviewing the available literature that comes from different formal- and function-based assessment practices that are commonly used in schools today. The authors also discuss how formal- and function-based assessments both serve a purpose and are recommended to be used in conjunction with one another to best demonstrate a student's academic profile. Practitioners are ultimately encouraged to use a multi-informant, multi-modal approach when it comes to psychoeducational evaluations for individuals with ASD as it is evident that an individualized approach is essential due to the discrepancy of academic profiles related to this disability category as well as understanding that no measure is a perfect representation of any one skill.     

Perceptions of Science and Technology: A Comparison of Criminal Justice and Traditional Science and Technology Majors

Due to the increased demands for science and technology (S&T) in criminal justice (CJ) occupational preparation and in response to the growing interest of students in forensics, this study explores the attitudes of CJ majors towards S&T. Despite the acknowledgment that S&T skills are helpful in gaining employment, CJ majors reported less favorable attitudes than S&T majors, reporting: less interest in gaining S&T knowledge, fewer social benefits of S&T, and greater concerns that S&T are dangerous to society than traditional forensic science majors. While these findings raise some fundamental questions about the suitability of increased representation of S&T courses within CJ curriculum, authors argue that CJ curricular goals must continue to evolve to foster innovative students who are academically prepared to meet the twenty-first century challenges of their future careers.     

From eligibility assessment to intervention for students with autism spectrum disorder

The mean age of identification for autism spectrum disorder (ASD) occurs during the months before children enter kindergarten. The incidence and prevalence of “autistic behaviors” in the general population are not fully understood, however, census-matched studies of these behaviors have yielded important new data. As children presenting with behaviors characteristic of ASD enter school, school psychologists are increasingly faced with the assessment and determination of individuals with disabilities improvement act (IDEIA) eligibility, often with students yet to be formally diagnosed with ASD. In many US states, a clinical diagnosis of ASD is not required for a child's eligibility to be served under the category of other health impaired as autism. Nor does a diagnosis of ASD automatically qualify a student as eligible under IDEIA. Eligibility hinges on educational impact. This article briefly reviews the current state of knowledge as it relates to understanding and evaluating children with suspected ASD in the schools, offers a framework for assessment, and discusses a set of proposed guidelines to assist in effectively utilizing assessment data in the development of individualized education plan goals.     

What motivates high-school students to pursue STEM careers? The influence of public attitudes towards science and technology in comparative perspective

This study examines the degree of association between students’ STEM occupational expectations and between-country differences in public attitudes toward science and technology (S&T). This study focuses on public attitudes among two different populations: students and adults. Three-level Hierarchical Generalised Linear Models are employed to analyse large-scale international data from the Programme for International Student Assessment (PISA). Results reveal that public attitudes toward S&T are positively associated with students’ expectations around pursuing STEM careers. Further analyses reveal that the positive association between public attitudes toward S&T and STEM career expectations remains consistent across gender, but that this association varies across performance levels. The positive association between public attitudes toward S&T and STEM career expectations is stronger for low achievers in science than for high achievers. These results advance the scholarly understanding of the ways in which social factors may be associated with students’ choices in pursuing STEM occupations.     

A quasi-experimental research on the educational value of performance assessment

The purpose of this study is to demonstrate that performance assessment increases educational value in teachinglearning activities using a quasi-experimental research design. In this research, the three measurement criteria of educational value are suggested as ‘improvement & advancement,’ ‘sincerity & enthusiasm,’ and ‘individuality & wholeness.’ A pre-test was administered to 4 classes (156 students) in 7th grade. Classes were divided into an experimental group (2 classes, 79 students) and a control group (2 classes, 77 students), according to the pre-test results. Only the experimental group was involved in the performance assessment for 9 weeks. The results of this study show that performance assessment has a positive effect on the educational value of teaching-learning activities in schools.     

Reading assessments for students with ASD: a survey of summative reading assessments used in special educational schools in the UK

Schools have an obligation to assess the literacy skills of their students, and the provision of reading instruction to students includes the ability to measure progress in this area. However, the design of reading tests includes the ability not only to read words, but also the ability to verbalise them. This presents a particular challenge for practitioners working with students with autism spectrum disorders (ASD) who can be nonverbal in many cases. How this issue is generally overcome is currently unknown. A survey was developed, in the form of an online multiple‐choice questionnaire, in order to determine which tests are currently being used in the UK to assess the reading abilities of nonverbal students, and to examine the opinions of the education practitioners who use them, in relation to their suitability. Using the schools web directory, e‐mail invitations were sent to 1,050 special educational needs schools across the UK, and 70 schools responded to the invitation. Respondents suggested that the majority of practitioners hold little faith in the ability of current reading assessments to provide an accurate picture of reading ability for students with ASD, and this holds particularly true for nonverbal pupils with ASD. One purpose of education assessment is to establish a baseline of students’ ability in order to plan for lifelong learning and achievement. If there is an inability on the part of schools accurately to assess the reading abilities of nonverbal students with ASD, then it would be fair to assume that this could have a negative impact on the provision of learning opportunities for this population.     

How the Discovery Channel Television Show Mythbusters Accurately Depicts Science and Engineering Culture

High school science teachers, of course, want to motivate their students to consider studying science and engineering (S&E) in college. However, many high school students are not familiar with what science and engineering actually entail. They may know science as little more than “systematic discovery” and engineering as nothing but “math-intensive design.” Without appreciation for the rich culture of science and engineering, students will be unlikely to choose such a field of study. The Discovery Channel television show Mythbusters helps remedy the lack of understanding many people, especially young people, have about S&E. Mythbusters presents a highly accurate vignette of the culture of science and engineering. Episodes of the show were analyzed for instances in which the culture of science and engineering was accurately depicted. Many resources, including several publications of the National Research Council, informed the media analysis. To encourage more high school students to pursue S&E in college, they need to be exposed to the culture of S&E. Mythbusters provides a window into the often unseen realm of science and engineering, allowing its viewers, who are disproportionately represented among the young adult age bracket, to see what it means to conduct science and engineering on a routine basis. High school science instructors should look to Mythbusters to provide insight into the culture of S&E that textbooks often have difficulty conveying to students.     

Automatic assessment of cognitive and emotional states in virtual reality-based flexibility training for four adolescents with autism

Tracking students’ learning states to provide tailored learner support is a critical element of an adaptive learning system. This study explores how an automatic assessment is capable of tracking learners’ cognitive and emotional states during virtual reality (VR)-based representational-flexibility training. This VR-based training program aims to promote the flexibility of adolescents with autism spectrum disorder (ASD) in interpreting, selecting and creating multimodal representations during STEM-related design problem solving. For the automatic assessment, we used both natural language processing (NLP) and machine-learning techniques to develop a multi-label classification model. We then trained the model with the data from a total of audio- and video-recorded 66 training sessions of four adolescents with ASD. To validate the model, we implemented both k-fold cross-validations and the manual evaluations by expert reviewers. The study finding suggests the feasibility of implementing the NLP and machine-learning driven automatic assessment to track and assess the cognitive and emotional states of individuals with ASD during VR-based flexibility training. The study finding also denotes the importance and viability of providing adaptive supports to maintain learners’ cognitive and affective engagement in a highly interactive digital learning environment.     

Comparison of Direct Instruction and Discrete Trial Teaching on the Curriculum-based Assessment of Language Performance of Students with Autism

There is limited research demonstrating direct instruction (DI) as an effective language intervention for students with autism spectrum disorders (ASD) and developmental disabilities (DD). Existing research has shown that instruction using partial implementation of DI programs resulted in student learning (Ganz, 2007 Ganz, M. L. (2007). The lifetime distribution of the incremental societal costs of autism. Archives of Pediatrics & Adolescent Medicine, 161, 343&349.[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]) and instruction using whole lessons resulted in positive instructional effects for students with ASD and DD (Ganz, 2007 Ganz, M. L. (2007). The lifetime distribution of the incremental societal costs of autism. Archives of Pediatrics & Adolescent Medicine, 161, 343&349.[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]). However, it is not known whether DI is more effective than other language interventions. The purpose of this study was to compare DI to an established intervention, discrete trial teaching. Thirteen students with ASD or participated in the study and data were collected using curriculum-based assessment. An independent samples t-test indicated that there was a statistically significant difference in student performance for the group who received DI. Results and their implications will be discussed.     

Technikons and Higher Education Restructuring

The paper examines technikons in South Africa with respect to some of their internal characteristics, and in relation to debates about their place in the system of higher education as a whole, currently being assessed by the new National Commission on Higher Education. Internally it is shown that technikons lag the universities by 1:3 in terms of student enrolments, and nearly half of the 100,000 (in 1991) students were enrolled at the distance learning Technikon RSA. The past decade has seen a shift away from science and technology (S&T) fields of study at technikons, despite their establishment as 'institutes of technology'. African and female students are shown to be especially under-represented in S&T fields; staff are significantly underqualified in comparison with the universities. The profile of technikons poses a number of questions in relation to the restructuring of the higher education system (HES). Issues of national and regional governance of the HES, articulation between universities-technikons-technical colleges, and ways of enhancing S&T fields and establishing staff development programmes are considered.     

The impact of criteria-referenced formative assessment on fifth-grade students’ theater arts achievement

The purpose of this study was to examine the effects of criteria-referenced formative assessment (CRFA) on students’ achievement in theater arts. The role of type of task in differentiating the treatment-achievement relationship was explored. The analytical sample included 520 fifth-grade students from 13 schools in New York City. Schools were randomly assigned to the treatment or control group. Selection bias in treatment assignment at student level was adjusted with propensity score analysis. CRFA had a positive effect on students’ achievement on performance tasks (d = 0.25), but no significant effect on students’ performance on the analytical constructed response tasks or the theater vocabulary multiple-choice items.     

Using teacher inquiry to support technology-enhanced formative assessment: a review of the literature to inform a new method

In this paper we review the literature on teacher inquiry (TI) to explore the possibility that this process can equip teachers to investigate students’ learning as a step towards the process of formative assessment. We draw a distinction between formative assessment and summative forms of assessment [CRELL. (2009). The transition to computer-based assessment: New approaches to skills assessment and implications for large-scale testing. In F. Scheuermann & J. Björnsson (Eds.), JRC Scientific and technical reports. Ispra: Author; Webb, M. (2010). Beginning teacher education and collaborative formative e-assessment. Assessment & Evaluation in Higher Education, 35, 597–618; EACEA. (2009). National testing of pupils in Europe: Objectives, organisation and use of results. Brussels: Eurydice; OECD. (2010b). Assessing the effects of ICT in education (F. Scheuermann & E. Pedró, Eds.). Paris: JRC, OECD]. Our review of TI is combined with a review of the research concerning the way that practices with technology can support the assessment process. We conclude with a comparison of TI and teacher design research from which we extract the characteristics for a method of TI that can be used to develop technology-enhanced formative assessment: teacher inquiry into student learning. In this review, our primary focus is upon enabling teachers to use technology effectively to inquire about their students’ learning progress.     

Theory and practice in a science education course for elementary teachers

In this action research work, I analyze the theory–practice integration in teacher preparation within the context of a science and technology (S&T) education teaching methodology course aimed at future elementary teachers. The course was designed, developed and evaluated taking into account this relationship as one of its axes. The results reveal that the proposal favors progress in theory–practice integration and stimulates the students’ reflection and research. By means of a written report and an oral presentation, the students communicated their reflections on S&T teaching in the context of their concomitant professional practice. They also collected and presented resources and activities they found useful for teaching, practiced the use of certain educational S&T kits, and looked for and shared S&T pieces of news from different mass media. They additionally conducted a school learning project simulation and reflected on it. Besides, they prepared three written essays about theoretical–practical assigned readings and discussed them. In this paper, I refer to achievements and difficulties that emerged during the process, relating them to other research findings in the area and to theoretical considerations. Finally, I make proposals for further efforts in this field.     

An inventory of peer assessment diversity

Since Topping published his literature review on peer assessment in 1998 Topping, K. 1998. Peer assessment between students in colleges and universities. Review of Educational Research, 68: 249–76. [Crossref], [Web of Science ®] [Google Scholar], the number of studies on this subject has doubled, if not tripled. However, along with this expansion, the diversity of peer assessment applications increased equally fast. Based on recent literature, this contribution focuses specifically on the diversity that has come to exist in order to update Topping's typology, adding eight variables, extending eight others with extra sub‐dimensions, and including the implementation factors. Five original variables were absorbed into larger entities, leaving 20 variables which were finally grouped into five clusters. For teachers or advisors, this inventory may be of interest as a source of inspiration or as a checklist of important decisions to make when working with peer evaluation. For researchers, it may be a guide towards covering the full scope of particularities when introducing their peer assessment design. Moreover, the framework developed in this paper might relieve the possible confusion originating from the use of a single term to cover a multitude of sometimes incompatible practices.     

Strategies for using feedback students bring to higher education

This article explores strategies for acting on feedback that students bring to their higher education, by exploring student perceptions on the guidance they received from their teachers in school or college. Whilst assessment issues have been the subject of a range of research studies, it is the contention of this article that little research has been undertaken to identify the strategies that students possess for ‘using’ feedback. This small‐scale research study was stimulated by M.R. Weaver’s 2006 Weaver, M.R. 2006. Do students value feedback? Student perceptions of tutors’ written response. Assessment and Evaluation in Higher Education, 31(3): 379–394. [Taylor & Francis Online] [Google Scholar] finding that only a quarter of students had received guidance on how to use feedback prior to university. This exploration identifies the guidance for using feedback that 350 students in Humanities brought to their degree course. Analysis of responses revealed the strategies that students had been introduced to, identifying nine indigenous categories. Whilst almost 40% of these students indicated they had received guidance for using feedback, scrutiny of responses showed that many of these students confused ‘actual feedback’ with ‘guidance on how to use’ feedback. The implications of such starting positions for students and staff are considered: questioning the adequacy of such a foundation for students to be able to make effective use of the feedback they will receive in their degree course, and finally suggesting initiatives by which staff could enable students to get more out of feedback.     

Getting Everyone to the Fair: Supporting Teachers in Broadening Participation in Science and Engineering Fairs

Journal of Science Education and Technology - Science and Engineering (S&E) fairs are a valuable educational activity and are believed to increase students’ engagement and learning in...     

Objective structured professional assessments for trainee educational psychologists: an evaluation

Objective Structured Professional Assessments (OSPAs) were developed and evaluated at three universities in the United Kingdom, to supplement supervisor assessments of trainee educational psychologists’ placement practice. Participating second year students on three educational psychology doctoral programmes (n = 31) and tutors (n = 12) were surveyed. Scenarios, developed with experienced practitioners, were considered authentic and relevant. Tutor feedback indicated that the marking scheme was a good basis for judgements, but required further calibration and standardisation. Strengths and limitations relating to authenticity of scenarios and practical arrangements were identified. Students considered OSPAs to be a valid assessment of communication and perspective-taking skills, although some found them to be anxiety provoking. Levels of authenticity and complexity were deemed appropriate, although there were concerns over time allowed and how the assessment criteria were interpreted. The discussion highlights how OSPAs address issues relating to assessment reliability by improving objectivity, reducing bias and providing uniformity to student assessment experiences.     

Autism spectrum disorder assessment and evaluation research in 10 school psychology journals from 2007 to 2017

School psychologists are increasingly engaged in service provisions for students eligible for special education services under the eligibility category of autism, including conducting school‐based assessments and evaluations. Evaluations occur for a variety of reasons such as special education eligibility decision‐making, treatment and intervention planning, and progress monitoring. Publications in school psychology journals emphasizing autism spectrum disorder (ASD) assessment and evaluation are vital to quality training and practitioner utilization of quality practices. In the current study, researchers conducted a systematic review of publicaftions in 10 school psychology journals from 2007 to 2017 to assess the current state of ASD assessment and evaluation research in the field of school psychology. Implications for researchers, trainers, and practitioners are discussed.     

Motivation and metacognition when learning a complex system

Our cognitive-motivational process model (Vollmeyer & Rheinberg, 1998) assumes that motivational factors (i.e., mastery confidence, incompetence fear, interest, and challenge) affect performance via mediators. Previous studies (Vollmeyer, Rollett, & Rheinberg, 1997) found that strategy systematicity and motivational state during learning mediate the impact of initial motivation on the learning of a complex system. Potential mediators could be other cognitive (e.g., hypothesis testing) and metacognitive aspects, in that more motivated learners (high mastery confidence, low incompetence fear, high interest) analyse more deeply. Verbal protocols from 44 students who learnt to control a complex dynamic system were collected. We measured their initial motivation (on the four factors specified), then during learning we assessed their strategy systematicity and motivational state. Additionally, we analysed the verbal protocols to obtain indicators of learners’ cognitive and metacognitive processes. Performance measures were levels of knowledge acquisition and application. The cognitive-motivational process model was replicated. Qualitative cognitive aspects were added as mediators, however, the results for metacognition were problematic, partly because participants gave relatively few clearly expressed metacognitive statements.     

Formative and shared assessment in higher education. Lessons learned and challenges for the future

Formative assessment is the process by which teachers provide information to students during the learning process to modify their understanding and self-regulation. An important process within this is shared assessment, which refers to student involvement in the assessment and learning practice, a process of dialogue and collaboration between teacher and students aimed at improving the learning process, both individually and collectively. The purpose of this paper is to review the current state of affairs in depth. This paper therefore highlights, on the one hand, the lessons learned through research and development in higher education (i.e. providing clear learning goals and feedback, guiding learning, involving students in learning and assessment, promoting feedback as a process of dialogue, and making processes viable). On the other hand, these lessons also suggest some challenges and difficulties that must be addressed in the future in order to further improve formative and shared assessment in higher education. These include the need for more research on its effects, further conceptual clarification, the intersubjectivity of the process, recognition of the divergent processes and ethical principles, students’ involvement not only in assessment but also in determining academic grades, and broading learning goals and objetives in FA & SA.