Clinical teachers' perceptions of medical students' English language proficiency

Medical educators from the Faculty of Medicine at the University of Adelaide, South Australia, have expressed reservations about the adequacy of some undergraduate medical students' English language proficiency for satisfactory academic and clinical performance. This study explores the occurrence and nature of the comments made in writing by clinical teachers about the English language proficiency of 568 students over a period of 4 years. The frequency and nature of the comments made by clinicians have important implications for the planning and implementation of pedagogical strategies to support non-English-speaking background medical students experiencing difficulties with their course due to language. Although the University of Adelaide has introduced initiatives in response to some of the problems that have been identified, it is recommended that any teaching interventions require careful evaluation through a longitudinal research design to ensure that their aims are being achieved.     

Medical students' essay-writing skills: criteria-based self- and tutor-evaluation and the role of language background

OBJECTIVES: An exercise is described which aimed to make clear to first-year undergraduate medical students the expected writing skills required for an essay examination in one discipline. SUBJECTS: Many students were from a non-English speaking background and over one-third of students, regardless of language background, had limited experience in this type of essay writing. PROCEDURE: For this exercise, a practice essay was written by each student for formative assessment. The essay was rated by a tutor and by the student according to well-defined criteria. This allowed for comparisons to be made in a structured and objective way between the judgements of the student and the assessor. RESULTS: Students found the exercise to be very useful, although whether essay writing skills actually improved could not be established. Students from non-English speaking backgrounds tended to be most harsh in their self-evaluations, yet tutor-evaluations generally showed these students to have better writing skills than other students. Indeed, correlations between self- and tutor-evaluations were quite low. CONCLUSIONS: It is evident that students and their educators may be unclear about each others' expectations. By making explicit the requirements of an exercise, misunderstandings may be minimized and it is possible that student performance could improve, though further research is required to verify these hypotheses. It is suggested that students should be encouraged to evaluate their own work and should be instructed in writing skills throughout their medical degree education.     

Undergraduate medical students: who seeks formative feedback?

OBJECTIVE: This study aimed to monitor which undergraduate students collected formative feedback on their degree essays and to quantify any correlations between gender or summative mark achieved and whether formative feedback was sought. METHODS: We carried out a study at the University of Aberdeen Medical School, involving a total of 360 Year 3 students, comprising all 177 students in the 2004 cohort and 183 in 2005. Data on gender and summative mark were routinely collected during the degree assessment processes in March 2004 and 2005. Students signed on receipt of their feedback. RESULTS: Less than half the students (46%) collected their formative feedback: 47% in 2004, and 45% in 2005. Overall, females were significantly more likely than males to seek formative feedback (P = 0.004). Higher achievers were significantly more likely than lower achievers to seek their feedback (P = 0.020). CONCLUSIONS: Our findings indicate that these medical students, particularly males and poor students, may not use assessment feedback as a learning experience. Female and better students are keener to seek out formative feedback that might be expected to help them continue to do well. We need to explore further why so many students do not access formative feedback, and develop strategies for addressing this issue effectively.     

Student feedback in problem based learning: a survey of 103 final year students across five Ontario medical schools

CONTEXT: Problem based learning (PBL) has become an integral component of medical curricula around the world. In Ontario, Canada, PBL has been implemented in all five Ontario medical schools for several years. Although proper and timely feedback is an essential component of medical education, the types of feedback that students receive in PBL have not been systematically investigated. OBJECTIVES: In the first multischool study of PBL in Canada, we sought to determine the types of feedback (grades, written comments, group feedback from tutor, individual feedback from tutor, peer feedback, self-assessment, no feedback) that students receive as well as their satisfaction with these different feedback modalities. SUBJECTS AND METHODS: We surveyed a sample of 103 final year medical students at the five Ontario schools (University of Toronto, McMaster University, Queens University, University of Ottawa and University of Western Ontario). Subjects were recruited via E-mail and were asked to fill out a questionnaire. RESULTS: Many students felt that the most helpful type of feedback in PBL was individual feedback from the tutor, and indeed, individual feedback was one of the more common types of feedback provided. However, although students also indicated a strong preference for peer and group feedback, these forms of feedback were not widely reported. There were significant differences between schools in the use of grades, written comments, self-assessment and peer feedback, as well as the immediacy of the feedback given. CONCLUSIONS: Across Ontario, students do receive frequent feedback in PBL. However, significant differences exist in the types of feedback students receive, as well as the timing. Although rated highly by students at all schools, the use of peer feedback and self-assessment is limited at most, but not all, medical schools.     

Language background, proficiency in English, and selection for language development

With a diverse language background profile in an Australian medical student population, teaching interventions are necessary for students whose English language proficiency is not adequate for the study of medicine. This paper describes the screening of written and aural English language proficiency in 143 first year undergraduate students using a standardized instrument. Students identified as experiencing language difficulties were subsequently assessed by interview and allocated to faculty-based Language Development Programme. Students with the greatest need for language support participated in a full 2 year programme. Those requiring less assistance were offered more limited support in the form of specific modules within the programme. Students allocated to the full programme were significantly weaker in language proficiency compared to those offered specific modules and those not offered a placement. The information gathered during the structured interview is valuable in establishing for medical educators specific areas in which language-related teaching for students who require it can be directed. Future research is required to evaluate the effectiveness of faculty-based language interventions in terms of improvement in language proficiency over time and the effect of any improvements in language proficiency on academic and clinical performance.     

An evaluation of the Team Objective Structured Clinical Examination (TOSCE)

OBJECTIVES: To evaluate the validity, reliability and feasibility of the TOSCE, a new means of formative assessment for medical students, from the perspectives of examiners, simulated patients and students. DESIGN: Teams of five students rotate through five clinical stations, performing one of four tasks in turn, whilst the fifth member of the team 'rests'. SETTING: St George's Hospital Medical School, London. SUBJECTS: Third-year medical students, their examiners and simulated patients. RESULTS: All participants felt the TOSCE session had high validity although some students were unfamiliar with some subjects. Stations were double-marked and agreement rates and Cohen's Kappa ranged from 67 to 94% and 0.04-0.88, respectively. Analysis of the causes of disagreement led to improved marking schedules and enhanced reliability. CONCLUSIONS: The feasibility of the exercise as a formative assessment was demonstrated by the confidence of the staff participating both as examiners and simulated patients and from the majority of students who welcomed the learning experience.     

Language background,English language proficiency and medical communication skills of medical students

Associations between language background, English language proficiency and medical communication skills were investigated in a group of 149 third year undergraduate medical students studying at an Australian university. Written and aural English proficiency were assessed with the Screening Test for Adolescent Language (STAL). Medical communication skills and the fluency of spoken language were scored during an Observed Structured Clinical Interview (OSCI), rated by a standardized patient and a clinician. An association was found between language background, performance on the STAL and spoken language proficiency. Satisfactory performance in medical communication skills was not associated with language background or overall performance on the STAL. In this study it was the global rating of unsatisfactory spoken language fluency that was associated with poorer performance in medical communication skills under examination conditions.     

Computer-aided learning: an overvalued educational resource?

AIM: The aim of this study was to evaluate the place of computer-aided learning in a basic science course in the undergraduate medical curriculum at the University of Adelaide. METHODS: A software program was written which would allow students to study the anatomy and physiology of the liver and biliary tree in three different styles. Identical content was produced, matched for each style (problem-based, didactic and free text response) and students randomly allocated to one of four groups (three computer and one control). Students were tested before and after access to the program. RESULTS: Ninety second-year students completed the study. Those students who had access to the material in the problem-based and free text response styles did no better in the post-study test than the controls, whilst the group who had studied the didactically presented computer material performed significantly better than the other three groups. All three computer groups accessed the material on a similar number of occasions, but the group who had access to the free text entry program spent significantly less time on computer study. CONCLUSIONS: If computer materials are to be provided as a learning resource for the basic medical sciences, provision must be made for the style of teaching of the course and the style of learning of the students attending that course.     

Effect of ethnicity and gender on performance in undergraduate medical examinations

OBJECTIVE: To assess the effect of ethnicity and gender on medical student examination performance. DESIGN: Cohort study of Year 3 medical students in 2002 and 2003. SETTING: Royal Free and University College Medical School, Imperial College School of Medicine. SUBJECTS: A total of 1216 Year 3 medical students, of whom 528 were male and 688 female, and 737 were white European and 479 Asian. OUTCOME MEASURE: Performance in summative written and objective structured clinical examinations (OSCEs) in July 2002 and 2003. RESULTS: White females performed best in all OSCEs and in 3 out of 4 written examinations. Mean scores for each OSCE and 2 out of 4 written examinations were higher for white students than for Asian students. The overall size of the effect is relatively small, being around 1-2%. CONCLUSION: Students of Asian origin, of both genders, educated in the UK, using English as their first language, continue to perform less well in OSCEs and written assessments than their white European peers.     

Evaluating problem-based learning in a multilingual student population

OBJECTIVES: The University of Natal Medical School in South Africa provides training for a student body composed of two groups: one with English as a first language and the other with an African language as a first language and English as the second. A new methodology was developed to evaluate an innovative course using modified problem-based learning techniques in this heterogeneous environment. DESIGN: The learning model proposed required achieving a balance of three components: content, enquiry/learning process and social interaction/group process. A multidimensional system, felt to be consistent with this educational philosophy, was developed using seven different quantitative and qualitative techniques. SETTING: The University of Natal Medical School. SUBJECTS: First-year multilingual medical students. RESULTS: The results revealed that social interaction was highly successful in reducing barriers between the student groups and between students and facilitators. However, the emphasis on group participation may have overshadowed the enquiry process, leading to superficial discussions of problems and feelings of repetitiveness. During the course students and facilitators expressed concern that the innovative assessments used did not assess the course content adequately. While the group presentations and projects were useful exercises for consolidation and group interaction, they did not enable facilitators to identify struggling students. CONCLUSIONS: The outcome of the evaluation stressed the need of achieving an appropriate balance both in the curriculum and assessments of the three components of the learning model, particularly in a setting where student backgrounds and language ability differ. Multidimensional methodology is needed for effective evaluation that promotes critical reflection.     

Language background and communication skills of medical students

Objective: The increasing proportion of medical students whose primary language is other than English and recent reports of poor communication skills of medical graduates has generated community concern about the methods of selection of students and their communication skills training. This paper examines the relationship between language background and examination performance in oral communication skills in Year 5 medical students.

Method: Questionnaire data from all Year 5 students in the 1992 general practice terms were matched to examination results.

Results: Seventy percent of students responded. Most students whose primary language was not English passed, although some required remedial communication skills tuition. The most powerful predictors of poor performance were recent arrival in Australia and living in an environment where English was not spoken at home.

Conclusion: Students with poor English oral communication skills should be encouraged to speak English away from the medical school and should be offered additional tuition so that their skills in other languages are not lost to the health‐care system.     

A comparison of performance by medical and midwifery students in multiprofessional teaching

AIM: At Dundee University, midwifery and medical students are taught obstetrics together in a 2-week intensive course. We set out to test the hypothesis that staff time and effort could be saved by using shared resources in teaching a multidisciplinary group of students to an acceptable level. METHOD: In order to measure the knowledge gain by two different groups of students, we tested the students before and after a timetabled computer-assisted learning (CAL) session focusing on how to interpret a cardiotocograph (CTG). Also, half of each student group was given extra CTG teaching before the CAL session. RESULTS: The medical students (n=38) increased their median score from 9 to 17 after the CAL (P<0.001) but the midwifery students (n=13) only increased their median score from 12 to 14 after the CAL (n.s.). However, when given a tutorial and CAL, the post-test scores for both medical and midwifery students were similar and significantly higher than pre-test scores (median score increase from 8.5 to 18 for medical students, P<0.001, n=34, and from 9 to 16 for midwifery students, P<0.01 n=11). There was no significant knowledge gain by the medical students who undertook the additional tutorial. CONCLUSION: We conclude that shared resources could be used by medical and midwifery students to reach equivalent levels of skill in CTG interpretation. However, in order to achieve equivalence, staff time and effort was wasted as medical students were given unnecessary tuition.     

Assessment of English-language proficiency for general practitioner registrars

INTRODUCTION: English-language proficiency of medical practitioners is an issue attracting increasing attention in medical education. To best provide language education support, it is essential that learning needs are assessed and that useful feedback and advice are provided. We report the outcomes of a language assessment that was embedded within the context of a comprehensive general practice learning-needs analysis. METHODS: A group of general practitioner registrars (N = 18) training in Adelaide, South Australia, participated in the learning-needs analysis. The analysis used reliable, validated rating scales that provided information on both verbal and written language skills. These scales were used in the context of an objective structured clinical interview. The interviews were videotaped to enable multiple ratings per candidate. Following the learning-needs analysis, ratings were collated and fed back individually to participants according to a feedback report and template. RESULTS: Of this sample, 5 (28%) were found to have no need for any assistance with either spoken or written language, 5 had poor handwriting, 5 were considered to have minor difficulties, and 3 (17%) were identified as having substantial spoken and written English-language difficulties. These outcomes allowed medical educators to focus the language education support offered to the general practitioner registrars appropriately. CONCLUSIONS: Language skills can be usefully assessed within a more comprehensive learning-needs analysis. In combination with this assessment, the provision of specific feedback and recommendations for appropriate language-learning opportunities is essential.     

Early identification of 'at-risk' students by the parents of paediatric patients

INTRODUCTION: Assessment of medical student clinical skills is best carried out using multiple assessment methods. A programme was developed to obtain parent evaluations of medical student paediatric interview skills for feedback and to identify students at risk of poor performance in summative assessments. METHOD: A total of 130 parent evaluations were obtained for 67 students (parent participation 72%, student participation 58%). Parents completed a 13-item questionnaire [Interpersonal Skills Rating Scale (IPS) maximum score 91, higher scores = higher student skill level]. Students received their individual parent scores and de-identified class mean scores as feedback, and participants were surveyed regarding the programme. Parent evaluation scores were compared with student performance in formative and summative faculty assessments of clinical interview skills. RESULTS: Parents supported the programme and participating students valued parent feedback. Students with a parent score that was less than 1 standard deviation (SD) below the class mean (low IPS score students) obtained lower faculty summative assessment scores than did other students (mean +/- SD, 59% +/- 5 versus 64% +/- 7; P < 0.05). Obtaining 1 low IPS score was associated with a subsequent faculty summative assessment score below the class mean (sensitivity 0.38, specificity 0.88). Parent evaluations combined with faculty formative assessments identified 50% of students who subsequently performed below the class mean in summative assessments. CONCLUSIONS: Parent evaluations provided useful feedback to students and identified 1 group of students at increased risk of weaker performance in summative assessments. They could be combined with other methods of formative assessment to enhance screening procedures for clinically weak students.     

An evaluation study of the didactic quality of clerkships

CONTEXT: Previous qualitative research at the University of Antwerp revealed dissatisfaction amongst medical students about clinical clerkships. OBJECTIVE: To use quantitative methods to further explore student perceptions of their clinical teaching. METHOD, SAMPLE AND SETTING: Monthly questionnaire administered to final-year medical students at the University of Antwerp, Belgium. RESULTS: The response rate was 83.9%. Many of the qualitative findings were reproduced. Educational resources were not optimally used. The junior doctor was the most important clinical teacher. Many activities were passive experiences. Coaching, feedback and supervision associated most with general satisfaction of the clerkship, however, these dimensions were often considered suboptimal. CONCLUSIONS: Clinical clerkships do not automatically provide an ideal learning environment for medical students.     

Medical students as standardized patients to assess interviewing skills for pain evaluation

PURPOSE: The use of medical students as standardized patients in a performance assessment of pain evaluation was studied. METHODS: Fifty-two pairs of second-year medical students participated. One student portrayed a patient presenting with cancer pain and was interviewed by the other medical student. The student-patient then rated the interview using a checklist of pain assessment and general interviewing skills. The interviews were audiotaped and also rated independently. RESULTS: Based on student-patient ratings, 36 (69%) students demonstrated 9 or more of the 11 pain-specific checklist items, compared to 34 (65%) students according to the trained rater. Highly specific pain-related items had higher agreement than broader interviewing skill items. There would be differences in the summary assessments of students depending on which rating data were used. DISCUSSION: Medical students represent a readily accessible resource as patients for clinical simulations. Students tended to overestimate the performance of fellow students, but acting as a standardized patient had educational value, and can be used to extend simulated patient encounters within the curriculum. Further investigation is needed to improve the reliability of the feedback provided by student-patients.     

Identifying the factors that determine feedback given to undergraduate medical students following formative mini-CEX assessments

OBJECTIVE: We aimed to examine the factors that determine provision of feedback to students following mini-clinical evaluation exercise (mini-CEX) assessments. METHODS: We carried out a pilot study of all final year medical students at the University of Aberdeen between November 2005 and June 2006. The study involved 396 mini-CEX encounters (173 students and 105 assessors). We retrospectively analysed the components of feedback recorded on the marking sheets. Each component of feedback was coded as a binary response (feedback recorded or not). Logistic regression was used to determine the degree and significance of the factors that influenced feedback. Specialty block, range of marks, assessor group and assessor satisfaction were entered into the analysis using SPSS Version 14. RESULTS: The provision of feedback by our assessors was poor. In 22.7% of cases, positive features were not identified; in 28.2% of cases, no suggestions for development were highlighted; in 49.7% of cases, no action plan was formulated. Assessors who gave a wider range of scores for the specific skill domains were more likely to record areas for development and action plans. Specialty block influenced feedback on areas for development. Suggesting an action plan was significantly associated with assessor group, and academic trainees were the most consistent providers of feedback. Assessor satisfaction was significantly associated with highlighting 'anything especially good'. CONCLUSIONS: Numerous factors were associated with the provision of feedback in our cohort. Assessor training may address this variability. However, this would limit the diversity of assessors, which may be undesirable. More research is required on assessor training and recruitment.     

Standardized patients in the early acquisition of clinical skills

OBJECTIVES: Facing dramatic reductions of inpatient beds and fewer faculty tutors, the Queen's University medical school has had to consider alternatives to the traditional inpatient encounter for the early acquisition of clinical skills. The purpose of this study was to investigate the feasibility and impact on students of a model for a first-year clinical skills course, using exclusively standardized patients in place of inpatients, and using a smaller ratio of faculty to students. DESIGN: Twenty volunteers were selected from the first-year class of 75 students to participate as the experimental group. The remaining 55 students formed the control group. SETTING: Queen's University medical school, Ontario, Canada. SUBJECTS: First-year medical students. RESULTS: In measures of student satisfaction with the amount of performance feedback received during the course, there was a non-significant trend towards greater satisfaction among the experimental group compared to the control group. This occurred despite a tutor to student ratio of 1.5:10 in the experimental group compared to 3:10 in the control group. In the student evaluation of their tutors, mean scores for the two groups were 4.28 for the experimental group and 4. 06 for the control group (P = 0.10). The mean OSCE scores for the two groups were 76.4 for the experimental group and 76.5 for the control group (P = 0.93). There is no practical or statistical difference in either of these two scores. CONCLUSIONS: We conclude that this new model is feasible, was well received by the students and that the reduced tutor to student ratio and lack of inpatient encounters did not appear to adversely affect their learning.