Attending rounds: What do the all-star teachers do?

Aim: To examine differences in the types of teaching activities performed during rounds between the most effective and least effective inpatient teaching attendings.

Methods: Participants included 56 attending physicians supervising 279 trainees. Trained observers accompanied teams during rounds and recorded the frequencies of educational activities that occurred. Students and residents then rated their satisfaction with the teaching on rounds.

Results: Attending physicians with the highest learner satisfaction scores performed significantly more teaching activities per patient than attending physicians who were rated as average or less-effective (2.1 vs. 1.4 vs. 1.5; p?=?.03). There were significant differences in the frequencies of 3 out of the 9 specific teaching activities observed, including answering specific patient-care related questions (77% vs. 66% vs. 47%; p?=?.003), teaching on learner chosen topics (8% vs. 2% vs. 2%; p?=?.02), and providing feedback (31% vs. 10% vs. 0.1%; p?=?.001).

Conclusions: Specific categories of teaching activities—patient-specific teaching, teaching on learner-identified topics, and providing real-time feedback—are performed more frequently by the highest-rated attending physicians, which can guide faculty development.     

Scholar outcomes for dental internship research program in Saudi Arabia: A qualitative evaluation

Abstract

Objective: To explore the potential, challenges and needs for internship research activities in achieving scholar outcomes among graduates.

Methods: A qualitative general needs assessment and evaluation of an internship research program was conducted at King Abdulaziz University, Faculty of Dentistry (KAUFD), KSA, from December 2014 to February 2015 using focus groups and interviews. The participants included: administrates, faculty, and internship students. Data were transcribed and analyzed following the grounded theory.

Results: The participants were two administrative personnel, 21 faculty members, and 16 internship students. Results were clustered around five main domains; curriculum design, faculty, students, administrative, and institutional domain. Reported potentials included: a multi-faceted educational intervention approach, and building evidence-based skills and inquiry minds among graduates. Time, load, and incentives were major challenges reported by faculty. Interesting and achievable research topics were major challenges reported by students. Areas that needed development included: equipped research personnel, aligned administrative and institutional support, faculty skills, students’ knowledge and skills, aligned curriculum, and clear program goals, objectives, and outcomes.

Conclusion: Curriculum design, faculty and students’ skills; as well as administrative and institutional support were found to play major roles in the success of the current internship research program at KAUFD.     

Sure,or unsure? Measuring students’ confidence and the potential impact on patient safety in multiple-choice questions

Background: Multiple-choice questions (MCQs) provide useful information about correct and incorrect answers, but they do not offer information about students’ confidence.

Methods: Ninety and another 81 medical students participated each in a curricular neurology multiple-choice exam and indicated their confidence for every single MCQ. Each MCQ had a defined level of potential clinical impact on patient safety (uncritical, risky, harmful). Our first objective was to detect informed (IF), guessed (GU), misinformed (MI), and uninformed (UI) answers. Further, we evaluated whether there were significant differences for confidence at correct and incorrect answers. Then, we explored if clinical impact had a significant influence on students’ confidence.

Results: There were 1818 IF, 635 GU, 71?MI, and 176 UI answers in exam I and 1453 IF, 613 GU, 92?MI, and 191 UI answers in exam II. Students’ confidence was significantly higher for correct than for incorrect answers at both exams (p?p?=?0.01). At exam II, students’ confidence was significantly higher for incorrect harmful than for incorrect benign (p?p?=?0.01).

Conclusions: We were pleased to see that there were more informed than guessed, more uninformed than misinformed answers and higher students’ confidence for correct than for incorrect answers. Our expectation that students state higher confidence in correct and harmful and lower confidence in incorrect and harmful MCQs could not be confirmed.     

Simulation fails to replicate stress in trainees performing a technical procedure in the clinical environment

Introduction: Simulation-based training (SBT) has become an increasingly important method by which doctors learn. Stress has an impact upon learning, performance, technical, and non-technical skills. However, there are currently no studies that compare stress in the clinical and simulated environment. We aimed to compare objective (heart rate variability, HRV) and subjective (state trait anxiety inventory, STAI) measures of stress theatre with a simulated environment.

Methods: HRV recordings were obtained from eight anesthetic trainees performing an uncomplicated rapid sequence induction at pre-determined procedural steps using a wireless Polar RS800CX monitor © in an emergency theatre setting. This was repeated in the simulated environment. Participants completed an STAI before and after the procedure.

Results: Eight trainees completed the study. The theatre environment caused an increase in objective stress vs baseline (p?=?.004). There was no significant difference between average objective stress levels across all time points (p?=?.20) between environments. However, there was a significant interaction between the variables of objective stress and environment (p?=?.045). There was no significant difference in subjective stress (p?=?.27) between environments.

Discussion: Simulation was unable to accurately replicate the stress of the technical procedure. This is the first study that compares the stress during SBT with the theatre environment and has implications for the assessment of simulated environments for use in examinations, rating of technical and non-technical skills, and stress management training.     

Faculty experience and engagement in a longitudinal integrated clerkship

The authors sought to understand rewards and challenges of teaching third-year medical students in the University of Colorado School of Medicine (CUSOM) Denver Health Longitudinal Integrated Clerkship (DH-LIC) compared to teaching in rotation-based clerkships (RBCs). The authors considered implications for the recruitment and retention of faculty in clinical educational programs. Preceptors completed surveys at baseline and year-end. Of eligible faculty, 28 of 40 completed both baseline and year-end surveys. The majority (85.2%) of faculty were satisfied with the DH-LIC and 85.7% continued to teach in year-two of the program. Faculty reported increased satisfaction from teaching and improved teaching and mentoring skills. Faculty familiarity with DH-LIC students was significantly higher than with students previously taught (p?=?.004); 89.3% of faculty knew their DH-LIC student well enough to tailor instruction to individual learning needs. Teaching techniques utilized at baseline and end of year differed significantly; faculty reported asking questions to promote thinking, providing feedback to students, and providing students with practice in clinical reasoning more frequently in the DH-LIC. Innovative models of education such as LICs offer a strategy to recruit and retain excellent, invested faculty in outpatient settings.     

The association between Situational Judgement Test (SJT) scores and professionalism concerns in undergraduate medical education

Abstract

Introduction: Situational judgement tests (SJTs) are widely used in selecting medical students and doctors. Emerging evidence suggests SJTs are capable of testing an individual’s ability to respond to role-relevant professionalism scenarios, however, evidence is lacking for their use in identifying students with concerning professional behaviours.

This study aimed to determine the association between medical student professionalism-based SJT scores and recorded professionalism concerns during training.

Methods: A cross-sectional study was conducted utilising SJT scores from second-year medical students and occurrences of student professionalism concerns. Concerns were reviewed and mapped to General Medical Council standards. Multivariate logistic regression was used to determine associations between SJT scores and professionalism concerns.

Results: 247 students were included in the study. For every point increase in SJT score, students were 10% less likely to have multiple professionalism concerns [OR (95% CI) 0.90 (0.83–0.97); p?=?.007].

Students scoring below 1 and 2 standard deviations from the mean score were 4 and 11-times more likely to have multiple concerns [OR (95% CI) 4.52 (1.12–18.25); p?=?.034] and [OR (95% CI) 11.45 (1.72–76.15); p?=?.012].

Conclusion: Lower SJT scores were significantly associated with an increased risk of professionalism concerns. These findings support the potential for SJT exams to identify medical students that may require closer supervision and remediation during undergraduate education.     

Comparison of faculty assessment and students’ self-assessment of performance during clinical case discussions in a pharmacotherapy capstone course

Objectives: The primary objective of this study was to compare faculty assessment and third year students' self-assessment of performance in clinical case discussions. The secondary objective was to evaluate if student characteristics influence self-assessments.

Methods: This retrospective analysis compared faculty and student self-assessment scores for two clinical case discussions using Spearman’s correlation and Wilcoxon’s signed ranks test. Chi-squared test was used to compare frequency of faculty and student self-assessments indicating the highest possible rating for the pooled score and for each individual component. The pooled score included three individual components: level of engagement, quality of contribution, and professionalism.

Results: Pooled faculty and student self-assessments correlated for both the first (r?=?0.41, p?r?=?0.35; p?p?=?0.25) and second (58.6% vs. 47.4%, p?=?0.05) clinical case discussions. Student characteristics (age, gender, and grade point average at graduation) did not influence self-assessments.

Conclusions: Students’ self-assessment correlated with faculty assessment of performance during clinical case discussions. Increased use of self-assessments for professional development in pharmacy and other healthcare professional curricula should be considered.     

Faculty development: From rubies to oak

Abstract

Introduction: Faculty development has played a significant role in health professions education over the last 40?years. The goal of this perspective is to present a portrait of faculty development in Medical Teacher since its inception and to highlight emerging trends moving forward.

Method: All issues of Medical Teacher were reviewed, using the search terms faculty development, staff development, professional development, or in-service training for faculty. The search yielded 286 results of which 145 focused specifically on faculty development initiatives, reviews, or frameworks.

Findings: This review demonstrated a significant growth in publications related to faculty development in Medical Teacher over the last 40?years, with a primary focus on teaching improvement and traditional approaches to faculty development, including workshops, short courses and other structured, group activities. The international nature of faculty development was also highlighted.

Recommendations: Moving forward, it is suggested that we: broaden the scope of faculty development from teaching to academic development; expand our approaches to faculty development, to include peer coaching, workplace learning and communities of practice; utilize a competency-based framework to guide the development of faculty development curricula; support teachers’ professional identities through faculty development; focus on organizational development and change; and rigorously promote research and scholarship in faculty development.     

Taking stock of what is known about faculty development in competency-based medical education: A scoping review paper

Abstract

Purpose: The primary objective was to inventory what is currently known about faculty development (FD) for competency-based medical educations (CBME) and identify gaps in the literature.

Methods: A scoping review methodology was employed. Inclusion criteria for article selection were established with two reviewers completing a full-text analysis. Quality checks were included, along with iterative consultation on data collection and consensus decision making via a grounded theory approach.

Results: The review identified 19 articles published between 2009 and 2018. Most articles (N?=?15) offered suggestions as to what should happen with FD in CBME, but few (N?=?4) adopted an experimental design. Six main themes were identified with three main features of FD noted across themes: (1) The importance of direct and timely feedback to faculty members on their teaching and assessment skills. (2) The role of establishing shared mental models for CBME curricula. (3) That FD is thought of longitudinally, not as a one-time bolus.

Conclusion: This work illustrates that there is limited, high quality research in FD for CBME. Future FD activities should consider employing a longitudinal and multi-modal program format that includes feedback for the faculty participants on their teaching and assessments skills, including the development of faculty coaching skills.     

Does hearing the patient perspective improve consultation skills in examinations? An exploratory randomized controlled trial in medical undergraduate education

Purpose: Medical education increasingly includes patient perspectives, but few studies look at the impact on students’ proficiency in standard examinations. We explored students’ exam performance after viewing video of patients’ experiences.

Methods: Eighty-eight medical students were randomized to one of two e-learning modules. The experimental group saw video clips of patients describing their colposcopy, while the control group viewed a clinician describing the procedure. Students then completed a Multiple Choice Questionnaire (MCQ) and were assessed by a blinded clinical examiner in an Objective Structured Clinical Examination (OSCE) with a blinded simulated patient (SP). The SP scored students using the Doctors’ Interpersonal Skills Questionnaire (DISQ). Students rated the module’s effect on their skills and confidence. Regression analyses were used to compare the effect of the two modules on these outcomes, adjusting for gender and graduate entry.

Results: The experimental group performed better in the OSCE than the control group (odds ratio 2.7 [95%CI 1.2–6.1]; p?=?0.016). They also reported significantly more confidence in key areas, including comfort with patients’ emotions (odds ratio 6.4 [95%CI 2.7–14.9]; p?<?0.0005). There were no other significant differences.

Conclusion: Teaching that included recorded elements of real patient experience significantly improved students’ examination performance and confidence.     

Peer review improves psychometric characteristics of multiple choice questions

Abstract

Purpose: For new and emerging medical schools, developing a system to peer-review and evaluate the assessment processes through faculty development programs can be a challenge. This study evaluates the impact of peer-review practices on item analysis, reliability, and the standard error of measurement of multiple-choice questions for summative final examinations.

Methods: This study used a retrospective cohort design of two consecutive academic years in 2012 and in 2013. Psychometric analyses of multiple-choice questions of three summative final examinations in Medicine, Pediatrics, and Surgery for sixth year medical students at the College of Medicine Taif University were used. Formal peer review of multiple-choice questions began in 2013, using guidelines from the National Board of Medical Examiners. Psychometric analyses of multiple-choice questions included item analysis (item difficulty and item discrimination) and calculation of internal-consistency reliability and the standard error of measurement. Data analyses were conducted using Stata.

Results: Results showed significant improvement in psychometric indices, particularly item discrimination and reliability by .14 and .12 points, respectively, following the implementation of the peer review process across the three exams. Item difficulty remained unchanged for Pediatrics and Surgery.

Conclusion: Peer-review practices of multiple-choice questions using guidelines can lead to improved psychometric characteristics of items; these findings have implications for faculty development programs in improving item quality, particularly for medical schools in early stages of transforming assessment practices.     

Large-scale multi-site OSCEs for national competency examination of medical doctors in Indonesia

Background: Implementing large-scale multi-site objective structured clinical examination (OSCEs) for national competency examination in a low resource country is challenging.

Aims: To describe the first national OSCE for national competency examination of medical doctors in Indonesia and evaluate the reliability, validity, feasibility, acceptability, and educational impact.

Methods: We collected electronically the OSCE scores from 49 out of 73 medical schools that participated to assess reliability and validity. We conducted electronic survey to examiners, examinees, SP trainers, and OSCE coordinators to assess feasibility, acceptability, and educational impact.

Results: The Cronbach’s alpha coefficient across station was 0.79. There was strong correlation between rubric and global rating scores in each station (coefficient correlation ranges from 0.705 to 0.82). The content validity ratio was 0.97. The coefficient correlation between OSCE and MCQ was 0.335 (p?=?0.00). All 49 medical schools were able to conduct OSCE simultaneously. Examiners, examinees, SP trainers, and OSCE coordinators had good perception regarding feasibility and acceptability of OSCE. Both examiners and examinees indicated good educational impact of OSCE application. The cutting score based on the borderline regression method was 61.96%. There were 67.39% of the examinees achieved similar or above the cutting score.

Conclusion: With 12 stations 15?min each, the reliability coefficient across station is intermediate. Content validity is good. It is feasible and acceptable to implement large-scale multi-site OSCEs in Indonesia. Examiners and examinees perceive good educational impact on OSCE implementation.     

Evidence-based medicine (EBM) for undergraduate medical students: A six-step,integrative approach

Abstract

Background: Nowadays, there is an increasing interest in practicing evidence-based medicine (EBM) to provide valid and up-to-date information about health problems and procedures for solving these problems. However, studying EBM among other medical education disciplines remains unsatisfactory.

Aim: To design and pilot a basic course on EBM for undergraduate medical students in order to raise the awareness of the Saudi medical education community about this discipline.

Subjects and methods: The developed course targeted undergraduate Year 4 medical students at Ibn Sina National College for Medical Studies in Jeddah, Saudi Arabia. The course was integrated longitudinally into ?Integrated Multisystem Module? of Year 4. Students and faculty perceptions were evaluated for program evaluation purposes. Course design was based on Kern et al. six-step approach for curriculum development.

Results: Students overall perception of this course was positive except for some points related to the time allocated and their training on using evidence databases. Faculty members who participated in implementing the course perceived it positively.

Conclusions: Teaching EBM is recommended and applicable. It is not resource-intensive, and can be conducted through developing and integrating EBM courses in the undergraduate medical curriculum. Students and teachers agree upon the need and importance of teaching such discipline.     

Cutoff score evaluation of undergraduate dental curriculum

Abstract

Background and objectives: Exams at the Faculty of Dentistry (KAUFD) are usually constructed to match King Abdulaziz University (KAU) policy of a fixed 60% cutoff score, though they have never been tested or evaluated. The purpose of this study was to validate the cutoff scores of three final fifth-year written exams of the undergraduate Endodontic course to assess whether they were similar to KAU regulation using the Angoff rating method.

Design and settings: This study was conducted between May 2014 and February 2015 at the Faculty of Dentistry, King Abdulaziz University.

Methods: Using the Angoff rating method, three final fifth-year undergraduate Endodontic written exams were evaluated by four senior faculty members.

Results: The cutoff scores for exams 1, 2 and 3 were 57.4%, 62.9% and 63.1%, respectively. Adjusting the exams’ cut off scores would cause changes in some students’ results.

Conclusions: Although the cutoff scores for all exams were close to 60%, slight deviation from the accepted cutoff score could definitely affect the students’ results. Therefore, all exams should be validated before being given to students to certify that the cutoff score is credible and defensible.     

Clinical teaching performance improvement of faculty in residency training: A prospective cohort study

Purpose: The purpose of this study is to investigate how aspects of a teaching performance evaluation system may affect faculty’s teaching performance improvement as perceived by residents over time.

Methods: Prospective multicenter cohort study conducted in The Netherlands between 1 September 2008 and 1 February 2013. Nine hundred and one residents and 1068 faculty of 65 teaching programs in 16 hospitals were invited to annually (self-) evaluate teaching performance using the validated, specialty-specific System for Evaluation of Teaching Qualities (SETQ). We used multivariable adjusted generalized estimating equations to analyze the effects of (i) residents’ numerical feedback, (ii) narrative feedback, and (iii) faculty’s participation in self-evaluation on residents’ perception of faculty’s teaching performance improvement.

Results: The average response rate over three years was 69% for faculty and 81% for residents. Higher numerical feedback scores were associated with residents rating faculty as having improved their teaching performance one year following the first measurement (regression coefficient, b: 0.077; 95% CI: 0.002–0.151; p?=?0.045), but not after the second wave of receiving feedback and evaluating improvement. Receiving more suggestions for improvement was associated with improved teaching performance in subsequent years.

Conclusions: Evaluation systems on clinical teaching performance appear helpful in enhancing teaching performance in residency training programs. High performing teachers also appear to improve in the perception of the residents.     

Peer-assisted learning in a student-run free clinic project increases clinical competence

Background: Germany's first student-run free clinic (SRFC) for medically underserved patients has been established at the medical faculty of the Goethe-University, Frankfurt/Germany. Participating students are educated in a Peer Assisted Learning program (PAL). Little is known about the effectiveness of PAL in SRFCs.

Methods: We conducted a randomized, controlled, prospective study involving 50 participants. Students were either tested before or after receiving PAL. Knowledge and skill level were measured by theoretical and practical tests. In addition, curricular Objective Structured Clinical Examination (OSCE) results were compared between the groups.

Results: Students receiving PAL had significantly better results in theoretical (p?<?0.001) and practical (p?<?0.001) tests, as well as in the OSCE (p?<?0.01). A control test showed no significant difference (p?=?0.205) indicating similar prerequisites between the groups.

Conclusion: Improved results of the study group indicate an increase in the clinical knowledge and skills. PAL appears to be suitable for the training of basic medical skills and family medicine related knowledge and similar teaching projects could be based on it at other SRFCs.     

An explanatory mixed methods study on the validity and validation of students’ assessment results in the undergraduate surgery course

Abstract

Background/purpose: There is inadequate evidence of reported validity of the results of assessment instruments used to assess clinical competence. This study aimed at combining multiple lines of quantitative and qualitative evidence to support interpretation and use of assessment results.

Method: This study is a mixed methods explanatory research set in two stages of data collection and analysis (QUAN : qual). Guided by Messick’s conceptual model, quantitative evidences as reliability and correlation coefficients of various validity components were calculated using students’ scores, grades and success rates of the whole population of students in 2012/2013 and 2013/2014 (n=?383; 326). The underlying values that scaffold validity evidences were identified via Focus Group Discussions (FGD) with faculty and students; sampling technique was purposive; and results were analyzed by content analysis.

Results: (1) Themes that resulted from content analysis aligned with quantitative evidences. (2) Assessment results showed: (a) content validity (table of specifications and blueprinting in another study); (b) consequential validity (positive unintended consequences resulted from new assessment approach); (c) relationships to other variables [a statistically significant correlation among various assessment methods; with combined score (0.64–0.86) and between mid and final exam results (r?=?0.672)]; (d) internal consistency (high reliability of MCQ and OSCE: 0.81, 0.80); (3) success rates and grades distribution alone could not provide evidence to advocate an argument on validity of results.

Conclusion: The unified approach pursued in this study created a strong evidential basis for meaningful interpretation of assessment scores that could be applied in clinical assessments.