Exploring how students think: a new method combining think‐aloud and concept mapping protocols

Medical Education 2010: 44 : 926–935 Objectives A key element of medical competence is problem solving. Previous work has shown that doctors use inductive reasoning to progress from facts to hypotheses and deductive reasoning to move from hypotheses to the gathering of confirmatory information. No individual assessment method has been designed to quantify the use of inductive and deductive procedures within clinical reasoning. The aim of this study was to explore the feasibility and reliability of a new method which allows for the rapid identification of the style (inductive or deductive) of clinical reasoning in medical students and experts. Methods The study included four groups of four participants. These comprised groups of medical students in Years 3, 4 and 5 and a group of specialists in internal medicine, all at a medical school with a 6‐year curriculum in France. Participants were asked to solve four clinical problems by thinking aloud. The thinking expressed aloud was immediately transcribed into concept maps by one or two ‘writers’ trained to distinguish inductive and deductive links. Reliability was assessed by estimating the inter‐writer correlation. The calculated rate of inductive reasoning, the richness score and the rate of exhaustiveness of reasoning were compared according to the level of expertise of the individual and the type of clinical problem. Results The total number of maps drawn amounted to 32 for students in Year 4, 32 for students in Year 5, 16 for students in Year 3 and 16 for experts. A positive correlation was found between writers (R = 0.66–0.93). Richness scores and rates of exhaustiveness of reasoning did not differ according to expertise level. The rate of inductive reasoning varied as expected according to the nature of the clinical problem and was lower in experts (41% versus 67%). Conclusions This new method showed good reliability and may be a promising tool for the assessment of medical problem‐solving skills, giving teachers a means of diagnosing how their students think when they are confronted with clinical problems.     

A medical ethical reasoning model and its contributions to medical education

Medical Education 2010: 44 : 864–873 Objectives Ethical reasoning in medicine is not well understood and medical educators often find it difficult to justify what and how they teach and assess in medical ethics. To facilitate the development of moral values and professional conduct, a model of ethical reasoning was created. The purposes of this paper are to describe the ethical reasoning model and to indicate how it can be used to foster moral and ethical behaviours. Methods The ethical reasoning model was created from information derived from two sources: (i) an examination of different ethical models described in the literature, and (ii) think‐aloud interviews with ethical experts in Taiwan and Canada. All the components and cognitive steps used by experts in ethical decision making were extracted and categorised. Interview subjects consisted of 16 voluntary ethics experts. The ethical reasoning models reported in the literature were divided into two groups according to whether they were justification‐based or task‐based models. Neither of the two types represented the ‘whole picture’ of ethical reasoning in medicine. This analysis enabled us to identify five universal cognitive steps and the gaps between ‘logical decision’ and ‘action’. Results The think‐aloud interviews verified the multi‐dimensional components or steps used by experts when resolving ethical problems. The resulting model, designated the Medical Ethical Reasoning (MER) Model, reflects interactions within three domains: medical and ethical knowledge; cognitive reasoning processes, and attitude. Conclusions The MER Model accurately reflects how doctors resolve ethical dilemmas and is seen to be helpful in identifying what and how educators should teach and assess in ethical reasoning. The model can also serve as a communication framework for curricular design. A ‘humane’ doctor is competent in providing quality, ethical patient care. Making an appropriate ethical decision is the foundation for subsequent ethical behaviours. By contrast with the abundant evidence cited in previous research describing how doctors solve medical problems, there is little empirical evidence indicating how doctors make appropriate ethical decisions. Thus, the cognition of ethical reasoning in medicine is not well understood. This paper represents a step towards overcoming this problem.     

Do extended matching multiple-choice questions measure clinical reasoning?

PURPOSE: At the Faculty of Medicine at the Katholieke Universiteit Leuven, Belgium, we have developed a final examination that consists of extended matching multiple-choice questions. Extended matching questions (EMQs) originate from a case and have 1 correct answer within a list of at least 7 alternatives. If EMQs assess clinical reasoning, we can assume there will be a difference between the ways students and experienced doctors solve the problems within the questions. This study compared students' and residents' processes of solving EMQs. METHODS: Twenty final year students and 20 fourth or fifth year residents specialising in internal medicine solved 20 EMQs aloud. All questions concerned diagnosis or pathogenesis. Ten EMQs related to internal medicine and 10 questions to other medical disciplines. The session was audio-taped and transcribed. RESULTS: The residents correctly answered significantly more questions concerning internal medicine than did the students. Their reasoning was more "forward" and less "backward". No difference between residents and students was found for the other questions. The residents scored better on internal medicine than on the other questions. They used more backward and less forward reasoning when solving the other questions than they did with the internal medicine questions. The better half of the respondents used significantly more forward and less backward reasoning than did the poorer half. CONCLUSION: In accordance with the literature, medical expertise was characterised by forward reasoning, whereas outside their area of expertise, the subjects switched over to backward reasoning. It is possible to assess processes of clinical reasoning using EMQs.     

A cross-cultural study of students' approaches to professional dilemmas: sticks or ripples

Medical Education 2012: 46 : 245–256 Context Medical educators internationally are faced with the challenge of teaching and assessing professionalism in their students. Some studies have drawn attention to contextual factors that influence students’ responses to professional dilemmas. Although culture is a significant contextual factor, no research has examined student responses to professional dilemmas across different cultures. Methods Semi‐structured interviews inquiring into reactions towards, and reasoning about, five video clips depicting students facing professional dilemmas were conducted with 24 final‐year medical students in Taiwan. The interviews were transcribed and analysed according to the theoretical framework used in prior Canadian studies using the same videos and interview questions. Results The framework from previous Canadian research, including the components of principles, affect and implications, was generally applicable to the decision making of Taiwanese students, with some distinctions. Taiwanese students cited a few more avowed principles. Taiwanese students emphasised an additional unavowed principle that pertained to following the advice of more senior trainees. In addition to implications for patients, team members or themselves, Taiwanese students considered the impact of their responses on multiple relationships, including those with patients’ families and alumni residents. Cultural norms were also cited by Taiwanese students. Conclusions Medical educators must acknowledge students’ reasoning in professionally challenging situations and guide students to balance considerations of principles, implications, affects and cultural norms. The prominence of Confucian relationalism in this study, exhibited by students’ considerations of the rippling effects of their behaviours on all their social relationships, calls for further cross‐cultural studies on medical professionalism to move the field beyond a Western individualist focus.     

Values and CPR decisions: a comparison of physicians and administrators in training.

A report of a study to analyze the effect of sociocultural patient characteristics (age, terminal diseases, drug abuse, alcoholism, mental retardation, dementia, suicide attempts, institutionalization, noncompliance with medical regimens, violent crimes, lack of support system, or relationship to a staff physician) on decisions to initiate or withhold cardiopulmonary resuscitation in an emergency situation. Pairs of vignettes were presented to residents in internal medicine and graduate students in an MBA program for comparisons of physicians' decisions with administrators' decisions. On some patient characteristics there were significant differences between the two groups. For most factors (drug abuse, multiple suicide attempts, age, violent crime, lack of known support systems, and relationship to staff), doctors are more likely to initiate CPR than are business students representing health care administrators. In chronic, long-term situations (carcinoma or heart disease, dementia, mental retardation, and institutionalization), the doctors are less likely to initiate CPR than the business students. If objectivity is a goal in deciding whether or not to initiate CPR, physicians should be aware of differences between their opinions and others'.     

Evaluando la enseñanza de la bioética: formando «médicos virtuosos» o solamente médicos con habilidades éticas prácticas

Objective

In the last decades, bioethics has been incorporated into the academic training of the Medical Schools. Some studies analyze the ethical-moral development of medical students and the effect of ethical education in other countries. This evaluation is done by measuring Kohlberg's moral reasoning (virtuous doctors), or ethical sensitivity to resolve clinical cases (physicians with ethical skills). The following study is proposed to assess the impact of bioethics training on these two variables, in Spanish medical students.

The multiple mini-interview for selection of international medical graduates into family medicine residency education

Context The multiple mini‐interview (MMI) was used to measure professionalism in international medical graduate (IMG) applicants for family medicine residency in Alberta for positions accessed through the Alberta International Medical Graduate (AIMG) Program. This paper assesses the evidence for the MMI’s reliability and validity in this context. Methods A group of 71 IMGs participated in our 12‐station MMI designed to assess professionalism competency. A 10‐point scale evaluated applicants on ability to address the objectives of the situation, interpersonal skills, suitability for a residency and for family medicine, and overall performance. We conducted generalisability and decision studies to assess the reliability of MMI scores. We assessed the validity by examining the differences in MMI scores associated with session, track and socio‐demographic characteristics of applicants and by measuring the correlations between MMI scores and scores on compulsory examinations, including the AIMG objective structured clinical examination, the Medical Council of Canada Evaluating Examination (MCCEE) and the Medical Council of Canada Qualifying Examination Part I (MCCQE I). We measured the correlation between MMI and non‐requisite MCCQE Part II (MCCQE II) scores that were provided. Results The reliability as indicated by the generalisability coefficient associated with average station scores was 0.70 with one interviewer per station. There were no statistically significant differences in total MMI scores or mean station sum scores based on session, track, applicant age, gender, years since medical school completion, or language of medical school. There were low, non‐significant correlations with OSCE overall (r = 0.15), MCCEE (r = 0.01) and MCCQE I (r = 0.06) scores and a higher non‐significant correlation with MCCQE II scores (r = 0.33). Conclusions There is evidence that the MMI offers a reliable and valid assessment of professionalism in IMG doctors applying for Canadian family medicine residencies and that this clinically situated MMI assessed facets of competency other than those assessed by the OSCE.     

Student and faculty assumptions about the nature of uncertainty in medicine and medical education

Clinical reasoning involves an element of uncertainty. Teaching clinical reasoning involves understanding how students view uncertainty as well as how medical problems are solved. This study uses Perry's model of intellectual development to explore changes in how medical students, residents, and instructors think about the nature of knowledge. A total of 31 medical students, residents, and instructors completed the Widick and Knefelkamp Measure of Intellectual Development revised to focus specifically on uncertainty in medicine. Consistent with Perry's theory, scores reflected increasing degrees of acceptance of the role of uncertainty in medicine with increasing experience. Based on these results, it is concluded that to improve the effectiveness of teaching problem solving in medicine, faculty must challenge the assumptions held by medical students about the certainty of medical knowledge while teaching the process of clinical diagnosis.     

Ethical reasoning and decision-making in the clinical setting: assessing the process

Clinical ethical reasoning and analysis are skills as central to good patient care as the efficient application of biomedical knowledge to diagnosis and prognosis. However, experience in teaching clinical ethics to senior medical students has indicated that simply trying to ‘apply’ the knowledge learnt about ethical theories, principles, concepts and rules in the clinical setting does not ensure ethical competence in clinical decision-making. In 1992, we developed and piloted a three-session programme that focused on a more systematic approach to the way students identified and attempted to manage ethical issues in their clinical practice. This programme was modified and improved in 1993 and further expanded in 1994. Our experience suggests that many students are now better able to bridge what has been called the ‘gap’ between the possession of ethical knowledge and its actual use in clinical decision-making. The remaining problem was assessment. How do you assess clinical ethical reasoning and decision-making? In the preclinical years of medical education, knowledge-based assessment tools, like the modified essay question (MEQ), provide a means for assessing the sensitivity of students to ethical issues. However, such tools permit neither an appraisal of how students actually make clinical ethical decisions, nor which factors students perceive as important in making an actual clinical decision. In order to make this type of appraisal, we developed a format for a written case report that facilitated our assessing the process as well as the end-product, the decision. The results obtained from the use of the written case report in the annual assessment of our senior medical students in 1993, while encouraging, also identified a number of unexpected problems.     

How student models of expertise and innovation impact the development of adaptive expertise in medicine

Objectives  The ability to innovate new solutions in response to daily workplace challenges is an important component of adaptive expertise. Exploring how to optimally develop this skill is therefore of paramount importance to education researchers. This is certainly no less true in health care, where optimal patient care is contingent on the continuous efforts of doctors and other health care workers to provide the best care to their patients through the development and incorporation of new knowledge. Medical education programmes must therefore foster the skills and attitudes necessary to engage future doctors in the systematic development of innovative problem solving. The aim of this paper is to describe the perceptions and experiences of medical students in their third and fourth years of training, and to explore their understanding of their development as adaptive experts.
Methods  A sample of 25 medical students participated in individual 45–60-minute semi-structured interviews. Interviews were audiotaped, transcribed and entered into NVivo qualitative data analysis software to facilitate a thematic analysis. The analysis was both inductive, in that themes were generated from the data, and deductive, in that our data were meaningful when interpreted in the context of theories of adaptive expertise.
Results  Participants expressed a general belief that, as learners in the health care system, exerting any effort to be innovative was beyond the scope of their responsibilities. Generally, students suggested that innovative practice was the prerogative of experts and an outcome of expert development centred on the acquisition of knowledge and experience.
Conclusions  Students' perceptions of themselves as having no responsibility to be innovative in their learning process have implications for their learning trajectories as adaptive experts.     

Aspects of expertise which are jointly valid for German sociomedicine and statutory health care

A project group of the Medical Advisory Board of the German Federal Rehabilitation Council (BAR) developed fundamental joint principles on experts' opinions according to the social law code no. IX (SGB IX). The principles aim at medical experts working in different social organisations and statutory health care insurance. It was intended to create a "sociomedical language" which should be used as jointly as possible by experts in rehabilitation and social medicine and which is based on the ICF (International Classification of Functioning, Disability and Health, WHO 2001). Its stringent application will increase the utility of medical expertise across different institutions. The authors recommend to evaluate whether this model could provide a tool in the communication and cooperation between different sectors of the health system. Part I describes the theoretical model, Part II its application to a virtual individual case history.     

Aspects of expertising which are jointly valid for German sociomedicine and statutory health care

A project group of the Medical Advisory Board of the German Federal Rehabilitation Council (BAR) developed fundamental joint principles on experts' opinions according to the social law code no. IX (SGB IX). The principles aim at medical experts working in different social organisations and statutory health care insurances. It was intended to create a "sociomedical language" which should be used as jointly as possible by experts in rehabilitation and social medicine and which is based on the ICF (International Classification of Functioning, Disability and Health, WHO 2001). Its stringent application will increase the utility of medical expertise across different institutions. The authors recommend to evaluate whether this model could provide a tool in the communication and cooperation between different sectors of the health system. Part I describes the theoretical model, Part II its application to a virtual individual case history.     

Cognitive Aspects of Clinical Performance During Patient Workup: The Role of Medical Expertise

This paper examines the role of medical expertise in clinical reasoning, using a complete workup of a patient with an endocrine disorder. Endocrinologists, housestaff, and final year medical students were asked to develop the case. This consisted of history-taking, interpreting physical examination results, request for tests and their interpretations, and providing therapeutic and patient management plans, and an explanation of the pathophysiology underlying the problem. The subjects were also asked to provide explanations supporting their decisions during the patient workup. A variety of techniques deriving from cognitive psychology were used to analyze the data. The main concern was how expertise affected the building of relationship between the components of the workup. Experts formed integrated knowledge-rich structures that were generated during the history-taking and used consistently throughout the workup. Housestaff formed more tentative and less integrated representations which were modified during the patient encounter. Students representations superficially resembled those of experts, but were knowledge-lean. The results are interpreted in terms of clinical performance of endocrinologists, housestaff, and students during the workup, and its relationship to the explanation task as an indicator of clinical competence of the patient problem provided by the subjects. It is proposed that the explanation given after the problem solving process could be used as an indicator of clinical competence. This revised version was published online in June 2006 with corrections to the Cover Date.     

Checklists improve experts’ diagnostic decisions

Context Checklists are commonly proposed tools to reduce error. However, when applied by experts, checklists have the potential to increase cognitive load and result in ‘expertise reversal’. One potential solution is to use checklists in the verification stage, rather than in the initial interpretation stage of diagnostic decisions. This may avoid expertise reversal by preserving the experts’ initial approach. Whether checklist use during the verification stage of diagnostic decision making improves experts’ diagnostic decisions is unknown. Methods Fifteen experts interpreted 18 electrocardiograms (ECGs) in four different conditions: undirected interpretation; verification without a checklist; verification with a checklist, and interpretation combined with verification with a checklist. Outcomes included the number of errors, cognitive load, interpretation time and interpretation length. Outcomes were compared in two analyses: (i) a comparison of verification conditions with and without a checklist, and (ii) a comparison of all four conditions. Standardised scores for each outcome were used to calculate the efficiency of a checklist and to weigh its relative benefit against its relative cost in terms of cognitive load imposed, interpretation time and interpretation length. Results In both analyses, checklist use was found to reduce error (more errors were corrected in verification conditions with checklists [0.29 ± 0.77 versus 0.03 ± 0.61 errors per ECG], and fewer net errors occurred in all conditions with checklists [0.39 ± 1.14 versus 1.04 ± 1.49 errors per ECG]; p < 0.01 for both). Checklists were not associated with increased cognitive load (verifications with and without checklists: 3.7 ± 1.9 and 3.3 ± 2.0, respectively; conditions with and without checklists: 4.0 ± 1.8 versus 3.9 ± 2.0, respectively [p = not significant for both]). Checklists resulted in greater interpretation times and lengths (p < 0.01 for all). However, checklists were efficient in terms of the cognitive load invested, interpretation time and interpretation length (p < 0.01 for all). Conclusions Among ECG interpretation experts, checklist use during the verification stage of diagnostic decisions did not increase cognitive load or cause expertise reversal, but did reduce diagnostic error.     

Postgraduate education in medical ethics in Japan

The objective of this paper was to investigate what kind of postgraduate education in medical ethics medical residents in Japan receive and what they want for ethical education and guidelines. Sixteen teaching hospitals that provide a general internal medicine residency programme in Japan were used (145 medical residents working at the departments of general internal medicine). A total of 114 residents participated in our survey, yielding a response rate of 79%. Of these, 28% received education in medical ethics more than once a month; 24% were offered it only when ethical problems were involved in actual patient care; and 18% answered that opportunities were very rare and sporadic. A full 30% had received no education in medical ethics at all. Many residents (71%) learned medical ethics from individual supervising doctors. A majority of the residents had been taught about informed consent (79%) and doctor–patient relationships (54%); 46% had learned about the appropriateness of truth telling and of ethical decisions regarding withholding and withdrawing a life-sustaining treatment, respectively. A total of 85 residents (75%) wanted to have more comprehensive education in medical ethics, 13% could not decide, and 12% did not want it. Many (66%) thought that both doctors and ethical philosophers should jointly teach medical ethics in postgraduate residency programmes. The results suggest that many residents desire more comprehensive and interdisciplinary education in medical ethics and educators in Japan should aim to develop education programmes to meet these desires.     

Burnout and job stress among mongolian doctors and nurses

Due to the inherent demands of their profession, doctors and nurses are at great risk of suffering from burnout caused by job stress. This study examined the prevalence of burnout among doctors and nurses in Mongolia and identified the factors influencing their burnout. A self-administered questionnaire of 180 doctors (45.9%) and 212 nurses (54.1%) resulted in a response rate of 87%. Burnout was measured by the Copenhagen Burnout Inventory (CBI) in three scales: personal burnout, work-related burnout, and client-related burnout. Job stress was measured by the effort-reward imbalance (ERI) model. Compared with the prior studies of hospital staffs in other countries, doctors and nurses in Mongolia had relatively higher burnout rates, with personal, work-related and client-related average scores of 45.39, 44.45, and 32.46, respectively. Multiple regression analysis revealed that ERI significantly influenced all dimensions of burnout but over-commitment significantly influenced only personal and work-related burnout. Both ERI and over-commitment were different among professions.     

Multiple mini-interviews predict clerkship and licensing examination performance

OBJECTIVE: The Multiple Mini-Interview (MMI) has previously been shown to have a positive correlation with early medical school performance. Data have matured to allow comparison with clerkship evaluations and national licensing examinations. METHODS: Of 117 applicants to the Michael G DeGroote School of Medicine at McMaster University who had scores on the MMI, traditional non-cognitive measures, and undergraduate grade point average (uGPA), 45 were admitted and followed through clerkship evaluations and Part I of the Medical Council of Canada Qualifying Examination (MCCQE). Clerkship evaluations consisted of clerkship summary ratings, a clerkship objective structured clinical examination (OSCE), and progress test score (a 180-item, multiple-choice test). The MCCQE includes subsections relevant to medical specialties and relevant to broader legal and ethical issues (Population Health and the Considerations of the Legal, Ethical and Organisational Aspects of Medicine[CLEO/PHELO]). RESULTS: In-programme, MMI was the best predictor of OSCE performance, clerkship encounter cards, and clerkship performance ratings. On the MCCQE Part I, MMI significantly predicted CLEO/PHELO scores and clinical decision-making (CDM) scores. None of these assessments were predicted by other non-cognitive admissions measures or uGPA. Only uGPA predicted progress test scores and the MCQ-based specialty-specific subsections of the MCCQE Part I. DISCUSSION: The MMI complements pre-admission cognitive measures to predict performance outcomes during clerkship and on the Canadian national licensing examination.     

Promoting Resident Autonomy to Maintain Quality of Life

In response to the COVID-19 pandemic, Canadian governments and healthcare organizations implemented restrictions on continuing care residents. From an ethical lens, governments and healthcare organizations were focused on preventing harm through promoting beneficence and non-maleficence; however, this was at the expense of resident autonomy. The rights of continuing care residents were stripped away when they were not given the opportunity to make informed decisions regarding their care and day-to-day life. Governments and healthcare organizations denied them the dignity to experience the positive outcomes that result from risk-taking based on their personal values and preferences. In an attempt to prevent resident harm from COVID-19 cases and deaths, governments and continuing care facilities forced residents into isolation. This negatively affected residents’ quality of life in the form of physical, mental, and cognitive health deterioration. Moving forward, governments and healthcare organizations need to take the time to engage residents in decision-making and policy development that affects their care, treatment, and support system. Governments and healthcare organizations must promote and safeguard resident autonomy to maintain quality of life.