Misconceptions and miscarriages in multiple choice questions.

A modification of the standard format of multiple choice question (MCQ) examinations, recently introduced in certain medical schools in this country, is decribed. The scheme allows for the variation of marks allocated to different in the paper, depending upon the relevance, importance and degree of difficulty of each question. However, the manner in which this new system is being implemented in some cases transgresses some fundamental principles of MCQ examinations. The consequence of this is that the average mark for the class is unintentionally low, with the good students separated from the main body of the class by a disproportionate number of marks. In addition, the examination lends itself to abuse by the enterprising student who is familar with the system of mark allocation.     

FINAL EXAMINATIONS IN SURGERY: SUPPORT FOR THE RETENTION OF THE CLINICAL EXAMINATION

Students' results in the final M.B., B.S. (W.A.) examination in surgery between 1976 and 1979 inclusive (375 students) were analysed. Marks in all parts of the examination, namely clinical, multiple choice, essays, pathology, and fifth year multiple choice, were correlated with each other and the total mark in surgery. Multivariate analysis showed that the marks in the clinical examination accounted for most of the variation in a student's total mark, and this was only in part due to its heavy weighting in Western Australia. The authors of this paper strongly support the retention of the clinical examination in surgery and consider that the loading placed on this part of the final examination in Western Australia is justified.     

The historic predictive value of Canadian orthopedic surgery residents’ orthopedic in-training examination scores on their success on the RCPSC certification examination

Background

Positive correlation between the orthopedic in-training examination (OITE) and success in the American Board of Orthopaedic Surgery examination has been reported. Canadian training programs in internal medicine, anesthesiology and urology have found a positive correlation between in-training examination scores and performance on the Royal College of Physicians and Surgeons of Canada (RCPSC) certification examination. We sought to determine the potential predictive value of the OITE scores of Canadian orthopedic surgery residents on their success on their RCPSC examinations.

Methods

A total of 118 Canadian orthopedic surgery residents had their annual OITE scores during their 5 years of training matched to the RCPSC examination oral and multiple-choice questions and to overall examination pass/fail scores. We calculated Pearson correlations between the in-training examination for each postgraduate year and the certification oral and multiple-choice questions and pass/fail marks.

Results

There was a predictive association between the OITE and success on the RCPSC examination. The association was strongest between the OITE and the written multiple-choice examination and weakest between the OITE and the overall examination pass/fail marks.

Conclusion

Overall, the OITE was able to provide useful feedback to Canadian orthopedic surgery residents and their training programs in preparing them for their RCPSC examinations. However, when these data were collected, truly normative data based on a Canadian sample were not available. Further study is warranted based on a more refined analysis of the OITE, which is now being produced and includes normative percentile data based on Canadian residents.     

A comparison of traditional assessment with the objective structured clinical examination (OSCE)

As the objective structured clinical examination (OSCE) was being introduced into the medical curriculum, its validity was evaluated and the results obtained were compared with three components of the traditional assessment, i.e. tutors' mark, clinical assessment (long case) and multiple-choice questions (MCQs) in 170 5th-year students. The OSCE appeared to be a reliable and valid test of clinical skills, although this became apparent only with increasing experience. Traditional methods, in particular those which are inherently subjective, were found to upgrade students more often than OSCE scores. There were some correlations between OSCE marks and results of traditional examinations, especially the clinical assessment and the MCQ. Accordingly the OSCE appears to measure aspects of both clinical competence and theoretical knowledge. It has many advantages over traditional methods and should therefore be considered for inclusion in the assessment of medical students.     

Quality control in the Part I F.R.A.C.S. examination.

The conduct of College examinations has seemed a mystery to many candidates. Both candidates and supervisors of training have echoed this concern with the R.A.C.S. new Part 1 examination. This paper describes the work of the Board of Examiners of the Royal Australasian College of Surgeons and the procedures by which the new Part 1 examination is constructed, administered, analysed and refined. It sets out how the examination has been developed into an educational instrument to provide feedback to candidates, to teachers and to examiners, so that each can improve his performance. In addition, the Board has acquired data from each candidate on his learning opportunities, and from each candidate's mentor on various performance characteristics of the candidate.     

Multiple-choice examinations: adopting an evidence-based approach to exam technique

Negatively marked multiple-choice questions (MCQs) are part of the assessment process in both the Primary and Final examinations for the fellowship of the Royal College of Anaesthetists. It is said that candidates who guess will lose marks in the MCQ paper. We studied candidates attending a pre-examination revision course and have shown that an evaluation of examination technique is an important part of an individual's preparation. All candidates benefited substantially from backing their educated guesses while only 3 out of 27 lost marks from backing their wild guesses. Failure to appreciate the relationship between knowledge and technique may significantly affect a candidate's performance in the examination.     

QUALITY CONTROL IN THE PART I F.R.A.C.S. EXAMINATION

The conduct of College examinations has seemed a mystery to many candidates. Both candidates and supervisors of training have echoed this concern with the R.A.C.S. new Part I examination. This paper describes the work of the Board of Examiners¹ of the Royal Australasian College of Surgeons and the procedures by which the new Part I examination is constructed, administered, analysed and refined. It sets out how the examination has been developed into an educational instrument to provide feedback to candidates, to teachers and to examiners, so that each can improve his performance. In addition, the Board has acquired data from each candidate on his learning opportunities, and from each candidate's mentor on various performance characteristics of the candidate.     

Primary FRCS Eng: I Evaluation of assessment techniques

In five consecutive Primary Examinations for the Fellowship of the Royal College of Surgeons of England, the scores of candidates in the multiple choice question paper, written paper, and oral interview have been analysed for mutual correlations and for the reproducibility of the written paper score. The conclusions reached were that all these scores correlate with each other, that no score can be left out without reducing the reliability of the examination, that the marking of written papers in a close-marking system is remarkably reproducible, and that the oral score contributes most, the multiple choice question paper the least, to the overall assessment.     

Stress examination of traumatized lateral ligaments of the ankle

In the diagnostic analysis of acute injury to the lateral ankle ligaments, ankle arthrography and plain film stress examination play an important role. Inasmuch as the treatment of choice, whether conservative or surgery, and insofar as the therapeutic results may be determined by the extent of injury, it is important to have access to quantitative information on the functional properties of the involved ligaments at the time of the initial diagnostic studies. This can be obtained by measuring the strain on the anterior talofibular (tf) and calcaneofibular (cf) ligaments as a function of pressure applied to the distal tibia. Any device that ensures that the externally applied pressure is proportionally carried through to the ligaments can be used for this purpose. The results of 25 quantitative stress examinations using a Telos stress device are compared with arthrography findings and findings at surgery. Stress examination correlates well with both arthrographic and operative findings. The method is also useful for the evaluation of chronic ankle instability, follow-up examinations, and for the detection of vestiges of previous trauma of the contralateral ankle. The procedure is straightforward and noninvasive and can be performed within 15 minutes.     

Management of stab wounds to the thoracoabdominal region. A clinical approach.

We studied 324 patients admitted to Harlem Hospital Center from July 1981 to June 1986 with stab wounds of the thoracoabdominal region (area limited by a coronal circle through the fifth interspaces anteriorly and seventh interspaces posteriorly from above to a subcostal circle 5 cm caudad to the costal margins and 12th ribs from below). We divided this region into 12 zones (six symmetrical zones on each side) using the above upper and lower limits and the costal margins, the midlines, and the anterior and posterior axillary lines. This study was conducted to determine the incidence of transdiaphragmatic penetration for the thoracoabdominal region stab wounds in each of the 12 zones identified for the first time and applied in this study, and the reliability and safety of surgical management based mainly on physical examination. The highest incidence of transdiaphragmatic penetration occurred in stab wounds of the left anterior lower thoracic zone (21.7%). The lowest incidence was 0% and the overall incidence was 11%. Physical examination was accurate in making the diagnosis in 95.4% of all cases and no mortality was associated with a delay in diagnosis that may have resulted from the serial physical examinations. We conclude that this method of selective operative management based on physical examinations is accurate and safe.     

Pressure-flow studies: short-time repeatability.

The within and between examination variation in selected test parameters and test results in repeated pressure-flow studies was determined in a prospective study of consecutive pressure-flow examinations in 22 patients. The patients were pressure-flow tested twice within a month. Furthermore, it was evaluated whether there was a systematic change in the measured parameters during retesting. By using the Abrams-Griffiths nomogram, patients were classified as obstructed, equivocal, or unobstructed. Within and between examination variations in classification were evaluated. We found a systematic variation in P(det.Qmax) during testing, which in the absence of statistically significant systematic variations in P(det.Close) and Qmax, indicates a physiological effect of repeated pressure-flow studies, resulting in a less-obstructed second voiding. Supporting this, we found that all patients who changed group of classification of bladder outlet obstruction in the first examination shifted to a group of less obstruction, as did 66% of the patients who changed group of classification of bladder outlet obstruction in the second examination. Still, 80, respectively 85%, of the patients remained in the same group of classification of bladder outlet obstruction during retesting in the first and second examinations, respectively. Classifying the degree of bladder outlet obstruction by Qmax, P(det.Qmax), and P(det.Close) 85% of the patients reproduced their test results accurately in both examinations and taking only the first voiding in both examinations into account 95% reproduced their test results.     

Reliability of manual medical examination techniques of the cervical spine. Study of quality assurance in manual diagnosis

OBJECTIVE: Chiropractic techniques are of particular importance for the examination of the cervical spine. The aim of this study was to assess interexaminer reliability of examination techniques of the cervical spine in subjects with and without musculoskeletal distortions of the neck. The interrater method was used with five independent examiners. METHOD: Twenty patients suffering from neck diseases and 20 asymptomatic subjects of similar age and gender were randomized and assessed by five examiners blind to patient histories. Statistical analysis was carried out using multiple logistic regression and the calculation of kappa. RESULTS: Compared to asymptomatic subjects, patients experienced pain significantly often when pressure was applied to the cervical zygapophysial joints and the superficial neck muscles (p < or = 0.05 and p < or = 0.01). In addition, segmental function tests induced kinesialgia significantly more often in patients than in asymptomatic subjects (p < or = 0.05 and p < or = 0.01). A significant relationship was not found between the patient's health status and the findings from muscle palpation and functional examination of the motion segments. The assessment of agreement within examiners beyond chance had to be calculated, aside from for few exceptions, little to moderate (0.2 < kappa < or = 0.6). CONCLUSIONS: Chiropractic techniques are an essential part of every examination of the cervical spine. The clinical impact has not been scientifically established until up to now. Based on our findings and literature, we conclude that interexaminer reliability of manual diagnosis in the examination of the cervical spine should be improved by standardizing the examination process and setting guidelines for documentation and evaluation criteria. Controlled and frequently repeated training sessions also contribute to the reproducibility of findings from manual examinations.     

Correlation of clinical examination characteristics with three sources of chronic low back pain.

BACKGROUND CONTEXT: Research has demonstrated some progress in using a clinical examination to predict discogenic or sacroiliac (SI) joint sources of pain. No clear predictors of symptomatic lumbar zygapophysial joints have yet been demonstrated. PURPOSE: To identify significant components of a clinical examination that are associated with symptomatic lumbar discs, zygapophysial joints and SI joints. STUDY DESIGN: A prospective, criterion-related concurrent validity study performed at a private radiology practice specializing in spinal diagnostics. PATIENT SAMPLE: The sample consisted of 81 patients with chronic lumbopelvic pain referred for diagnostic injections. OUTCOME MEASURES: Contingency tables were constructed for nine features of the clinical evaluation compared with the results of diagnostic injections. Statistical analysis included chi-squared test for independence, phi and odds ratios with confidence intervals. METHOD: Patients received blinded clinical examinations by physical therapists, and diagnostic injections were used as the criterion standard. RESULTS: Significant relationships were found between discogenic pain and centralization of pain during repeated movement testing, and pain when rising from sitting. Lumbar zygapophysial joint pain was associated with absence of pain when rising from sitting. Sacroiliac joint pain was related to three or more positive pain provocation tests, pain when rising from sitting, unilateral pain and absence of lumbar pain. CONCLUSIONS: Significant correlations exist between clinical examination findings and symptomatic lumbar discs, zygapophysial and SI joints. The strongest relationships were seen between SI joint pain and three or more positive pain provocation tests, centralization of pain for symptomatic discs and absence of pain when rising from sitting for symptomatic lumbar zygapophysial joints.     

An analysis of the results of the Part 1.Examination for the F.R.A.C.S.

An analysis of the results of the candidates undertaking the Part 1 F.R.A.C.S. examination between February 1973 and February 1975 inclusive is presented. Particular attention is paid to those who failed the examination at the first attempt, and their subsequent progress has been related to their performance at this time. The analysis forms the basis for the information which is now given to unsuccessful candidates and their clinical supervisors.     

Subclavian revascularization. A quarter century experience.

OBJECTIVE: Twenty-five years of experience with subclavian revascularizations were reviewed to determine the long-term patency rates of different extrathoracic approaches. SUMMARY BACKGROUND DATA: Although it is generally agreed that proximal subclavian stenosis should be treated by an extrathoracic route whenever possible, the optimum procedure is debated. Alternatives include subclavian carotid bypass, subclavian-to-subclavian or axillo-axillary bypasses, and the authors' preferred technique of subclavian carotid transposition (SCT). METHODS: Records were researched for the past 25 years in a single specialty surgical clinic for extrathoracic subclavian revascularizations. One hundred ninety such procedures were identified, and hospital charts and office medical records were reviewed for procedure, preoperative symptoms, blood pressure differentials, and postoperative complications. Patency was determined by physical examination, differential blood pressures, Doppler spectral analysis, duplex examinations, and arteriography. RESULTS: Bypass procedures were used infrequently, and although the results are reported, they are excluded from any analysis. Subclavian carotid transposition was used in 178 procedures. All anastomoses were found to be patient at follow-up, except for one, which failed at 26 months. Mean follow-up was 46 months, with five patients lost to follow-up. Overall mortality rate was 2.2%, with the mortality falling to 1.1% if only subclavian carotid transposition patients are included. CONCLUSIONS: Subclavian carotid transposition should be the treatment of choice for routine subclavian carotid occlusive disease because of its exceptional long-term patency and low morbidity.     

The elusive colonic malignancy. A need for definitive preoperative localization

Colonoscopy with biopsy is the standard of practice for the diagnosis of colonic malignancies. Unfortunately, the inability of endoscopy to obtain precise distance measurements from the anal verge can make localization of lesions at operation difficult. For this reason, preoperative barium enema or intraoperative colonoscopy have been advocated to further pinpoint the sites of those lesions not thought to be easily located at operation. Five patients are presented in whom malignant lesions of the colon were diagnosed and verified histologically, but were later undetectable at operation or subsequent colonoscopic examinations. Four of these patients underwent laparotomy and three received colon resections. None of these patients' tumors were identified during intraoperative colonoscopy, in the resected bowel on pathologic examination, or on follow-up colonoscopy. A fifth patient is presented who spontaneously passed a polyp containing invasive adenocarcinoma, but multiple colonoscopic examinations have failed to identify the site of the lesion. To date, none of these tumors have recurred with periods of follow-up ranging from 6 months to 2 years. These patients demonstrate a poorly documented and little understood aspect of the behavior of colonic malignancies, i.e., the ability to spontaneously regress or slough from the bowel wall. Based on these instances, localization of potentially malignant colon lesions is recommended with submucosal dye injections at initial endoscopy or with colonoscopy in the operating room immediately prior to operation.     

AN ANALYSIS OF THE RESULTS OF THE PART I EXAMINATION FOR THE F.R.A.C.S.1

An analysis of the results of the candidates undertaking the Part I F.R.A.C.S. examination between February 1973 and February 1975 inclusive is presented. Particular attention is paid to those who failed the examination at the first attempt, and their subsequent progress has been related to their performance at this time. The analysis forms the basis for the information which is now given to unsuccessful candidates and their clinical supervisors.     

Carcinoid tumor of the appendix. A not especially rare entity]

Carcinoid tumors arise from neuroendocrine system and one of their preferred sites is the appendix. Most of appendiceal carcinoids almost always are clinically silent and are incidentally found at histological examination. For this reason, histological examination of the appendix is recommended in every case of the appendicectomy. Prognostic factors are: tumor stage, histologic pattern and differentiation. The authors present 6 cases of appendiceal carcinoid tumor, clinically silent and found at histologic examination (3 cases of appendicectomy for acute appendicitis, 3 cases of appendicectomy associated with abdominal surgical interventions for other pathology) and compare their experience with the most recent literature on this subject. The conclusion is drawn that the size of this tumor is the main factor that surgeons must consider for the choice of surgical treatment. Incidental, clinical silent, small (less than 2 cm in size) appendiceal carcinoid tumors can be treated by appendicectomy, and they do not need follow-up. Greater (more than 2 cm in size) appendiceal carcinoid tumors, in young patients must be treated with a right hemicolectomy and they need follow-up (periodically radiological, ultrasonographic and endoscopic examinations, tumor markers) because there is the possibility of recurrence or metastases.     

A new approach to the low-back physical examination. Behavioral assessment of mechanical signs

A rationale for a new approach to the low-back physical examination was developed. A set of 21 tests, 17 assessing organic and four assessing nonorganic signs, were organized into an examination according to specified criteria, and the reliability of the patient-reported and examiner-observed measures within the examination assessed. Primary outcome measures included patient reports of their pain location, aggravation and examiner-observed pain behaviors resulting from the maneuvers. Two pain behavior composites, conceptualized as outcome measures, were developed, one based on the 17 organic tests and one based on the four nonorganic tests. Design: The reliability of the physical examination was assessed using a short-term test-retest paradigm. Three raters, two experienced orthopaedic surgeons and an RN with no previous experience in administering physical examinations were trained in the examination methods. Patients were assigned to one of three rater pairs and examined twice within a single day. During each examination both raters evaluated each patient; however, rater role as examiner or observer was reversed across examination. Results: Forty-two patients were examined. Average times of 13.9 and 11.6 minutes were required to complete examinations 1 and 2, respectively. In addition, the time required to complete the examination decreased as the examiners became more familiar with the procedure, suggesting that an experienced examiner would usually be able to complete the examination in approximately 10 minutes. Within-examination reliabilities for the patient-reported measures (pain location and aggravation) were universally high, as expected, since these rating required the rater only to correctly hear and code patient responses.(ABSTRACT TRUNCATED AT 250 WORDS)