Internet‐based assessment of medical students’ ophthalmoscopy skills

Acta Ophthalmol. 2010: 88: 854–857

Abstract.

Purpose: Ophthalmoscopy is an important skill in undergraduate medical education. The use and outcome of a novel Internet‐based method for assessing motor skills‐related aspects of ophthalmoscopy skills in medical students are described. Methods: A total of 139 students in Malmö and 255 students in Umeå were assessed during four consecutive semesters. An optic disc photograph was obtained in each student. At assessment each student was asked to examine the eye of a fellow student. Fifteen disc photographs were displayed on a computer screen. One of the photographs was from the fellow student. The remaining images had been randomly selected. The student was asked to identify the optic disc of the fellow student from the 15 photographs on the screen. In one semester, the time spent on this by each student was recorded. Results: All students completed the task. The average pass ratio was 96.4% (ranging from 94.0% to 98.0%). Median student times were 5.53 mins in Malmö and 6.36 mins in Umeå. Conclusions: This is the first automated, Internet‐based assessment of an ophthalmic motor skill performed in co‐operation between universities. The method used bears more similarities to real‐life ophthalmoscopy than methods that use model eyes. Time investments and infrastructure demands were comparatively low. The method was designed for stand‐alone assessment of ophthalmoscopy or as an objective structured clinical examination station, but it may also be used during the learning process. The collaboration between universities was easily transformed into routine practice and similar projects should be encouraged to increase the standardization of assessment.     

Assessment of age‐related maculopathy using subjective vision tests

This paper reviews non‐standard, clinical vision tests that may be used to detect the earliest visual loss in age‐related maculopathy (ARM), before fundus changes are detected. We recommend a clinical test battery for all patients aged 60 years and older, comprising low luminance/low contrast (SKILL) VA or low contrast VA, desaturated D‐15 colour vision assessment, flicker perimetry, glare recovery and dark adaptation if possible, together with conventional assessments of case history, ophthalmoscopy and high contrast visual acuity (VA) for the detection and diagnosis of ARM. Reading rate is also discussed as a potential indicator of early visual loss. For monitoring the progressive visual loss in age‐related macular degeneration (AMD) and determining the requirements for optometric vision rehabilitation, we recommend more conventional clinical vision tests of distance and near visual acuity, reading rate, the effects of varying illumination and a functional central visual field assessment.     

Sunglasses,the European directive and the European standard

Purpose: The Optics and Radiometry Laboratory (ORLAB) provides a testing service for the Australian sunglass industry to assist its compliance with the mandatory Consumer Product Safety Standard. In doing so, a number of sunglasses carrying the CE mark (in effect a claim of compliance with the European Union Personal Protective Equipment Directive) have been tested. Since there is no mandatory testing in Europe (the sunglass industry is deemed self regulating), evaluating their performance will give an insight into compliance with the Directive. Methods: Consecutive submissions to ORLAB over a period of 18 months were examined for CE marked sunglasses. The results for 646 CE marked pairs of sunglasses tested during 2003 and first half 2004 were examined and evaluated for compliance with EN 1836 (the European sunglass standard) on the basis of refractive power, prismatic power, transmittance (visible, UV and coloration) and polarization. Results: A total of 17.3% failed EN 1836 and 3.3% were borderline (i.e. within our uncertainties of measurement of the requirement). The failures comprised: 0.9% lens defect; 12.5% polarization alignment error; 2.7% excessive vertical prism; 22.3% excessive horizontal prism; 27.7% excessive spherical power; 14.3% excessive cylindrical power; 1.8% excessive UV transmittance; and 23.2% transmittance difference between lenses (always gradient tints). Conclusion: With up to 20% non‐compliance, self regulation is not working particularly well in providing the public with complying sunglasses.     

Evaluation of adding item‐response theory analysis for evaluation of the European Board of Ophthalmology Diploma examination

Purpose: To investigate whether introduction of item‐response theory (IRT) analysis, in parallel to the ‘traditional’ statistical analysis methods available for performance evaluation of multiple T/F items as used in the European Board of Ophthalmology Diploma (EBOD) examination, has proved beneficial, and secondly, to study whether the overall assessment performance of the current written part of EBOD is sufficiently high (KR‐20 ≥ 0.90) to be kept as examination format in future EBOD editions. Methods: ‘Traditional’ analysis methods for individual MCQ item performance comprise P‐statistics, Rit‐statistics and item discrimination, while overall reliability is evaluated through KR‐20 for multiple T/F items. The additional set of statistical analysis methods for the evaluation of EBOD comprises mainly IRT analysis. These analysis techniques are used to monitor whether the introduction of negative marking for incorrect answers (since EBOD 2010) has a positive influence on the statistical performance of EBOD as a whole and its individual test items in particular. Results: Item‐response theory analysis demonstrated that item performance parameters should not be evaluated individually, but should be related to one another. Before the introduction of negative marking, the overall EBOD reliability (KR‐20) was good though with room for improvement (EBOD 2008: 0.81; EBOD 2009: 0.78). After the introduction of negative marking, the overall reliability of EBOD improved significantly (EBOD 2010: 0.92; EBOD 2011:0.91; EBOD 2012: 0.91). Conclusion: Although many statistical performance parameters are available to evaluate individual items, our study demonstrates that the overall reliability assessment remains the only crucial parameter to be evaluated allowing comparison. While individual item performance analysis is worthwhile to undertake as secondary analysis, drawing final conclusions seems to be more difficult. Performance parameters need to be related, as shown by IRT analysis. Therefore, IRT analysis has proved beneficial for the statistical analysis of EBOD. Introduction of negative marking has led to a significant increase in the reliability (KR‐20 > 0.90), indicating that the current examination format can be kept for future EBOD examinations.     

1998 European Cataract Outcome Study: report from the European Cataract Outcome Study Group

PURPOSE: To collect clinical data on cataract surgery to allow participating surgeons to compare their performance with that of their colleagues in an anonymous manner. SETTING: Surgeons from 31 surgical units providing cataract surgery in 13 European countries. METHODS: Every patient at each participating unit having surgery during 1 study month was evaluated. Data were reported to the coordinating center at the time of surgery and at the final examination. When the study was closed 6 months after surgery, all participants were provided with the outcomes from their own patients so they could compare them with outcomes from other centers. RESULTS: The study included preoperative and intraoperative data on 2950 patients. Complete follow-up data were available for 2731 patients. The surgical audit included surgically induced astigmatism, proximity of target refraction, and the frequency of major complications. For each variable, a large variation in outcome between participating centers was found. Most centers had results both above and below average for different variables. CONCLUSION: Cataract surgery data collected from 31 units in 13 European countries allowed participants to compare their performance with that of their colleagues in an anonymous manner. Significant variation was found in the outcomes among the units, with many units reporting results above and below the averages.