The impact of computerized physician medication order entry in hospitalized patients--a systematic review

OBJECTIVE: To identify all published studies evaluating computerized physician order entry (CPOE) in the inpatient setting and uniformly classify these studies on outcome measure and study design. DATA SOURCES: All studies that evaluated the effect of CPOE on outcomes pertaining to the medication process in inpatients were electronically searched in MEDLINE (1966 to August 2006), EMBASE (1980 to August 2006) and the Cochrane library. In addition, the bibliographies of retrieved articles were manually searched. Articles were selected if one of their main objectives was CPOE evaluation in an inpatient setting. REVIEW METHOD: Identified titles and abstracts were independently screened by three reviewers to determine eligibility for further review. RESULTS: We found 67 articles, which included articles on CPOE evaluation on some outcome at the time of ordering. Most papers evaluated multiple outcome measures. The outcome measures were clustered in the following categories: adherence (n=22); alerts and appropriateness of alerts (n=7); safety (n=21); time (n=7); costs and (organizational) efficiency (n=23); and satisfaction, usage and usability (n=10). Most studies used a before-after design (n=35) followed by observational studies (n=24) and randomized controlled trials (n=8). CONCLUSION: The impact of CPOE systems was especially positive in the category adherence to guidelines, but also to some extent in alerts and appropriateness of alerts; costs and organizational efficiency; and satisfaction and usability. Although on average, there seems to be a positive effect of CPOE on safety, studies tended to be non-randomized and were focused on medication error rates, not powered to detect a difference in adverse drug event rates. Some recent studies suggested that errors, adverse drug events (ADEs) and even mortality increased after CPOE implementation. Only in the category time the impact has been shown to be negative, but this only refers to the physician's time, not the net time. Except for safety, on the whole spectrum of outcomes, results of RCT studies were in line with non-RCT study results.     

Impact of a computerized physician order entry system on nurse-physician collaboration in the medication process

Background

Due to their efficiency and safety potential, computerized physician order entry (CPOE) systems are gaining considerable attention in in-patient settings. However, recent studies have shown that these systems may undermine the efficiency and safety of the medication process by impeding nurse-physician collaboration.

Objective

To evaluate the effects of a CPOE system on the mechanisms whereby nurses and physicians maintain their collaboration in the medication process.

Setting and methodology

Six internal medicine wards at the Erasmus Medical Centre were included in this study. A questionnaire was used to record nurses’ attitudes towards the effectiveness of the former paper-based system. A similar questionnaire was used to evaluate nurses’ attitudes with respect to a CPOE system that replaced the paper-based system. The data were complemented and triangulated through interviews with physicians and nurses.

Results

Response rates for the analyzed questions in the pre- and post-implementation questionnaires were 54.3% (76/140) and 52.14% (73/140). The CPOE system had a mixed impact on medication work: while it improved the main non-supportive features of the paper-based system, it lacked its main supportive features. The interviews revealed more detailed supportive and non-supportive features of the two systems. A comparison of supportive features of the paper-based system with non-supportive features of the CPOE system showed that synchronisation and feedback mechanisms in nurse-physician collaborations have been impaired after the CPOE system was introduced.

Conclusion

This study contributes to an understanding of the affected mechanisms in nurse-physician collaboration using a CPOE system. It provides recommendations for repairing the impaired mechanisms and for redesigning the CPOE system and thus for better supporting these structures.     

Usability evaluation of pharmacogenomics clinical decision support aids and clinical knowledge resources in a computerized provider order entry system: A mixed methods approach

BackgroundPharmacogenomics (PGx) is positioned to have a widespread impact on the practice of medicine, yet physician acceptance is low. The presentation of context-specific PGx information, in the form of clinical decision support (CDS) alerts embedded in a computerized provider order entry (CPOE) system, can aid uptake. Usability evaluations can inform optimal design, which, in turn, can spur adoption.ObjectivesThe study objectives were to: (1) evaluate an early prototype, commercial CPOE system with PGx-CDS alerts in a simulated environment, (2) identify potential improvements to the system user interface, and (3) understand the contexts under which PGx knowledge embedded in an electronic health record is useful to prescribers.MethodsUsing a mixed methods approach, we presented seven cardiologists and three oncologists with five hypothetical clinical case scenarios. Each scenario featured a drug for which a gene encoding drug metabolizing enzyme required consideration of dosage adjustment. We used Morae^® to capture comments and on-screen movements as participants prescribed each drug. In addition to PGx-CDS alerts, ‘Infobutton^®’ and ‘Evidence’ icons provided participants with clinical knowledge resources to aid decision-making.ResultsNine themes emerged. Five suggested minor improvements to the CPOE user interface; two suggested presenting PGx information through PGx-CDS alerts using an ‘Infobutton’ or ‘Evidence’ icon. The remaining themes were strong recommendations to provide succinct, relevant guidelines and dosing recommendations of phenotypic information from credible and trustworthy sources; any more information was overwhelming. Participants’ median rating of PGx-CDS system usability was 2 on a Likert scale ranging from 1 (strongly agree) to 7 (strongly disagree).ConclusionsUsability evaluation results suggest that participants considered PGx information important for improving prescribing decisions; and that they would incorporate PGx-CDS when information is presented in relevant and useful ways.     

Regenstrief Institute's Medical Gopher: A next-generation homegrown electronic medical record system

ObjectiveRegenstrief Institute developed one of the seminal computerized order entry systems, the Medical Gopher, for implementation at Wishard Hospital nearly three decades ago. Wishard Hospital and Regenstrief remain committed to homegrown software development, and over the past 4 years we have fully rebuilt Gopher with an emphasis on usability, safety, leveraging open source technologies, and the advancement of biomedical informatics research. Our objective in this paper is to summarize the functionality of this new system and highlight its novel features.Materials and methodsApplying a user-centered design process, the new Gopher was built upon a rich-internet application framework using an agile development process. The system incorporates order entry, clinical documentation, result viewing, decision support, and clinical workflow. We have customized its use for the outpatient, inpatient, and emergency department settings.ResultsThe new Gopher is now in use by over 1100 users a day, including an average of 433 physicians caring for over 3600 patients daily. The system includes a wizard-like clinical workflow, dynamic multimedia alerts, and a familiar ‘e-commerce’-based interface for order entry. Clinical documentation is enhanced by real-time natural language processing and data review is supported by a rapid chart search feature.DiscussionAs one of the few remaining academically developed order entry systems, the Gopher has been designed both to improve patient care and to support next-generation informatics research. It has achieved rapid adoption within our health system and suggests continued viability for homegrown systems in settings of close collaboration between developers and providers.     

Utilization of evidence-based computerized order sets in pediatrics

Little is known about utilization of different evidence-based order sets within computerized physician order entry (CPOE) systems. We designed a retrospective study of resident and attending physician order set utilization to evaluate the use of three evidence-based computerized order sets (asthma, post-appendectomy care, and community-acquired pneumonia (CAP)), and examine patient and admission characteristics associated with order set utilization in pediatrics. We studied all 529 asthma patients, 277 appendectomy patients, and 210 CAP patients admitted between 1 November 2001 and 30 November 2003 during implementation of standardized order sets at a large, independent, not-for-profit pediatric institution. We analyzed order set utilization for the three order sets and tested the relationship between order set use and potential factors associated with utilization. Order set utilization varied by condition (X(2)=339.2, p<0.001), with the asthma order set use rate highest (88.1%), followed by appendectomy order set utilization (79.4%), and substantially lower CAP order set use (21.1%). We found that trends in order set utilization also varied by condition. Only the asthma order set showed a trend of increasing use after implementation (z= -3.02, p=0.002). In addition, factors associated with order set utilization varied. Uses of the asthma and post-appendectomy order sets were associated with factors such as admission unit and case complexity. CAP order set utilization was associated with case complexity but not admission source. We conclude that health services organizations looking to implement computerized order sets to reduce unnecessary practice variation while promoting best practices must consider the different factors that may influence the use of each order set rather than relying on a one-size-fits-all implementation strategy. Further, issues such as the level of physician involvement in order set development and consensus around order set content may be particularly important factors influencing order set utilization.     

Automated detection of medication administration errors in neonatal intensive care

ObjectiveTo improve neonatal patient safety through automated detection of medication administration errors (MAEs) in high alert medications including narcotics, vasoactive medication, intravenous fluids, parenteral nutrition, and insulin using the electronic health record (EHR); to evaluate rates of MAEs in neonatal care; and to compare the performance of computerized algorithms to traditional incident reporting for error detection.MethodsWe developed novel computerized algorithms to identify MAEs within the EHR of all neonatal patients treated in a level four neonatal intensive care unit (NICU) in 2011 and 2012. We evaluated the rates and types of MAEs identified by the automated algorithms and compared their performance to incident reporting. Performance was evaluated by physician chart review.ResultsIn the combined 2011 and 2012 NICU data sets, the automated algorithms identified MAEs at the following rates: fentanyl, 0.4% (4 errors/1005 fentanyl administration records); morphine, 0.3% (11/4009); dobutamine, 0 (0/10); and milrinone, 0.3% (5/1925). We found higher MAE rates for other vasoactive medications including: dopamine, 11.6% (5/43); epinephrine, 10.0% (289/2890); and vasopressin, 12.8% (54/421). Fluid administration error rates were similar: intravenous fluids, 3.2% (273/8567); parenteral nutrition, 3.2% (649/20124); and lipid administration, 1.3% (203/15227). We also found 13 insulin administration errors with a resulting rate of 2.9% (13/456). MAE rates were higher for medications that were adjusted frequently and fluids administered concurrently. The algorithms identified many previously unidentified errors, demonstrating significantly better sensitivity (82% vs. 5%) and precision (70% vs. 50%) than incident reporting for error recognition.ConclusionsAutomated detection of medication administration errors through the EHR is feasible and performs better than currently used incident reporting systems. Automated algorithms may be useful for real-time error identification and mitigation.     

Development of an instrument for measuring clinicians' power perceptions in the workplace

We report on the development of an instrument to measure clinicians’ perceptions of their personal power in the workplace in relation to resistance to computerized physician order entry (CPOE). The instrument is based on French and Raven’s six bases of social power and uses a semantic differential methodology. A measurement study was conducted to determine the reliability and validity of the survey. The survey was administered online and distributed via a URL by email to 19 physicians, nurses, and health unit coordinators from a university hospital. Acceptable reliability was achieved by removing or moving some semantic differential word pairs used to represent the six power bases (alpha range from 0.76 to 0.89). The Semantic Differential Power Perception (SDPP) survey validity was tested against an already validated instrument and found to be acceptable (correlation range from 0.51 to 0.81). The SDPP survey instrument was determined to be both reliable and valid.     

Making pharmacogenomic-based prescribing alerts more effective: A scenario-based pilot study with physicians

To facilitate personalized drug dosing (PDD), this pilot study explored the communication effectiveness and clinical impact of using a prototype clinical decision support (CDS) system embedded in an electronic health record (EHR) to deliver pharmacogenomic (PGx) information to physicians. We employed a conceptual framework and measurement model to access the impact of physician characteristics (previous experience, awareness, relative advantage, perceived usefulness), technology characteristics (methods of implementation-semi-active/active, actionability-low/high) and a task characteristic (drug prescribed) on communication effectiveness (usefulness, confidence in prescribing decision), and clinical impact (uptake, prescribing intent, change in drug dosing). Physicians performed prescribing tasks using five simulated clinical case scenarios, presented in random order within the prototype PGx-CDS system. Twenty-two physicians completed the study. The proportion of physicians that saw a relative advantage to using PGx-CDS was 83% at the start and 94% at the conclusion of our study. Physicians used semi-active alerts 74–88% of the time. There was no association between previous experience with, awareness of, and belief in a relative advantage of using PGx-CDS and improved uptake. The proportion of physicians reporting confidence in their prescribing decisions decreased significantly after using the prototype PGx-CDS system (p = 0.02). Despite decreases in confidence, physicians perceived a relative advantage to using PGx-CDS, viewed semi-active alerts on most occasions, and more frequently changed doses toward doses supported by published evidence. Specifically, sixty-five percent of physicians reduced their dosing, significantly for capecitabine (p = 0.002) and mercaptopurine/thioguanine (p = 0.03). These findings suggest a need to improve our prototype such that PGx CDS content is more useful and delivered in a way that improves physician’s confidence in their prescribing decisions. The greatest increases in communication effectiveness and clinical impact of PGx-CDS are likely to be realized through continued focus on content, content delivery, and tailoring to physician characteristics.     

Transitioning from a computerized provider order entry and paper documentation system to an electronic health record: Expectations and experiences of hospital staff

ObjectivesTo examine healthcare worker's perceptions, expectations, and experiences regarding how work processes, patient-related safety, and care were affected when a quaternary care center transitioned from one computerized provider order entry (CPOE) system to a full electronic health record (EHR).MethodsThe I-SEE survey was administered prior to and 1-year after transition in systems. The construct validity and reliability of the survey was assessed within the current population and also compared to previously published results. Pre- and 1-year post-implementation scale means were compared within and across time periods.ResultsThe majority of respondents were nurses and personnel working in the acute care setting. Because a confirmatory factor analysis indicated a lack of fit of our data to the I-SEE survey's 5-factor structure, we conducted an exploratory factor analysis that resulted in a 7-factor structure which showed better reliability and validity. Mean scores for each factor indicated that attitudes and expectations were mostly positive and score trends over time were positive or neutral. Nurses generally had less positive attitudes about the transition than non-nursing respondents, although the difference diminished after implementation.ConclusionsFindings demonstrate that the majority of responding staff were generally positive about transitioning from CPOE system to a full electronic health record (EHR) and understood the goals of doing so, with overall improved ratings over time. In addition, the I-SEE survey, when modified based on our population, was useful for assessing patient care and safety related expectations and experiences during the transition from one CPOE system to an EHR.     

The state of the evidence for computerized provider order entry: A systematic review and analysis of the quality of the literature

Objective

This paper presents the results of a systematic literature review and a formal analysis of the scientific quality of empirical research on computerized provider order-entry (CPOE) applications.

Design

Formal, systematic review techniques were used to search the literature, determine study relevance, and evaluate study quality.

Measurement

A search of multiple databases from 1976 through mid-2007 yielded a final set of 46 articles. Relevance criteria included: (1) a direct comparison of a CPOE system with a non-CPOE system; (2) implementation in a clinical setting; and (3) clinically relevant outcomes.

Results

Study quality varied widely. Three major areas were identified for improvement in future studies: (1) internal validity, especially in terms of study designs, blinding, and instrumentation bias; (2) construct validity of the phenomenon of CPOE itself; and (3) measurement strategies, including reliability and validity assessments.

Conclusions

The evidence for the impact of CPOE needs to be improved to support scientific generalizability. Several common confounds are found in this literature. Future researchers will want to address them to improve the strength of the inference between CPOE and clinical outcomes. Discussion focuses on methods to improve future CPOE research.     

Electronic health record: Design and implementation of a lab test request module

BackgroundThe electronic health record (EHR) has become a fundamental tool in health care. The ordering and inclusion of lab tests and results is one of the most frequently requested services by EHR users. We have designed, developed and implemented in Andalusia, an autonomous community in the south of Spain (8.3 million inhabitants), a unified lab test request module for the Andalusian public health system EHR.PurposeAfter implementing the module in 27 laboratories, our objective is to assess its impact on healthcare activities and to ascertain whether its functional design addresses the needs and expectations of users.MethodsWe surveyed laboratory and healthcare professionals to assess their opinion of the module's operation in daily practices and the effect it has had on pre- and post-analytical quality indicators (before and after lab test module implementation).ResultsAll the laboratories surveyed noted that the implementation of the laboratory module in the EHR improved the analytical process, highlighting better safety in patient identification, less programming or container errors and shorter response times. Clinical professionals gave the module a rating of 7.8 out of 10, positively highlighting the speed at which results are delivered and their integration in the EHR. In terms of the model's drawbacks, laboratories have highlighted its rigidity in solving errors and clinical professionals have noted the requirement of adapting to a new nomenclature. It is also necessary to expand coding to all the tests available in clinical laboratories.ConclusionsThe results of our survey indicate that the functional design of our analytical testing module is suitable for user needs, allowing to integrate information from multiple laboratories in a single region. Based on our experience, the key aspects for the success of this project have been: a design conceived for both laboratories and clinical professionals, the involvement of laboratories as a key element of the project, as well as sufficient time of local piloting before widespread implementation which is basic for the success of a computer application that affects so many potential users of the health care system.     

Impact of errors in paper-based and computerized diabetes management with decision support for hospitalized patients with type 2 diabetes. A post-hoc analysis of a before and after study

ObjectiveMost preventable adverse drug events and medication errors occur during medication ordering. Medication order entry and clinical decision support are available on paper or as computerized systems. In this post-hoc analysis we investigated frequency and clinical impact of blood glucose (BG) documentation- and user-related calculation errors as well as workflow deviations in diabetes management. We aimed to compare a paper-based protocol to a computerized medication management system combined with clinical workflow and decision support.MethodsSeventy-nine hospitalized patients with type 2 diabetes mellitus were treated with an algorithm driven basal-bolus insulin regimen. BG measurements, which were the basis for insulin dose calculations, were manually entered either into the paper-based workflow protocol (PaperG: 37 patients) or into GlucoTab^®—a mobile tablet PC based system (CompG: 42 patients). We used BG values from the laboratory information system as a reference. A workflow simulator was used to determine user calculation errors as well as workflow deviations and to estimate the effect of errors on insulin doses. The clinical impact of insulin dosing errors and workflow deviations on hypo- and hyperglycemia was investigated.ResultsThe BG documentation error rate was similar for PaperG (4.9%) and CompG group (4.0%). In PaperG group, 11.1% of manual insulin dose calculations were erroneous and the odds ratio (OR) of a hypoglycemic event following an insulin dosing error was 3.1 (95% CI: 1.4–6.8). The number of BG values influenced by insulin dosing errors was eightfold higher than in the CompG group. In the CompG group, workflow deviations occurred in 5.0% of the tasks which led to an increased likelihood of hyperglycemia, OR 2.2 (95% CI: 1.1–4.6).DiscussionManual insulin dose calculations were the major source of error and had a particularly strong influence on hypoglycemia. By using GlucoTab^®, user calculation errors were entirely excluded. The immediate availability and automated handling of BG values from medical devices directly at the point of care has a high potential to reduce errors. Computerized systems facilitate the safe use of more complex insulin dosing algorithms without compromising usability. In CompG group, missed or delayed tasks had a significant effect on hyperglycemia, while in PaperG group insufficient precision of documentation times limited analysis. The use of old BG measurements was clinically less relevant.ConclusionInsulin dosing errors and workflow deviations led to measurable changes in clinical outcome. Diabetes management systems including decision support should address nurses as well as physicians in a computerized way. Our analysis shows that such systems reduce the frequency of errors and therefore decrease the probability of hypo- and hyperglycemia.     

Bootstrapping a de-identification system for narrative patient records: Cost-performance tradeoffs

PurposeWe describe an experiment to build a de-identification system for clinical records using the open source MITRE Identification Scrubber Toolkit (MIST). We quantify the human annotation effort needed to produce a system that de-identifies at high accuracy.MethodsUsing two types of clinical records (history and physical notes, and social work notes), we iteratively built statistical de-identification models by annotating 10 notes, training a model, applying the model to another 10 notes, correcting the model's output, and training from the resulting larger set of annotated notes. This was repeated for 20 rounds of 10 notes each, and then an additional 6 rounds of 20 notes each, and a final round of 40 notes. At each stage, we measured precision, recall, and F-score, and compared these to the amount of annotation time needed to complete the round.ResultsAfter the initial 10-note round (33 min of annotation time) we achieved an F-score of 0.89. After just over 8 h of annotation time (round 21) we achieved an F-score of 0.95. Number of annotation actions needed, as well as time needed, decreased in later rounds as model performance improved. Accuracy on history and physical notes exceeded that of social work notes, suggesting that the wider variety and contexts for protected health information (PHI) in social work notes is more difficult to model.ConclusionsIt is possible, with modest effort, to build a functioning de-identification system de novo using the MIST framework. The resulting system achieved performance comparable to other high-performing de-identification systems.     

Developing a standard for de-identifying electronic patient records written in Swedish: Precision, recall and F-measure in a manual and computerized annotation trial

Background

Electronic patient records (EPRs) contain a large amount of information written in free text. This information is considered very valuable for research but is also very sensitive since the free text parts may contain information that could reveal the identity of a patient. Therefore, methods for de-identifying EPRs are needed. The work presented here aims to perform a manual and automatic Protected Health Information (PHI)-annotation trial for EPRs written in Swedish.

Methods

This study consists of two main parts: the initial creation of a manually PHI-annotated gold standard, and the porting and evaluation of an existing de-identification software written for American English to Swedish in a preliminary automatic de-identification trial. Results are measured with precision, recall and F-measure.

Results

This study reports fairly high Inter-Annotator Agreement (IAA) results on the manually created gold standard, especially for specific tags such as names. The average IAA over all tags was 0.65 F-measure (0.84 F-measure highest pairwise agreement). For name tags the average IAA was 0.80 F-measure (0.91 F-measure highest pairwise agreement). Porting a de-identification software written for American English to Swedish directly was unfortunately non-trivial, yielding poor results.

Conclusion

Developing gold standard sets as well as automatic systems for de-identification tasks in Swedish is feasible. However, discussions and definitions on identifiable information is needed, as well as further developments both on the tag sets and the annotation guidelines, in order to get a reliable gold standard. A completely new de-identification software needs to be developed.