Impact of CPOE on mortality rates--contradictory findings, important messages
Ammenwerth E, Talmon J, Ash JS, Bates DW, Beuscart-Zephir MC, Duhamel A, Elkin PL, Gardner RM, Geissbuhler A. Impact of CPOE on mortality rates--contradictory findings, important messages. Methods Inf Med. 2006; 45(6):586-93.
Background: Two recent studies focused on pediatric mortality rates after CPOE implementation. Han et al. published a paper in 2005 which showed an increased mortality rate in inter-facility transfer patients to the ICU the first 5 months after a commercial CPOE implementation at the Children’s Hospital of Pittsburgh (CHP) compared to the 13 months before (6.6 vs 2.8). A second study by Del Beccaro, published in 2006, found no difference in mortality 5 and 13 months post implementation compared to 13 months prior to implementation for all patients admitted to the Children’s Hospital and Regional Medical Center (CHRMC), Seattle, using the same commercial system.
Objective: The objective of this paper was to analyze the contradictory findings of the two studies and make recommendations for further research on implementing informatics systems.
Methods: The opinions of seven experts on CPOE systems from the United States and Europe were asked to comment on the two papers specifically focusing on the contradictions between them. The experts were: Joan Ash (Oregon Health and Science University), David Bates (Brigham and Women’s Hospital), Marie-Catherine Beuscart-Zéphir and Alain Duhamel (University Hospital of Lille), Peter Elkin (Mayo Clinic College of Medicine), Reed Gardner (University of Utah), and Antoine Geissbuhler (Geneva University Hospitals).
Results: The main explanation for the differences in the outcomes was felt to be the differences in the implementation of their respective CPOE systems. The Del Beccaro study occurred a year later and had the benefit of learning from the mistakes of the implementation done at Children’s Hospital of Pittsburgh. It was observed that CHP did not follow best practices for CPOE implementation by rapidly deploying their system over six days thus limiting the ability to make changes, order entry was not allowed until the patient physically entered the hospital, order sets were not developed prior to deployment, their network had stability issues and there were major policy changes such as moving medications to a central pharmacy which directly affected care.
Methodologically, there were flaws in both studies. Both were retrospective studies and as such can suggest association but not prove causality between CPOE implementation and mortality rates. In addition, the Del Beccaro study was done on a different population (all patients in the hospital versus inter-facility transfers to the ICU). As such, the Del Beccaro study is not a replication of the Han et al. study.
From a usability and human factor point of view, both studies were lacking. At best, they were case reports of an implementation as their comments on usability, socio-technical, organizational and safety issues were done by subjective assessment/observation without any formal modeling.
In regards to future research, all of the experts called for better evaluation and methodological studies of CPOE implementations.
Commentary: The implementation of a CPOE system is considered one of the most difficult activities a health organization can undertake as the system touches every workflow present. An analogy, mentioned in the article, is that a new CPOE system is much akin to an organ transplant- either it will be engrafted in to the organization as a whole or else it will be rejected with bad consequences.
The two studies are not directly comparable (their populations differ); still, there are lessons that can be drawn from them. Namely, the implementation of a CPOE is a socio-technological phenomenon requiring adherence to best practices. The knowledge field surrounding those practices is young and requires further innovation in evaluation methodologies to assess the best route to take. To underscore this point one of the experts questioned whether the studies were published in the journal Pediatrics as an indication of the maturity of our knowledge in this field.
