Development and field testing of a self-assessment guide for computer-based provider order entry

This is a review for Carl V. Vartian, MD, Hardeep Singh, MD, Elise Russo and Dean Sittig’s Development and Field Testing of a Self-Assessment Guide for Computer-Based Provider Order Entry.

Background

While newly-available computer-based physician order entry (CPOE) can provide many benefits for the medical organizations and providers that use them, there are certain patient safety risks that might be presented after a CPOE implementation. With the Meaningful use incentive promoting electronic health record (EHR) and CPOE adoption in both the inpatient and outpatient settings, the accelerated implementations have increased reports of negative effects of these systems’ use. As a result of these negative reports, there has been a push to create a method of assessing CPOE-specific risks for healthcare organizations.

Materials and Methods

The authors performed analysis on relevant literature based on EHRs/CPOE systems and patient safety and interviewed subject matter experts in order to create a base of 250 items to include in the assessment. These items were then preliminary tested at site visits with clinicians. After this first round of tests, the items had been compressed to 22 concepts to be assessed.

They presented this 22 item assessment to a group of chief medical informatics officers (CMIO) and asked to them to respond with how much deliberation the CPOE system presented for each of the items. The responses to the assessment were assembled and analyzed.

Results

There were nine CMIOs that returned the completed assessment, from various hospitals across the country, with a variety of EMRs in use. The CMIOs indicated that they were mostly able to complete the assessment by themselves and thought the items were concise and some suggested a few extra items to be assessed (a common theme in the respondents' interviews were the concerns of over-use of alerts). The respondents all agreed that the assessment was useful and perceived the purpose of the guide as a review for making sure the widely accepted practices in CPOE are implemented.

Discussion

The authors successfully created and field tested a SAFER assessment. Some of the takeaways from this project included:

• The opinions of what practices make up a safe and effective CPOE integration differed greatly from one CMIO to another.
• Some respondents thought that there were important items left out of the assessment.
• Wording of the assessment guide was unclear and needed to be more specific.
• Some of the recommended items in the assessment were not feasible with the various practices’ current software.

Commentary

In this article, the authors tried to help come up with a solution to a nationally escalated need. Taking in input from various resources to create a guideline for an assessment was a great way to start tackling this project. They tested this and have tweaked their original assessment guide to mirror what they had learned in this project which is great, but they could definitely be overloaded with feedback if they continue to test. Since they only had nine respondents in the test, they were able to go through and thoroughly include all feedback, but had they received a response from every hospital using an electronic medical record (EMR), a more analytical approach to feedback patterns would most likely need to be taken.