Development and field testing of a self-assessment guide for computer-based provider order entry.

From Clinfowiki
Revision as of 09:42, 20 November 2015 by Shyla Narasimhan (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

This is a review for the article by Vartian et al. "Development and field testing of a self-assessment guide for computer-based provider order entry."[1]


Though CPOE systems have become widely used over the last several years, many organizations have reported unintended consequences of these systems, and many more have not yet attained even a fraction of the patient safety benefits these systems could offer. In order to offer guidance on how to improve the safety of these systems (and in 8 other areas), the Office of the National Coordinator (ONC) for Health Information Technology (HIT) commissioned a team of experts to develop a guide that "healthcare organizations can use to proactively assess potential EHR-related safety problems" in this area (and the 8 others).[1] These guides are known as the SAFER (Safety Assurance Factors for EHR Resilience) Guides.

This paper describes the development and field tests of the CPOE Guide.


Guide Development

Guide development involved a literature review and an iterative process with subject matter experts (informatics, pharmacy, patient safety, human factors engineering, and usability) whereby various potential practices were vetted and revised. 250 items were cut down to about 20 by three of the investigators, and then these items were validated during 5 site visits to large and small ambulatory practices and hospitals. These visits involved one-on-one interviews with personnel who use CPOE. Based on feedback from these interviews, the guide was revised again.

Guide description: "The resultant draft consisted of 22 checklist-type items that represented CPOE-related safety practices, with additional detailed descriptions of these practices and examples of how to operationalize them included in a supplementary Appendix. For each item, respondents could indicate the degree of implementation of each practice at their respective sites. Possible responses included 'not implemented,' 'partially implemented in some areas,' 'fully implemented in some areas,' and 'fully implemented in all areas.'"[1]

Recommended Practices as part of the Guide:

  • 1. Coded allergen and reaction information (or No Known Allergies [NKA]) are entered and updated in the EHR prior to order entry.
  • 2. Evidence-based order sets are available for common tasks/conditions and are updated on a regular basis, and usage is monitored.
  • 3. User-entered orderable items are matched to (or can be looked up from) a list of standard terms.
  • 4. EHR can cancel and acknowledge receipt of an order with lab, radiology, and pharmacy.
  • 5. EHR is used for ordering medications, diagnostic tests, and procedures.
  • 6. There is minimal use of free-text order-entry (i.e., data are entered and stored in coded form).
  • 7. Order entry information is electronically communicated (i.e., via the computer/mobile messaging) to the appropriate people responsible for carrying out the order.
  • 8. Drug–allergy interaction checking occurs at entry of new medication orders or new allergies.
  • 9. Duplicate checking occurs for certain orders (excluding PRN medications).
  • 10. Drug–condition checking occurs for important interactions between drugs and selected conditions.
  • 11. Drug–patient age checking occurs for important age-related interactions.
  • 12. Dose-range checking occurs before medication orders are submitted for dispensing (e.g., maximum dose amoxicillin 2-g oral tablets).
  • 13. Only the most significant and actionable drug–drug interaction-related alerts, as determined by the facility, are presented to providers.
  • 14. Clinicians are required to re-enter their password, or a unique PIN, to “sign” or authenticate an order.
  • 15. Corollary (or consequent) orders are automatically suggested by certain medication entries and are linked to and carried forward with the original order.
  • 16. Users can access clinical reference materials, including institution-specific knowledge links, directly from the EHR.
  • 17. The Leap Frog Test is taken to ensure safety of CDS.
  • 18. Critical patient information is visible during the order-entry process.
  • 19. The clinician is notified (e.g., by icon to signify non-formulary medication or send-out test) when additional steps (electronic or manual) are needed to complete the order being requested.
  • 20. There is minimal use of abbreviations and acronyms, and when they are used, they are clearly spelled out in all on-screen or printed information displays.
  • 21. Additional safeguards prevent errors related to prescribing of high-risk medications in the EHR.
  • 22. Key metrics related to order-entry use are defined, measured, reported, and acted upon.

Field Testing

The guide was field tested with 9 chief medical information officers (CMIOs) at different organizations across the US. Subjects were recruited using the Association of Medical Directors of Information Systems (AMDIS) listserv.

Participants were asked to complete the assessment (the guide) and then to complete a structured phone interview administered by one of the authors.


The participants represented a diverse group of facilities using different electronic health records (EHRs) with varying configurations of the EHR/CPOE systems.

For the most part it did not take participants very long to complete the 22 item guide, and they felt comfortable doing so.

Responses to the guide items varied across the spectrum from non to full implementation. Only 5 recommended practices were fully implemented at all sites (these involved basic CPOE functionality: items 4 and 5 or allergy checking: items 1 and 8).

Participants did suggest additional items that they believed should be part of the guide, and consensus for best practices was not reached.

All participants responded that the guide was useful, though they believed so for differing reasons.


As a result of this project, the CPOE guide that had been developed was refined and then combined with a Clinical Decision Support (CDS) guide that had originally been developed separately.

It was determined that this guide is helpful and feasible to complete, despite the fact that CMIO's beliefs about what constitutes safe and effective CPOE use varies considerably from person to person (or organization to organization).


Considering how widespread CPOE systems are these days, it is essential for health care organizations to make sure they are implementing and using these systems as safely and effectively as possible. The SAFER Guide for CPOE with CDS is a useful assessment tool for organizations looking to benchmark and then improve CPOE safety.


  1. 1.0 1.1 1.2 Vartian CV, Singh H, Russo E, Sittig DF. Development and field testing of a self-assessment guide for computer-based provider order entry. J Healthc Manag. 2014 Sep-Oct;59(5):338-52.