Development and field testing of a self-assessment guide for computer-based provider order entry.

This is a review for the article by Vartian et al. "Development and field testing of a self-assessment guide for computer-based provider order entry."

Background

Though CPOE systems have become widely used over the last several years, many organizations have reported unintended consequences of these systems, and many more have not yet attained even a fraction of the patient safety benefits these systems could offer. In order to offer guidance on how to improve the safety of these systems (and in 8 other areas), the Office of the National Coordinator (ONC) for Health Information Technology (HIT) commissioned a team of experts whose job it was to develop a guide that "healthcare organizations can use to proactively assess potential EHR-related safety problems" in this area (and the 8 others). These guides are known as the SAFER (Safety Assurance Factors for EHR Resilience) Guides.

This paper describes the development and field tests of the CPOE Guide.

Methods

Guide Development

Guide development involved a literature review and an iterative process with subject matter experts (informatics, pharmacy, patient safety, human factors engineering, and usability) whereby various potential practices were vetted and revised. 250 items were cut down to about 20 by three of the investigators, and then these items were validated during 5 site visits to large and small ambulatory practices and hospitals. These visits involved one-on-one interviews with personnel who use CPOE. Based on feedback from these interviews, the guide was revised again.

Guide description: "The resultant draft consisted of 22 checklist-type items that represented CPOE-related safety practices, with additional detailed descriptions of these practices and examples of how to operationalize them included in a supplementary Appendix. For each item, respondents could indicate the degree of implementation of each practice at their respective sites. Possible responses included 'not implemented,' 'partially implemented in some areas,' 'fully implemented in some areas,' and 'fully implemented in all areas.'"

Field Testing

The guide was field tested with 9 chief medical information officers (CMIOs) at different organizations across the US. Subjects were recruited using the Association of Medical Directors of Information Systems (AMDIS) listserv.

Participants were asked to complete the assessment (the guide) and then to complete a structured phone interview administered by one of the authors.

Results

The participants represented a diverse group of facilities using different electronic health records (EHRs) with varying configurations of the EHR/CPOE systems.

For the most part it did not take participants very long to complete the 22 item guide, and they felt comfortable doing so.

Responses to the guide items varied across the spectrum from non to full implementation. Only 5 recommended practices were fully implemented at all sites (these involved basic CPOE functionality: items 4 and 5 or allergy checking: items 1 and 8).

Participants did suggest additional items that they believed should be part of the guide