Clinical Reminder Redesign for Ambulatory Clinic Nurses
Impact of Clinical Reminder Redesign on Learnability, Efficiency, Usability, and Workload for Ambulatory Clinic Nurses, Jason J. Saleem, PhD, J Am Med Inform Assoc. 2007;14:632-640. DOI 10.1197/jamia.M2163.
Question: Would a redesigned Clinical Reminder (CR) system be more ‘learnable’ for first time users and be more efficient and perceived as having better usability for all users compared to the original design.
Methods: Sixteen nurses were recruited and given a $50 gift certificate for participation. A simulated clinical work station presenting both versions (A, the original and B, the redesigned) was implemented. Time stamped video analysis was captured in addition to survey responses from the subjects. Both versions were produced to display similarly. The changes included: 1) the reminders were prefaced with a ‘P’ or ‘N’ for primary care provider or nurse, 2) the CR dialog box were accessible directly from the cover sheet, 3) the CR dialog boxes were standardized and information pertaining to the reason for the reminder being due was presented in the box, and 4) an electronic visit checklist was added summarizing what needed to be completed at that visit.
Protocol: Time to completion of a pain screening CR using a counter-balanced presentation to the participants was the surrogate for learnability. Then a five minute orientation was provided followed by a counter-balanced introduction of five clinical scenarios. Each scenario was presented in the A and B design to each participant. Only the time recorded for the first scenario encountered was used for analysis. Time to completion of each scenario was the marker for efficiency. The NASA Task Load Index (TLX), a questionnaire to measure perceived workload, was administered after each of the ten scenarios. The Likert-type usability questionnaire was administered when the participant completed all ten scenarios.
Results: The new CR had easier ‘learnability’. The time in seconds to complete the first CR presented was statistically shorter with version B, p<0.001.
Two of the five patient scenarios were completed in statistically significant shorter time; p=0.02 for the scenario including alcohol use screen, IHD aspirin therapy use, hypertension screen / BP check, nutrition / obesity screen and p=0.007 for the scenario containing hypertension screen / BP check, pneumococcal, tobacco screen.
CR version B scored significantly better on overall ease of use and overall satisfaction, p<0.05 (two-tailed). This system was also statistically better on the 13 detailed usability constructs.
The redesigned system was statistically better in the areas of mental demand, p=0.04 and frustration, p=0.03 (one-tailed).
A positive unanticipated effect was the ability to view the summarized patient information during the CR tasks. A negative unanticipated effect was confusion about the term ‘When’ in the detailed explanation for the reminder.
Discussion: The redesigned interface of CRs employing four modifications improved ease of use for first time users and it increased efficiency and usability. Re-labeling “When” in the detailed explanation of the CR with “Recommended Frequency” would likely resolve the negative effect in that part of the redesign. The authors recommend immediate implementation in the national VHA CPRS software of: 1)accessing CR dialog boxes by clicking directly on the names of the CRs on the interface prior to opening a progress note, 2) preface the CRs with either a ‘P’ or ‘N’ for primary care provider or nurse, and 3) standardize CR dialog formats.
Limitations: “A simulation study does not capture the full complexity of a sociotechnical system as it exists in real life.” Further, this study only applies to nurses. Additional evaluation to study providers and also provider and nurse interactions is needed.
Conclusions: Modest modifications to the existing CR format can significantly improve the ability of first time users to complete the CRs. These changes also improve efficiency and usability which leads to improved adoption and decreased workarounds. The findings may be of use to organizations transitioning from paper-based to electronic medical records or to organizations considering the addition of decision support.
Michael Lees BMI 512 Winter 2008