Outcomes assessment of clinical information system implementation: a practical guide

For two years, informatics journal clubs at the University of Maryland were used to conduct literature reviews of CIS implementations and outcomes evaluations over 3 decades. The findings were translated into this guide defining the “what, “how”, and “when” to examine the areas of outcomes.

Introduction

“What” to measure included; user satisfaction, clinical outcomes and financial impact. User satisfaction is a very common measurement of clinical information systems and a table was provided listing many resources. Clinical outcomes assessment is best measured using randomized controlled trials (RCT) but is not practical for most institutions. Instead, the authors suggest using related outcomes such as lowering medication errors, better quality of documentation, etc.

Results

Previous and current studies were summarized pointing out the positive and negative impact of CIS implementation. Financial impact outcomes assessed as Return on Investment (ROI) were defined and further recommended benefits and costs, both indirect and direct, which could be useful.

Several designs, both quantitative and qualitative, were recommended for how to assess CIS implementation outcomes. Advice on when to assess CIS implementation outcomes was given since the timing of evaluation is influenced by many factors.

Conclusion

This study provides useful recommendations and considerations on how to assess clinical information systems after implementation. The authors in this study provide a guide to outcomes assessment of CIS implementation specifically in the areas user satisfaction, clinical outcomes and financial impact. They also noted the need for more information in this area compared to other scientific fields since many CIS are implemented without outcomes assessment.