A Framework for Evaluating the Appropriateness of Clinical Decision Support Alerts and Responses

This is a review of McCoy et al's 2012 article, A Framework for Evaluating the Appropriateness of Clinical Decision Support Alerts and Responses.

Background

The authors noted that review of literature has shown that current methods of analyzing clinician responses to clinical decision support (CDS) alerts are not comprehensive. Their goal is to create a framework for evaluating the appropriateness or inappropriateness of alert overrides that could be used to evaluate other clinical decision support systems.

Methods

The authors developed the evaluation framework that considered both alert appropriateness measures and provider response appropriateness measures. The framework was then used to test a CDS for medication management of acute kidney injury (AKI) that the authors had previously launched. Their sample consisted of 300 patients that had AKI CDS alerts from November 2007 to October 2008. Two nephrologists analyzed the alert responses, with another nephrologist to adjudicate any subject matter disagreements.

Results

The authors noted that while only 20% of the CDS alerts were inappropriate, 82% of alerts were overridden. But because this evaluation framework was set to measure clinician alert response not only during the immediate ordering timeframe, but tracked modifications for the alerted medication order even after 24 hours, the authors reported that clinicians later responded appropriately to CDS alerts by modifying or discontinuing the orders, which lowered the inappropriate clinician overrides to 17%.