View source for On the alert: future priorities for alerts in clinical decision support for computerized physician order entry identified from a European workshop

This was adopted from the Coleman JJ et al.'s article "On the alert: future priorities for alerts in clinical decision support for computerized physician order entry identified from a European workshop".<ref> Coleman JJ et al. On the alert: future priorities for alerts in clinical decision support for computerized physician order entry identified from a European workshop. ''BMC Medical Informatics and Decision Making 2013, 13:111''. doi:10.1186/1472-6947-13-111. http://www.biomedcentral.com/1472-6947/13/111</ref>

= '''BACKGROUND''' =
== Computerized physician( or provider) order entry([[CPOE]]) and clinical decision support([[CDS]])[http://en.wikipedia.org/wiki/Clinical_decision_support_system] ==
CPOE systems allow users to prescribe using a computer system,reducing the risk of [http://www.slu.edu/Documents/nursing/LavinPrescribingErrors.pdf prescribing errors] resulting from illegible handwriting or transcription errors. They also have shown to reduce [[medication errors]] and adverse drug events([[ADE]]s) in hospitals. <ref>Ammenwerth E, Schnell-Inderst P, Machan C, Siebert U: The effect of electronic prescribing on medication errors and adverse drug events: a systematic review.
J Am Med Inform Assoc 2008, 15(5):585-600</ref> <ref>Hug BL, Witkowski DJ, Sox CM, Keohane CA, Seger DL, Yoon C, Matheny ME, Bates DW: Adverse drug event rates in six community hospitals and the potential impact of computerized physician order entry for prevention</ref> <ref>Reckmann MH, Westbrook JI, Koh Y, Lo C, Day RO: Does computerized provider order entry reduce prescribing errors for hospital inpatients? A systematic review.
J Am Med Inform Assoc 2009, 16(5):613-623</ref> <ref>Shamliyan TA, Duval S, Du J, Kane RL: Just what the doctor ordered. Review of the evidence of the impact of computerized physician order entry system on medication errors</ref> <ref>van Doormaal JE, van den Bemt PM, Zaal RJ, Egberts AC, Lenderink BW, Kosterink JG, Haaijer-Ruskamp FM, Mol PG: The influence that electronic prescribing has on medication errors and preventable adverse drug events: an interrupted time-series study</ref> <ref>Wolfstadt JI, Gurwitz JH, Field TS, Lee M, Kalkar S, Wu W, Rochon PA: The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review</ref> CPOE systems often have integrated CDS which has the potential to improve clinicians' decisions through guidance, [[Alerts|alerts]], and reminders. In principle, clinicians support the idea of CDS alerts in identifying and preventing erroneous or less optimal prescribing. <ref name="ko">Ko Y, Abarca J, Malone DC, Dare DC, Geraets D, Houranieh A, Jones WN, Nichol WP, Schepers GP, Wilhardt M: Practitioners' views on computerized drug-drug interaction alerts in the VA system</ref> <ref name="ammenwerth">Ammenwerth E, Jung M: Expectations and barriers versus cxCDSS-CPOE: a European user survey. In Proceedings of the PSIP International Workshop, Sofia, Bulgaria, 23 June 2011. Edited by Beuscart R, Tcharaktchiev D, Angelova G. Shoumen: Incoma; 2011:49-52</ref> <ref name="lapane">Lapane KL, Waring ME, Schneider KL, Dubé C, Quilliam BJ: A mixed method study of the merits of e-prescribing drug alerts in primary care.
J Gen Intern Med 2008, 23(4):442-446</ref> <ref name="weingart">Weingart SN, Simchowitz B, Shiman L, Brouillard D, Cyrulik A, Davis RB, Isaac T, Massagli M, Morway L, Sands DZ, et al.: Clinicians' assessments of electronic medication safety alerts in ambulatory care</ref>

== Alert specificity and sensitivity ==
In a CDS system,sensitivity is the ability of the system to alert prescribers correctly when patients are risk of experiencing drug-induced harm. The specificity of the CDS system is a measure of it's ability to distinguish between events that cause harm and non-events that will not. Safe alerting systems should have high specificity and sensitivity, present clear information, not unnecessarily disrupt workflow, and facilitate safe and efficient handling of [[alerts]]. <ref>van der Sijs H, Aarts J, Vulto A, Berg M: Overriding of drug safety alerts in computerized physician order entry.
J Am Med Inform Assoc 2006, 13(2):138-147</ref> 

== Knowledge of [[alert fatigue]] in CDS systems ==
CDS alerts have the potential to cause harm to patients by occurring too frequently. <ref name="ko"></ref> <ref name="ammenwerth"></ref> <ref name="lapane"></ref> <ref name="weingart"></ref> In most systems, majority of the alerts are overridden. <ref>Quinzler R, Schmitt SP, Pritsch M, Kaltschmidt J, Haefeli WE: Substantial reduction of inappropriate tablet splitting with computerised decision support: a prospective intervention study assessing potential benefit and harm.
BMC Med Inform Decis Mak 2009, 9:30</ref> <ref>van der Sijs H, Aarts J, Vulto A, Berg M: Overriding of drug safety alerts in computerized physician order entry.
J Am Med Inform Assoc 2006, 13(2):138-147</ref> <ref name="seidling"><Seidling HM, Schmitt SP, Bruckner T, Kaltschmidt J, Pruszydlo MG, Senger C, Bertsche T, Walter-Sack I, Haefeli WE: Patient-specific electronic decision support reduces prescription of excessive doses.
Qual Saf Health Care 2010, 19(5):e15</ref> <ref>Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS: Physicians' decisions to override computerized drug alerts in primary care.
Arch Intern Med 2003, 163(21):2625-2631</ref> Exposure to frequent false alarms can desensitize users so that they ignore and increasingly mistrust alarms. <ref>Getty DJ, Swets JA, Pickett RM, Gonthier D: System operator response to warnings of danger: a laboratory investigation of the effects of the predictive value of a warning on human response time</ref> Most of the focus on reducing override rates in CDS systems considers strategies such as the customization of the third party providers' set of alerts, <ref>Del Beccaro M, Villanueva R, Knudson K, Harvey E, Langle J, Paul W: Decision Support Alerts for Medication Ordering in a Computerized Provider Order Entry (CPOE) System: a systematic approach to decrease alerts.
Appl Clin Inform 2010, 1:346-362</ref> <ref>Resetar E, Reichley RM, Noirot LA, Doherty JA, Dunagan WC, Bailey TC: Strategies for reducing nuisance alerts in a dose checking application.
AMIA Annu Symp Proc 2005, 624-628</ref> <ref>Resetar E, Reichley RM, Noirot LA, Dunagan WC, Bailey TC: Implementing daily dosing rules using a commercial rule base.
AMIA Annu Symp Proc 2006, 1073</ref> <ref>Reichley RM, Seaton TL, Resetar E, Micek ST, Scott KL, Fraser VJ, Dunagan WC, Bailey TC: Implementing a commercial rule base as a medication order safety net.
J Am Med Inform Assoc 2005, 12(4):383-389</ref> implementation of highly specific algorithms <ref name="seidling"></ref> and use of tiered severity to stratify and reduce the number of interruptive alerts.<ref>Paterno MD, Maviglia SM, Gorman PN, Seger DL, Yoshida E, Seger AC, Bates DW, Gandhi TK: Tiering drug-drug interaction alerts by severity increases compliance rates.
J Am Med Inform Assoc 2009, 16(1):40-46</ref> <ref>Anton C, Nightingale PG, Adu D, Lipkin G, Ferner RE: Improving prescribing using a rule based prescribing system.
Qual Saf Health Care 2004, 13(3):186-190</ref> Other suggested strategies to counteract the alert fatigue include turning off frequently overridden alerts and directing time-dependent [[drug-drug interaction]] alerts to nurses.<ref>van der Sijs H, Lammers L, van den Tweel A, Aarts J, Berg M, Vulto A, van Gelder T: Time-dependent drug-drug interaction alerts in care provider order entry: software may inhibit medication error reductions.
J Am Med Inform Assoc 2009, 16(6):864-868</ref> <ref>van der Sijs H, Aarts J, van Gelder T, Berg M, Vulto A: Turning off frequently overridden drug alerts: limited opportunities for doing it safely.
J Am Med Inform Assoc 2008, 15(4):439-448</ref> Despite various improvement strategies,alert fatigue continues to occur and frustrate users.<ref>Perna G: Clinical alerts that cried wolf. As clinical alerts pose physician workflow problems, healthcare IT leaders look for answers.
Healthc Inform 2012, 29(4):18, 20</ref> To address the issues, European experts on CDS attended a workshop in Birmingham,UK where they agreed on a consensus on the current gaps in the research and the challenges of improving alerting in CDS systems.

= '''METHOD''' =
Researchers with a strong publication record in the field of CDS were identified and were invited to attend a two day workshop in Birmingham,UK in November 2011. The objectives of this workshop were:
# to identify key knowledge gaps in the study of CDS-based alerting;
# to identify research priorities on CDS-based alerting; and
# to identify research methodologies to evaluate alerts

= '''RESULTS''' =
== Knowledge gaps in the study of alerts in CDS systems ==
# [[Basic_statistical_concepts#Sensitivity|Sensitivity]] and [[Basic_statistical_concepts#Specificity|specificity]] of a CDS system
# Presentation and personalization of alerts
# Timing of alerts
# Relevance of the outcome measures in the study of alerts
# Measurement of the quality of alerts
# Design and firing of alerts/rules
# Legal issues- This was discussed in a American context,<ref>Kesselheim AS, Cresswell K, Phansalkar S, Bates DW, Sheikh A: Clinical decision support systems could be modified to reduce 'alert fatigue' while still minimizing the risk of litigation.
Health Aff (Millwood) 2011, 30(12):2310-2317</ref> with particular emphasis on the liability implications of CDS with drug-drug interactions <ref>Kesselheim AS, Cresswell K, Phansalkar S, Bates DW, Sheikh A: Clinical decision support systems could be modified to reduce 'alert fatigue' while still minimizing the risk of litigation.
Health Aff (Millwood) 2011, 30(12):2310-2317</ref> <ref>Ridgely MS, Greenberg MD: Too many alerts, too much liability: sorting through the malpractice implications of drug-drug interaction clinical decision support.
St Louis Univ J Health Law Pol 2012, 5:257-296</ref>
# [[Human Factors in health care|Human factors]] and [[usability]]

== Important research priorities ==
=== Determine the optimum sensitivity and specificity of a CDS system in practice ===
A perfect CDS system would be both 100% specific and 100% sensitive. Current systems tend to have high sensitivity but low specificity.<ref>Weingart SN, Simchowitz B, Padolsky H, Isaac T, Seger AC, Massagli M, Davis RB, Weissman JS: An empirical model to estimate the potential impact of medication safety alerts on patient safety, health care utilization, and cost in ambulatory care.
Arch Intern Med 2009, 169(16):1465-1473</ref> Sensitivities below 100% are risky and may contribute to patient harm, especially for the most injurious events.
It is important that the system is able to draw in additional information from beyond the [http://en.wikipedia.org/wiki/Knowledge_base knowledge base](KB) to increase specificity,for example through the integration of individual patient information such as lab values and co-morbidities with information on medicines.<ref>Troiano D, Jones MA, Smith AH, Chan RC, Laegeler AP, Le T, Flynn A, Chaffee BW: The need for collaborative engagement in creating clinical decision-support alerts.
Am J Health Syst Pharm 2013, 70(2):150-153</ref> <ref>Coleman JJ, Nwulu U, Ferner RE: Decision support for sensible dosing in electronic prescribing systems.
J Clin Pharm Ther 2011, 37(4):415-419</ref> <ref>Ferner RE, Coleman JJ: An algorithm for integrating contraindications into electronic prescribing decision support.
Drug Saf 2010, 33(12):1089-1096</ref> <ref>Seidling HM, Storch CH, Bertsche T, Senger C, Kaltschmidt J, Walter-Sack I, Haefeli WE: Successful strategy to improve the specificity of electronic statin-drug interaction alerts.
Eur J Clin Pharmacol 2009, 65(11):1149-1157</ref> The challenge is in ensuring that drug information is accurate,comprehensive and up-to-date,whilst keeping the process manageable in terms of expertise,time and resources. One solution may be the collaborative development and sharing of KBs between countries.<ref>Böttiger Y, Laine K, Andersson ML, Korhonen T, Molin B, Ovesjö ML, Tirkkonen T, Rane A, Gustafsson LL, Eiermann B: SFINX-a drug-drug interaction database designed for clinical decision support systems.
Eur J Clin Pharmacol 2009, 65(6):627-633</ref> <ref>Kawamoto K, Hongsermeier T, Wright A, Lewis J, Bell DS, Middleton B: Key principles for a national clinical decision support knowledge sharing framework: synthesis of insights from leading subject matter experts.
J Am Med Inform Assoc 2013, 20(1):199-207</ref> 
However, system quality may differ with regards to different alert categories, and differences when alerting for medications only,as opposed to a combination of medications and patient parameters.<ref>van der Sijs H, Bouamar R, van Gelder T, Aarts J, Berg M, Vulto A: Functionality test for drug safety alerting in computerized physician order entry systems.
Int J Med Inform 2010, 79(4):243-251</ref> <ref>van Doormaal JE, Rommers MK, Kosterink JGW, Teepe-Twiss IM, Haaijer-Ruskamp FM, Mol PGM: Comparison of methods for identifying patients at risk of medication-related harm.
Qual Saf Health Care 2010, 19:1-5</ref> By comparing differences in the design of current systems,it may be possible to identify a gold standard on which to base future CPOE systems.

=== Determine whether personalization of alerts will reduce alert fatigue ===
Customization of the setting in which the system is used could provide an opportunity to eliminate inappropriate alerts and requires further evaluation. This may improve usability and receptivity of CDS alerts.
Allowing individual users to personalize the interface design,like in smartphones,of CDS alerts may also reduce alert fatigue. Personalization of alerts may also be done in a automatic way based upon a user's familiarity with certain risk situations,training and expertise. For example,frequent users may require fewer alerts. The development of individualized alerts will require structured and systematic design to ensure that they are appropriately generated for each patient.

=== Determine whether appropriate timing of an alert within the prescribing process will reduce alert fatigue ===
Ideally, alerts should be displayed very early in the prescribing process. Alerting the user as early as possible and having complete information that can be integrated into a single CDS alert are not easily compatible. The authors described a grading such as:
* prescribing absolutely absolutely contraindicated;
* prescribe but only if certain conditions are met; and
* prescribe where benefits outweigh harm.
New research should focus on assessing the impact of the timing and number of alerts generated during one drug prescription.

=== Determine the relevance of the outcome measures in evaluating alerts ===
# Patient harm: This entails identifying harm specific to the prescribing process that may be prevented by the CDS
# Length of stay in the hospital: Easily measured but depends on several factors other than the quality of prescribing
# Mortality: Easily measured but depends on several factors other than the quality of prescribing
# Quality measures as stated by National Quality Forum,USA [http://en.wikipedia.org/wiki/National_Quality_Forum]<ref>Driving Quality and Performance Measurement—A Foundation for Clinical Decision Support. A Consensus Report.
[http://www.qualityforum.org/Publications/2010/12/Driving_Quality_and_Performance_Measurement_-_A_Foundation_for_Clinical_Decision_Support.aspx </ref> 
# Measures of clinical improvement
# Medication errors <ref>Ferner RE, Aronson JK: Clarification of terminology in medication errors: definitions and classification.
Drug Saf 2006, 29(11):1011-1022</ref>
# Costs
The correct balance needs to be established.

== Research methods to evaluate alerts ==
=== Expert opinion ===
Expert opinions may provide information on some research gaps,such as determining which outcome is most relevant to the specific research question.

=== Observations(Naturalistic) studies ===
This involves careful observation and recording of behaviors and events in their natural setting and can be very powerful when based on strong theoretical foundations. Such studies have an important role in the study of alerts in CDS systems.An evaluation of the outcomes pre- and post-introduction of a CDS system in one setting may provide useful information in the effect of CDS alerts.

=== Experimental studies ===
These can allow for the manipulation and testing of CDS alerts in a controlled environment. This may be the most practical way to investigate the ideal level of sensitivity and specificity,as well as determining the effects of the personalization of alerts for the user. However, they may not take into account the effect of stressful working environments on the user.

== Challenges to implementing research methods ==
The sequencing of events within the study design is important: the effectiveness of a naturalistic system depends on having a suitable [[Audit trail|audit trail]] and capturing the data reproducibly. Randomization and masking in experimental studies can be difficult in such circumstances. <ref>Chuang JH, Hripcsak G, Heitjan DF: Design and analysis of controlled trials in naturally clustered environments: implications for medical informatics.
J Am Med Inform Assoc 2002, 9(3):230-238</ref> There are also screen capturing problems with the naturalistic method. It would be beneficial to work with organizations that are looking to implement CPOE/CDS and monitor the steps to full implementation. One of the approaches could be a purposeful synthesis /integration of different studies leading to new insights. If supported by an expert panel,an iterative prospective study may be valuable.

= '''DISCUSSION''' =
== Summary of findings ==
The authors have identified several research priorities including:
# the need to determine the optimal sensitivity and specificity of alerts;
# whether adaptation to the environment or characteristics of the user may improve alerts; and
# whether timing and number of alerts improves alerts.
They recommend that the reduction of alert fatigue may be possible through the integration of patient, illness and medicine information, and through the development of an alert hierarchy. Specificity can be increased without sacrificing sensitivity through integration of and linkage of solid KBs and patient parameters,using well-tailored algorithms.

== Strengths and limitations of the approach ==
Workshop participants were experts in the use of CDS. However majority of the participants are academic researchers and other potentially relevant groups,such as CDS vendors,are not consulted.

= '''CONCLUSIONS''' =
The use of CDS systems within the CPOE is becoming an essential component of patient care. Research should be undertaken to determine whether the use of CDS alerts really improves patient outcomes,suing appropriate methodologies and appropriate outcome measures. Strategies must be developed to reduce the burden of CDS alerts without compromising patient safety.

= '''COMMENTS''' =
Though very useful for patient care, CDS alerts have the potential to harm the patient by causing alert fatigue. The authors did commendable work in identifying the knowledge gap parameters and the research criteria to accomplish that,especially by improving CDS sensitivity and specificity and also improve the timing and number of alerts. Further research is advised by the authors which hopefully leads to a CDS system which has 100% sensitivity and 100% specificity.

= '''REFERENCES''' =
<references/>
[[Category: Reviews]]
[[Category: CDS]]
[[Category: CPOE]]