Effects of computerized clinical decision support systems on practitioner performance and patient outcomes
This study focuses on reviewing whether CDS alerts really improve practitioner performance and patient outcomes.
They included English-language randomized and nonrandomized trials with a contemporaneous control group that compared patient care with a CDSS to routine care without a CDSS and evaluated clinical performance (ie, a measure of process of care) or a patient outcome. They stipulated that the CDSS had to provide patient-specific advice that was reviewed by a health care practitioner before any clinical action. Studies were excluded if the system (1) was used solely by medical students, (2) only provided summaries of patient information, (3) provided feedback on groups of patients without individual assessment, (4) only provided computer-aided instruction, or (5) was used for image analysis. Studies assessing CDSS diagnostic performance against a defined gold standard were not included in this review unless clinical use of the diagnostic CDSS was also compared with routine care. Based on these criteria, they reevaluated all studies from their previous reviews for inclusion.
Of the 97 controlled trials assessing practitioner performance, the majority (64%) improved diagnosis, preventive care, disease management, drug dosing, or drug prescribing. However, the effects of these systems on patient health remain understudied—and inconsistent when studied. Fifty-two trials assessed patient outcomes, often in a limited capacity without adequate statistical power to detect clinically important differences. Only 7 trials reported improved patient outcomes with the CDSS, and no study reported benefits for major outcomes such as mortality. Surrogate patient outcomes such as blood pressure and glycated hemoglobin were not meaningfully improved in most studies.
It showed that CDS alerts do improve practitioner performance however, patient outcomes were still undermined. See also: Patient centered care
This study, however, is about 10 years old. The field has rapidly grown since then and it would be interesting to see if there are recent studies measuring if CDS alerts affect patient outcomes.
Dr Garg et al published a systematic review of controlled trials to determine the effectiveness of CDSS on practitioner performance and patient outcomes. They searched MEDLINE , EMBASE, Cochrane Library, Inspec, and ISI databases and identified 100 studies that trialled the effect of diagnostic systems, reminder systems, disease management systems and drug-dosing (prescribing) systems. In this review, I would only focus on their findings about disease management systems. Improvement in practitioner performance in disease management was defined as adherence to recommended practice guidelines and improvement in patient outcomes was defined as statistically significant positive effect in at least 50% of the outcomes measured.
They identified 40 disease management studies. They found that in 62% of the studies that evaluated practitioner performance, there was an improvement. On the other hand, only 27 studies evaluated patient outcomes, of which only 5 (18%) demonstrated improvements. After that, they provided an account of outcomes measures per specific area of disease management (i.e. diabetes care, cardiovascular prevention, thrombolytic prescribing, urinary incontinence, HIV management etc.) and provided the number and percentage of studies in each disease category that demonstrated either improvement in performance or patient outcomes.
The authors provide some useful comments about the success and failure of CDSSs. They found that in studies in which users were automatically prompted to use the system, demonstrated better performance compared with studies in which users were required to actively initiate the system. They also found that systems developed internally were significantly more successful than systems developed externally.
A very good review with rigorous methodology, broad scope and relatively simple outcome measures to synthesise. It cannot be regarded as a meta-analysis in which a pooled effect size could be calculated, because the CDSSs and outcome measures were quite different and heterogeneous in each primary study that they identified and therefore, each study had to be evaluated on its own. There are two other reasons that can justify the poor results of the studies about disease management CDSS: the first is that there is a dearth of evidence about the effectiveness of standard (not supported with a CDSS) disease management programmes, which might undermine the effectiveness of CDSSs developed for them; and second, most seemingly successful disease management programmes have not been evaluated under a controlled trialled condition. Non-inclusion of pre-post studies in this systematic review could result in under-representation of the literature in this area.
Finally, the authors of the systematic review did not provide a summary account of the main functionalities or technical specification of the CDSSs in each study. For a systematic review on the effectiveness of CDSSs, this data seems to be a key to better understand the reasons for success and failure of each system.
They review controlled trials assessing the effects of computerized clinical decision support systems and to identify study characteristics predicting benefit.
Teams of 2 reviewers independently abstracted data on methods, setting, CDS and patient characteristics, and outcomes.
Improved practitioner performance was associated with CDSSs that automatically prompted users compared with requiring users to activate the system.
Many CDSSs improve practitioner performance. To date, the effects on patient outcomes remain understudied and, when studied, inconsistent.
CDSSs are obvious beneficial tools to clinicians. However, patient health should be studied more as a result of these CDSSs
Haynes, R. B., & Wilczynski, N. L. (2010). Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: Methods of a decision-maker-researcher partnership systematic review. Implement Sci, 5(1), 12. http://www.biomedcentral.com/content/pdf/1748-5908-5-12.pdf
- Garg AX, Adhikari NJ, McDonald H, et al. Effects of Computerized Clinical Decision Support Systems on Practitioner Performance and Patient Outcomes: A Systematic Review. JAMA.2005;293(10):1223-1238. http://jama.jamanetwork.com/article.aspx?articleid=200503&resultClick=3