Radiology order decision support: examination-indication appropriateness assessed using 2 electronic systems
This article was published in 2015 in Journal of the American College of Radiology
- Purpose: The goal of the study was to determine the effects of guideline implementation strategy using 2 commercial radiology clinical decision support (CDS) systems.
- Methods: The appropriateness and insurance dispositions of MRI and CT orders were evaluated using the Medicalis SmartReq and Nuance RadPort CDS systems during 2 different 3-month periods. Logistic regression was used to compare these outcomes between the 2 systems, after adjusting for patient-mix differences.
- Results: Approximately 2,000 consecutive outpatient MRI and CT orders were evaluated over 2 periods of 3 months each. Medicalis scored 60% of exams as “indeterminate” (insufficient information) or “not validated” (no guidelines). Excluding these cases, Nuance scored significantly more exams as appropriate than did Medicalis (80% versus 51%, P < .001) and predicted insurance outcome significantly more often (76% versus 58%, P < .001). Only when the Medicalis “indeterminate” and “not validated” categories were combined with the high- or moderate-utility categories did the 2 CDS systems have similar performance. Overall, 19% of examinations with low-utility ratings were reimbursed. Conversely, 0.8% of examinations with high- or moderate-utility ratings were denied reimbursement.
- Conclusions: The chief difference between the 2 CDS systems, and the strongest influence on outcomes, was how exams without relevant guidelines or with insufficient information were handled. Nuance augmented published guidelines with clinical best practice; Medicalis requested additional information utilizing pop-up windows. Thus, guideline implementation choices contributed to decision making and outcomes. User interface, specifically, the number of screens and completeness of indication choices, controlled CDS interactions and, coupled with guidance implementation, influenced willingness to use the CDS system.
The authors refer to several previous studies pointing out that the use of IT in healthcare resulted in practitioner performance improvements, complication and mortality rate reductions, and decrease of cost. This study was conducted to determine the effects of guideline implementation strategy using 2 commercial radiology clinical decision support (CDS) systems. The authors looked for exam appropriateness, insurance approvals, and user satisfaction..
The study was conducted at three stand-alone outpatient imaging centers that perform MRI and CT examinations. Twp CDS systems,Medicalis SmartReq and Nuance RadPor, were used to evaluate the appropriateness and insurance dispositions of MRI and CT orders during 2 different 3-month periods.
The study evaluated approximately 2,000 outpatient MRI and CT orders. One system, Nuance, showed significantly better result in scoring exams as appropriate (80% versus 51%, P < .001). This system also had better outcomes in predicting insurance reimbursement (76% versus 58%, P < .001). 19% of exams with low-utility rating were reimbursed; and only 0.8% of high- and moderate-utility exams were denied reimbursement.
The authors state that the difference between the 2 CDS systems was how they handle exams without relevant guidelines or with insufficient information. The better-performing system combined published guidelines with clinical best practice. The other system requested additional information utilizing pop-up windows. Decision making and outcomes were affected by guidelines implementation choices. The authors argue that simpler and easier user experience can lead to better system ratings and willingness to use CDS.
This study confirmed again that developers should be paying much attention to user interface design. Collecting feedback from test users should be done prior to implementation.
- Schneider E, Zelenka S, Grooff P, Alexa D, Bullen J, Obuchowski N. Radiology order decision support: examination-indication appropriateness assessed using 2 electronic systems. http://www-ncbi-nlm-nih-gov.ezproxyhost.library.tmc.edu/pubmed/25842015