Enhancement of Clinicians' Diagnostic Reasoning by Computer-Based Consultation
This is a review for Friedman's Enhancement of Clinicians’ Diagnostic Reasoning by Computer-Based Consultation A Multisite Study of 2 Systems.
Computer-based diagnostic decision support systems (DSS) were developed to improve health care quality by providing accurate, useful, and timely diagnostic information to clinicians. However, most studies have emphasized the accuracy of the computer system alone, without placing clinicians in the role of direct users. A partnership between the DSS system and the clinician is necessary in order for the DSS to be considered a successful system. This study explores the extent to which consultations with DSSs improve clinicians’ diagnostic hypotheses in a set of diagnostically challenging cases.
The design was a partially randomized controlled trial conducted in a laboratory setting, using a prospective balanced experimental design in 1995-1998, using three academic medical centers, none of which were involved in the development of the DSSs. A total of 216 physicians: 72 at each site, including 24 internal medicine faculty members, 24 senior residents, and 24 fourth-year medical students. One physician’s data were lost to analysis. Two DSSs, ILIAD (version4.2) and Quick Medical Reference (QMR; version 3.7.1), were used by participants for diagnostic evaluation of a total of 36 cases based on actual patients. After training, each subject evaluated 9 of the 36 cases, first without and then using a DSS, and suggested an ordered list of diagnostic hypotheses after each evaluation. Diagnostic accuracy, measured as the presence of the correct diagnosis on the hypothesis list and also using a derived diagnostic quality score, before and after consultation with the DSSs.
Correct diagnoses appeared in subjects’ hypothesis lists for 39.5% of cases prior to consultation and 45.4% of cases after consultation. Subjects’ mean diagnostic quality scores increased from 5.7 (95% confidence interval [CI], 5.5-5.9) to 6.1 (95% CI, 5.9-6.3) (effect size: Cohen d = 0.32; 95% CI, 0.23-0.41; P = .001). Larger increases (P = .048) were observed for students than for residents and faculty. Effect size varied significantly (P = .02) by DSS (Cohen d = 0.20; 95% CI, 0.08-0.32 for ILIAD vs Cohen d = 0.45; 95% CI, 0.31-0.59 for QMR).
The study supports the idea that “hands-on” use of diagnostic DSSs can influence diagnostic reasoning of clinicians.The The larger effect for students suggests a possible educational role for these systems.
It is interesting to see how much the clinicians decisions were influenced by the DSS. Also of interest is the larger increase of correct diagnoses observed for the students than for residents. It could suggest clinicians be introduced to the DSS systems as part of their medical curriculum. With DSS one concern I have is that it may lead the clinician to incorrect diagnosis as it may become difficult to keep considering alternate diagnoses if one diagnosis seems probable leading to an early closure of mind. In real life clinicians do consult with each other and through these consultations and further testing reach a diagnosis. Both the rate of correct and incorrect diagnoses is too high in this study due to the fact that information from key diagnostic tests like biopsy etc. was not provided. Another concern during the study was where the exam took place. Since the subjects originated from an academic setting, there's no way to guarantee that the results on this study could be applied to other practice venues.
Add review here.
- CP, Elstein AS, Wolf FM, et al. Enhancement of Clinicians' Diagnostic Reasoning by Computer-Based Consultation: A Multisite Study of 2 Systems. JAMA. 1999;282(19):1851-1856. doi:10.1001/jama.282.19.1851.http://jama.jamanetwork.com/article.aspx?article id=192106
1. Dhiman, G. J., K. T. Amber, et al. (2015). "Comparative outcome studies of clinical decision support software: limitations to the practice of evidence-based system acquisition." Journal of the American Medical Informatics Association 22(e1): e13-e20.http://jamia.oxfordjournals.org/content/22/e1/e13.abstract
2. Berner, E. S. and M. L. Graber (2008). "Overconfidence as a cause of diagnostic error in medicine." The American journal of medicine 121(5): S2-S23.http://www.ncbi.nlm.nih.gov/pubmed/18440350
3. Berner, E. S. (2006). Diagnostic Decision Support Systems: Why Aren’t They Used More And What Can We Do About It? AMIA Annual Symposium Proceedings, 2006, 1167–1168.http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1839633/