Performance of a Web-Based Clinical Diagnosis Support System for Internists
Performance of a Web-Based Clinical Decision Support System for Internists
Mark L. Graber, MD and Ashlei Mathew
BACKGROUND: Clinical Diagnosis Support Systems (CDSS) are aimed at assisting clinicians make the correct diagnosis and reduce medical errors. The early systems, ( e.g. QMR, Iliad, DXplain) proved to be somewhat useful but very time-consuming and with limited sensitivity and specificity. A new, second generation, Web-based CDSS called “Isabel” was lunched in the early 2000s. Its full name is “Isabel Diagnosis Reminder and Knowledge Mobilizing System” and is used for both pediatric and adult internal medicine cases. It accepts key clinical findings or whole-text entries. It uses natural language processing and is linked to major medical journals and textbooks. It interfaces with leading outpatient and inpatient EMR systems.
According to the developer, Isabel Health Care, Inc., the CDSS Isabel “has undergone a robust peer-reviewed validation process over 7 years to demonstrate its accuracy, effectiveness, and value.” In 2005 it received the HIT Product Innovation Award from Frost & Sullivan, a global growth consulting company. It is used by hospitals and academic centers in the US and abroad.
OBJECTIVE: This article describes the investigator’s evaluation of Isabel in the diagnoses of complex internal medicine cases involving adults. The effectiveness of this CDSS and its speed are both considered.
METHOD: The investigator tested 50 consecutive internal medicine case records published in the New England Journal of Medicine. The first method involved the entering of 3-6 key clinical findings from the case. The second method used whole-text entries from the entire case history. With both methods, the intent was to establish how often the correct diagnosis was included in the list of 30 diagnoses provided by Isabel.
RESULTS: By entering key clinical findings, the correct diagnosis was suggested in 48 out of 50 cases (96%). With the entire history pasted in, it was in 37 out of 50 cases (74%). Both methods were very fast and yielded results in 2-3 seconds.
CONCLUSIONS: The sensitivity in the first generation CDSSs was in the range of 50%-60% and took several minutes to produce results. Although Google has been suggested as an alternative due to its speed, it has been found that its sensitivity is as bad as the old CDSSs. However, Isabel was found to be both extremely fast and with high sensitivity, especially when entering key clinical findings instead of whole text. Since Isabel performed so well in the experimental setting, the investigator recommended further evaluation in the natural environment of clinical practice.
Submitted by Nicolas Thireos
