Rationale, design, and implementation protocol of an electronic health record integrated clinical prediction rule (iCPR) randomized trial in primary care

From Clinfowiki
Jump to: navigation, search

This is a review of research article by Mann et al.(2011), titled 'Rationale, design, and implementation protocol of an electronic health record integrated clinical prediction rule (iCPR) randomized trial in primary care'.


Evidence Based Medicine (EBM) has well validated Clinical Prediction Rules (CPR) that have not found their way into Electronic Health Records (EHR) in aiding Clinical Decision Support (CDS) due to inaccessibility at point-of-care. These are often considered by clinical users to be markers of quality care. Mann et al., searched literature about the integrated clinical prediction rules (iCPRs) but found none of integrated CPRs into EHRs in the ambulatory setting, and only one of proposed integrations in the inpatient setting. Hence they chose two well-validated CPRs (namely Walsh CPR for Streptococcal Pharyngitis and the Heckerling CPR for Pneumonia) and developed integrated clinical prediction rules (iCPRs) into an outpatient EHR.[1]


This iCPR study was designed to test the plausibility of initiating the strep throat and pneumonia CPRs into the EHR in a primary care practice. The aims strengthening this goal were to assess adoption and impact of the iCPR program in primary care.[1]


Prototype development An interdisciplinary team of expertise in CPRs, primary care, usability, clinical informatics, and in- depth knowledge of the pros and cons of CDS in the commercial EHR together planned the first prototype iCPR in about 3 months.[1]

The team contemplated following in designing the tool

Technical Considerations

Assessment Tool

After proper scrutiny of vendor opinions, provider familiarity with the vendor’s CDS tools, and provider workflow the team decided to go with dynamic flowsheets for calculations with minimized ‘clicks’ and manual data entry.[1]

Restriction of alerts

iCPR was activated only to outpatient primary care physicians in intervention group using EHR so as to integrate a seamless workflow.[1]

Alerts, overrides and triggers

Considering all the factors around alerts, the development team opted a combination of two-step mandatory alerts one was an early-in workflow passive and a later-in-workflow active. The finalized trigger points after rigorous discussion for the tool were one of three workflow locations: chief complaint, relevant and specific encounter diagnoses, or a less specific encounter diagnosis in combination with a relevant antibiotic order. The early-in-workflow passive mandatory alert triggered from the chief complaint, while the later-in-workflow active mandatory alert triggered from diagnosis and/or orders.[1]

Risk calculator

The team quickly realized that dynamic flowsheets are crucial for use to automatically calculate the risk probabilities and provide recommendations supported by validated CPRs.[1]

Bundled order sets, documentation, and patient instructions

The team linked bundled order sets to each of the probable risk states calculated by the iCPR tool. Three levels of the iCPR were created for strep throat–low-, intermediate-, and high-risk. Low risk led to a bundled order set without antibiotics, intermediate led to a workflow with rapid strep as the next step (with resulting low- or high-risk order sets), and high risk led to a bundled order set with pre-populated suggested antibiotic orders. The pneumonia iCPR had identical format but with only low- and high-risk states. Clinical experts teemed each bundled order set with the most common orders prescribed for strep throat and outpatient pneumonia treatment .They also teemed progress note of the visit and auto generated patient instructions in both English and Spanish.[1]

Usability testing

The usability testing was done in two rounds in five main domains: alerting, risk calculator, bundled ordering, progress note, and patient instructions with 8 providers in each round using ‘think aloud’ and thematic protocol analysis procedure and time-series analytic procedure respectively. [1]

Trial design

Practice setting

This randomized trial at a large urban academic medical center with population of over 55,000 visits annually and serving a major population of Hispanic, followed by African-American, white and others. The sample size was calculated at 0.05 significance and 80% power which estimated a total of 1,070 study subjects with half in each disease group.[1]

Provider eligibility, consent, and randomization

The randomized control trial included all primary care providers within the medical practice in outpatient unit of academic medical center. Faculty providers and any medical residents were assigned as 1:1 ratio into intervention and control groups. Only those providers in intervention group had accessibility to iCPR tool.[1]

Provider Training

All providers in the intervention were trained approximately 45 minutes on the integration of iCPRs and the output of each iCPR.[1]


The team formed a process measurement battery to evaluate the uptake of the iCPR tool by providers and to record the utilization of each part of the tool.[1]


The outcome measurement battery was designed to detect changes in clinical practice that are most likely to result from use of the iCPRs. The primary outcome was the difference in antibiotic prescribing frequency among patient encounters eligible for the iCPR tool among intervention compared to control providers. The secondary outcome measure was number of rapid strep tests and throat cultures ordered between intervention and control providers. The process outcome was percentage of eligible encounters accepting iCPR and bundled order set usage.[1]

Data monitoring and quality control

Weekly reports, respective diagnostic triggers and periodic chart reviews were conducted to monitor the appropriateness of tool triggering and to investigate any concerns raised by providers regarding usability or workflow disruptions.[1]

Statistical Analysis

Each component of the iCPR tool was evaluated using the t-test, Wilcoxon test, or the chi-square test in intervention and control groups. To test the effect of iCPR, a generalized estimating equation model was utilized with clinician as the cluster variable, antibiotic prescribing as the outcome variable, and intervention group as the only explanatory variable.[1]


Initially a rapid response team of informatics and clinical expertise supported via pager for the first week. Later they placed an option of messaging to team at run time. Furthermore, the lead clinician was available at floor level. Lastly, periodic focus groups were held to elicit users’ feedback for ongoing improvements.[1]


EBM features were successfully integrated into CDS tool with an excellent multi-disciplinary development team who worked with in vivo usability in contrast to traditional methods and standardized training which are most important factors in enhancing user acceptance.[1]


This approach highlights a more user-centered design of CDS, early involvement of providers in incorporating the EBM at the point of care. one that maximizes provider input and likely acceptance. These lessons should be generalized more broadly in CDS development of EBM and other point-of-care CDS tools.[1]


Evidence based medicine and its benefits is still misspent in healthcare domain. Even the very well validated CPRs are available, appropriate integration into CDS and training and standardized evaluation procedures are proven crucial through this trial.


  1. 1.00 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09 1.10 1.11 1.12 1.13 1.14 1.15 1.16 1.17 1.18 Mann, D.M., Kannry, J.L., Edonyabo, D., Li, A.C., Arciniega, J., Stulman,J., Romero,L., Wisnivesky, J., Adler, R., McGinn, T.G. Rationale, design, and implementation protocol of an electronic health record integrated clinical prediction rule (iCPR) randomized trial in primary care. Implementation Science 2011 6:109. Retrived from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3184082/