Return on investment
Return on investment (ROI) analysis is a quantitative method of evaluation. It is used to assess the potential earnings outcome from potential investment or business project. ROI analysis can be used to assist in making business decisions about the future. The practice of using ROI to determine whether or not to move ahead with a project is called ROI Analysis. ROI is defined as the gain from an investment minus the cost of the investment all divided by the cost of the investment.
Contents
History
Return on investment analysis appears to have been in use for an extremely long time.
As published on the Project Management Institute (PMI) website by Gary R. Heerkens (MBA, CBM, PMP), "ROI is an evaluation of the incremental financial benefit a company expects to receive for a given amount of incremental expenditure. The term incremental has considerable significance here. ROI calculations are based entirely upon the economic change (both positive and negative) that would result from approving a particular project." [1]
Descriptions for the following calculations were defined for the following:
- Net present value (NPV)
- Internal rate of return (IRR)
- Payback period [1]
Principal use
Measuring the future gain from a given project or investment. In this case, the attempt to quantitatively assess the future performance of an IT project is the principal use.
One could approach the ROI from the perspective of the Institute of Medicine Report, Crossing the Quality Chasm
- Safe: Reducing adverse drug events, inappropriate testing
- Effective: Reducing drug costs through appropriate prescribing
- Efficient: Reducing drug, laborotory, or radiologic utilization
- Timely: Reducing wait times
- Patient-centered: Reducing length-of-stay while hospitalized
- Equitable: Provides data to demonstrate equal delivery
Advantages
Return on investment analysis is a performance measure which allows for a quantitative measure to be applied to a business case decision. This can help remove emotion from a decision by having a standard measure to compare one investment to another.
Shortcomings
All potential variables can be difficult to enter into the analysis. Return on investment analysis can be difficult to include all necessary variables and can not predict potential new variables, or how the investment may trigger unintended consequences. If the return on investment analysis is performed for a small pilot project, the analysis doesn’t always hold true for larger projects.
Evaluating success of EMR implementation
Dean Sittig, professor at UT Houston's School of Biomedical Informatics, has suggested a new set of criteria for determining for an EMR implementation. Based on Koch's Postulates and Hill's criteria for causation, these criteria are designed specifically for EMR evaluation.
- Must have the hardware and software available before the effect is identified.
- Need to at least estimate state of affairs before system is implemented…manual review
- Show that clinicians are actually using the system that could produce the effect.
- Show that the effect increases with increasing availability and usage of the system.
- Show that all obvious “alternative explanations” for the effect are false.
- Show the effect goes away when the system goes away.
- Show that a similar effect occurs when a similar system is installed and used at a similar facility.
A survey of approximately 400 Healthcare IT News readers was conducted on EHR systems. Survey results on nine vendors were reported on September 14, 2015. Readers shared what they enjoyed and what they would change.[2]
A review from the U.S. Department of Health and Human Services - Health Information Technology discusses how ROI can be achieved. It summarizes results conducted by other studies. " [3]
Examples in Informatics
Return on Investment for a Computerized Physician Order Entry System Rainu Kaushal, Ashish K. Jha, Calvin Franz, John Glaser, Kanaka D. Shetty, Tonushree Jaggi, Blackford Middleton, Gilad J. Kuperman, Ramin Khorasani, Milenko Tanasijevic, David W. Bates Brigham and Women's Hospital CPOE Working Group J. Am. Med. Inform. Assoc. 2006;13(3):261-266. PrePrint published May 1, 2006; doi:10.1197/jamia.M1984
Comments on Return on Investment (ROI) As It Applies to Clinical Systems Mark E. Frisse J. Am. Med. Inform. Assoc. 2006;13(3):365-367. PrePrint published May 1, 2006; doi:10.1197/jamia.M2072 [Full Text] [PDF]
S. Wang A cost-benefit analysis of electronic medical records in primary care. The American Journal of Medicine, Volume 114, Issue 5, Pages 397-403
Return on Investment (ROI) Estimates
While barriers of determining actual ROI for EMR implementations exist, companies such as Dr. Cloud EMR are providing EMR and EHR ROI estimates based on each practice's details. This however does not suggest that it is entirely accurate and is only an estimate. DrCloudEMR is built by DrCloud Healthcare Solutions Inc, a wholly owned EnSoftek, Inc. subsidiary. [65] There are 2 main postulates for ROI which KOSH’s postulate and Sir Austin Bradford Hill’s criteria for Causation. Kosh’s postulate for CIS is i. The system or feature must be present in every case in which the benefit is observed. ii. The system must be isolated from the organization. iii. The benefit must be reproduced when the system is implemented in a new organization. iv. We must demonstrate that the system was used in the new organization. Hill’s Criteria for Causation includes (a) Strength of Association (b) Consistency of findings (c) Specificity of Association (d) Temporality (e) Dose-response (f) Plausibility (g) Coherence (h) Experimental Evidence and Analogy.
(a) Strength of Association tells us that the greater the change observed, the more likely the association is to be causal (e.g. If a EHR system is implemented and the CPOE feature greatly reduces medication errors, we could say that the implementation of the system had a causal effect on the reduction of medication errors and the strength of association is great).
(b) Consistency of Findings explains that if a change has been observed by different groups in different places with different circumstances and systems, the change is valid, so to speak. For example, if Company A (London, England, UK) implements System A , Company B (Houston, TX, USA) implements System B, and Company C (Guadalajara, Jalisco, Mexico) implements System C, and all three companies reduce medication errors using their respective systems, we can, again say that the CPOE feature of EHR systems can help reduce medication errors. It is important to note that the more consistent findings amongst different groups in different places, the better.
(c) Specificity of Association requires us to ask if there are any other factors which may have affected the change that we've observed. In regards to medication errors being reduced, one would have to ask if CPOE was the only factor involved. If errors could have been reduced due to other mechanisms in place besides CPOE alerts (e.g. better workflow in departments, new policies, etc.), the specificity of association could be considered weak. Weak does not imply wrong, but it does mean that more research has to be initiated.
(d) Temporality addresses the evaluation after an EHR system is implemented. Temporality asks us "were there any changes AFTER the system was implemented?" Usually this is harder to prove due to lack of data prior to EHR implementation, however, Sittig rates temporality as "strong."
(e) Dose-Response asks if the size of changes are directly correlated with the increase of system use (e.g. were medication errors greatly reduced due to the use of many medication alerts in the EHR system?). Usually, there is a strong and direct correlation between system use and the reduction of medication errors, as one example of a dose response in an EHR system.
(f) Plausibility must be shown; There must be some way to demonstrate that the EHR system was used the way it was intended to deliver certain results (e.g. Physicians must have used clinical support decisions the way the EHR system intended to reduce medication errors, in order to demonstrate plausibility.)
(g) Coherence simply states that changes caused by EHR systems should be caused by other EHR systems elsewhere. So, if medication errors are reduced by the use of one EHR system and that happens with the use of many other EHR systems, coherence exists.
(h) Experimental Evidence and Analogy is proving that when the system is not used properly or at all, that certain changes stop. So, if an EHR system is not being used properly or at all (after initial proper use), does a rise in medication errors resume? Experimental evidence is hard to obtain after EHR implementation because it requires not using the system for quite some time (which many would view as wasted money).
ROI Healthcare Project Tools
In the Health Catalyst's Executive Report "How to Drive ROI in Your Healthcare Improvement Projects", a four-step process and tools for ROI analysis were identified. [4]
- Define the project and the business need
- State proposal clearly
- Justify the business need
- Begin to quantify ROI
- Identify all costs
- Estimate benefits
- Identify direct benefits
- Identify indirect or intangible benefits and set improvement targets
- Identify all revenue opportunities
- Document assumptions
- Perform a sensitivity analysis
- Identify risks and alternatives
- Recruit and train, plan, and implement
- Evaluate costs, revenue and direct benefits
- Ensure nothing else has changed
- Perform Financial ROI calculations
- Review ROI calculations with the team
- Make adjustments
- Monitor and ensure sustained results [4]
Cite error: <ref> tags exist, but no <references/> tag was found
