Vendor Selection Criteria
Choosing an electronic medical record (EMR) vendor is an important task, with many aspects to consider. An EMR can completely redesign a hospital's practice, but can also optimize it in ways a paper-based system could not. With over 400 EMR vendors in the market, selecting the best option could be overwhelming. After clearly defining the needs of the company, the next step is to evaluate which vendors more closely match your list of priorities.
Here is a list of some of the important considerations when choosing, installing, implementing and upgrading an EMR system. 
- 1 Dr. Sittig's Overview of EMR Vendor Selection
- 2 Core clinical features
- 3 IT and technical requirements
- 4 Vendor Selection Criteria: Future relationship with vendor
- 5 Vendor Selection Criteria: Certification and meaningful use
- 6 Vendor Selection Criteria: Vendor assessment
- 7 Go live support
- 8 Regional Selection Center EHR Selection Criteria
- 9 Basic EHR Criteria
- 10 Analyzing EHR Business Requirements
- 11 Product Requirements
- 12 New Non Traditional Approach Overview
- 13 Research Functionality
- 14 Cost and Budget
- 15 Misc considerations
- 16 EHR Evaluation Resources
- 17 References
Dr. Sittig's Overview of EMR Vendor Selection
- Make The Plan
- Identify Decision makers
- Set Goals
- Make a Checklist of what should the EMR accomplish
- Map your Workflow
- Do a thorough Scan of your environment
- Prioritize needs
- Make EHR Functionality Checklist
- Develop a Request For Proposal (RFP)
- Select RFP recipients
- For example up to 5 vendors
- Narrow the field
- EHR Evaluation Form
- EHR Vendor Demonstrations
- Narrow the field
- For example up to 3 vendors
- Ask additional questions to vendors
- Check references
- Examples: consulting KLAS, Gartner etc
- Rank the vendors
- Functionality vs cost vs vendor characteristics
- functionality can be the institution's most important function
- cost can include the total amount from hardware, software, training, and support
- vendor characteristics can be important traits that are aligned with the institution's core values 
- Vendor selection tools
- Functionality vs cost vs vendor characteristics
- Site visits
- Select a finalist (between the last 2 competitors)
- Verify Commitment
- Determine approval of selection committees and discuss choice will all the key stakeholders.
- If possible repeat the Demo to all the staffs
- For uncovered concerns, verify all the references and repeat verification steps if necessary
- Formal Contract Negotiation
- Ask vendors to spell out all the current as well as the future costs including the vendor time commitments for training and implementation
- Ask vendor to put source code in escrow just in case Vendor go out of business
- Before signing the software contracts,make sure you have familiar attorney to review
- Follow all the above process
- Know that the process takes time and do not rush because the end result can be expensive.
- Follow the process without skipping any steps.
Core clinical features
IT and technical requirements
Vendor Selection Criteria: Future relationship with vendor
Vendor Selection Criteria: Certification and meaningful use
Vendor Selection Criteria: Vendor assessment
Go live support
Regional Selection Center EHR Selection Criteria
The Regional Selection Center suggests that medical practices compare EHR vendors using the following criteria before selecting a specific product. 
- Will the vendor’s product accomplish key practice goals?
- Clarify EHR system start-up prices before selecting a vendor including costs associated with hardware, software, maintenance and upgrades, phase payment options, lab and pharmacy interfaces, connection to health information exchange, and customized quality reports.
- What implementation support does the vendor offer?
- What are the costs, roles, and responsibilities associate with the data migration strategy?
- What are your sever options?
- What is the products ability to integrate with other products?
- What are the privacy and security capabilities of the product and what is the back-up plan?
- How does the product link payments with EHR incentive rewards, which are necessary to meet the practices EHR implementation goals and milestones?
- What is the vendor’s stability and market presence?
- What is the cost to link the product to HIE?
- What are the costs associated with legal counsel for contract review versus open sources through medical associations?
Basic EHR Criteria
- ONC‐ATCB certification (Six certifying bodies ) 
- HIPAA privacy and security compliant 
- Meaningful use reporting
- Stage 1 (2011-2012) Data Capture and Sharing
- Stage 2 (2014) Advance Clinical Processes
- Stage 3 (2016) Improved Outcomes
- Ability to generate county, state, and federal reports
- Support HL7 messaging standard 
- Support Secure Sockets (SSL) digital certificate
- Audit trail capabilities
Analyzing EHR Business Requirements
The evaluation team or decision team should consist of clinical healthcare professionals including house staff, technical support professionals, administrators, and finance department team from all areas of the institution. Each person brings to the table a different perspective of usage from their daily job responsibilities. Institution-wide involvement creates awareness and knowledge of EMRs and their benefits, as well as an understanding of the upcoming modifications in the workflows. In summary, the following tasks must be followed before starting EHR business requirements analysis:
- Assemble an Evaluation Team
- Define the Product, Material or Service
- Define the Technical and Business Requirements
- Define the Vendor Requirements
- Publish a Requirements Document for Approval
- Is the EHR system HIE certified? The organization might need an EHR system that meets the national interoperability standards; a system with the capability of transferring health information within and across organizational and state boundaries. Implementing an HIE certified system will eliminate the need to create a custom interface in the future that may be very costly to the organization. An HIE certified system will "reduce adoption barriers due to high interface costs, low reliability, and unknown development costs for the vendor". 
- Does the EHR package come with a fully integrated Computer Practitioner Order Entry (CPOE) system?
- If you are a large academic teaching hospital, does the EHR meet the special It requirements? Specifically, does the EHR accommodate the numerous handoffs in care that are the result of resident education and regulatory requirements?
- An EMR at an academic medical center must be evaluated on its ability to conduct large-scale research with data capture and retrieval, as well as attaining regulatory compliance regarding billing.
Clinical Process Assessment and Improvement
- The EMR vendor, in response to requirements defined in the request for proposal (RFP), should describe a plan for evaluation and assessment of the as-is clinical processes and workflows, identification of needed improvements and a proposal for implementing workflow optimizations necessary to launch the EMR. 
New Non Traditional Approach Overview
New adaptive methods to be considered regarding the successful HIT (Healthcare Information Technology) implementation: R.T.Blake et al., did an interesting case study about The Christ Hospital which underwent a world class successful HIT implementation. The Christ Hospital (TCH) is 555 bed nonprofit acute care hospital in Cincinnati, Ohio. It has successfully implemented HIT by firmly focusing to the goals set and following a strategic yet non-traditional methods in every area, and was exceptionally exemplary in collaboration with the different stakeholders. For more information on The Christ Hospital visit their website.  There are many adoptable methods for Healthcare Organizations (HCO) to be learnt from TCH HIT Implementation.
- Stakeholder analysis
- Formal Request for Proposal (RFP) vs Non Request for Proposal (Non RPF)
- Separate vendors for each identified core IT implementation areas
- Transparency in communication with stakeholders for collaboration
- Personalization of HIT
It is important to recognize the role of key Healthcare Organizations (HCO) Stakeholders. Stakeholders in healthcare can be broadly divided into internal and external. Internal stakeholders consist mainly of physicians, nurse, hospital administrators while external comprise of IT providers, vendors, and consultants.
Physicians and nurses have direct contact with patient, more so nurses with monitoring, administering medications and discharge/after care services. There are many healthcare workers under the supervision of registered or licensed practical nurses.Hospital administrators have often challenging workforce, hospital missions, complex regulatory and day- to day operations which target directly or indirectly quality of patient care.Hospital administrators have often challenging workforce, hospital missions, complex regulatory and day- to day operations which target directly or indirectly quality of patient care. 
Separate vendors for each identified core IT implementation areas
Identification of core areas which are in dire need of HIT have to be recognized before the HIT implementation. And each area identified should be supervised by a different vendor.This can best be adopted in time bound projects and could prove cost effective.
Personalization of HIT
For the smooth work flow which is primarily patient centered, the HIT should focus on the user friendly for physicians and nurses who can handle them with ease, so they contribute to the successful patient’s outcomes.
Transparency in communication with stakeholders for collaboration
This is essential as there are various stakeholders involved, always giving scope for power struggles and may fail the implementation as well as integration and may eventually face resistance from end users.Enterprise governance with good leader even from non IT can work successfully if utilized from the start. Also, this will enable the focus on patient care rather than technology.Starting from vendor selection to discuss with them, the criteria upon which they were selected and capability to contribute as per needs, also prior feedback from end users, all stakeholders perspectives and opinions before, during and after implementation can build a strong interpersonal relationships with primary target of patient centered care environment.
The ability to have an openness for the reporting of problems in a particular level to the advisory committee/ any which can resolve them within hours rather than dragging them for days and weeks. These problems with solutions can be circulated among all the stakeholders.
- Does the vendor bid include the product's ability to be used as a tool for research. If "no", is an add-on solution for research capability needed and will it entail extra cost. 
- Does the EMR have informed consent alerts and documentation systems for clinical research?
- If the proposed installation site is a research hospital, what are the research capabilities of the EMR?
- How is research achieved?
- How are reports produced?
- How is data exported from the production system?
- Which database is used for reporting? For research? Vendor or other?
- Does the vendor provide natural language processing for entry data or document?
- Support for research billing including research orders (6)
- Does the EHR system provide data mining capabilities to support clinical research?
- Are there limits on the fields that researchers can use?
- How difficult is it to interface with the database?
- Does the EMR have smart search abilities to search through physician notes for specific keywords while extracting research data.
- Does the vendor assist in research endeavors, or is it left up to the institution?
- Does the EMR have the flexibility to capture documents necessary for specific clinical trials like adverse events, enrollments, and etc? 
- If research involves use of pathological/histological samples, does the EMR system tie into the laboratory management systems and database to allow for retrieval and processing of samples?
- Does the EMR system provide functionality to identify clinical related concepts?
- Does the EMR provide the flexibility to normalize the clinical concepts found in the document?
- Does the EMR provide the assistance to automatically generate the de-identified document for research purpose?
- Does EMR system provide functionality to implement study-screening parameter into patients’ registration, scheduling, medication lists, diagnostic codes (ICD) and clinical notes? 
- Does EMR system provide functionality to contact potential candidates and notify the patient’s providers of potential study eligibility? 
- Does EMR system provide capability to systematically captured study specific data and electronically exports deidentified study data to clinical trial electronic data capture system? 
- Vendor's ability to support research is especially important for large academic hospitals. In academic settings, research productivity is evaluated on publication and funding. To assess the ability of each vendor’s EMR to support research, the same two criteria for assessing productivity can be used— 1) publications that were the result of or 2) funded research made possible through EMR use or data analysis from the EMR’s data warehouse.  To assess this one can follow Mt. Sinai's example in selecting a vendor as follows:
- After a rigorous investigation of vendor statements and industry statements, a standardized questionnaire was used to consistently ask about items specific to the site contacted. Sites suggested by vendors, as well as community listservs, were contacted; the user community was engaged.
- Literature search:
- PubMed was used to search MEDLINE covering the fields of medicine, nursing, dentistry, veterinary medicine, the health care system, and the preclinical sciences.
- Also queried was CRISP (Computer Retrieval of Information on Scientific Projects), a searchable database of federally-funded biomedical research projects.
Patient Quality Improvement
- Link to patient education materials database or service and the ability to load practice specific documents for patient handouts. Must be able to access and print these from visit encounter interface.
- Does EHR system provide better integration among providers by improved information sharing, viewable and up-to-date medication and allergy lists and order entry at point of care or off-site ?
- Will the EHR system provide standardization of data, order sets, and care plans helping to implement common treatment of patients using evidence-based medicine ?
- Are vendor-provided and supported order sets available?
- Does EHR system provide access to experts for rural health care providers by sharing best practices and allowing for specialized care through telemedicine ?
- Can EHR system manipulate and provide feedback on population management trended data and treatment and outcome studies for more convenient, faster, and simpler disease management?
- Can the system pull from clinical documentation for billing, quality reporting and patient safety reporting?
- How does the vendor use certified EHR technology in ways that can be measured significantly in quality and in quantity?
- Will the system support utilization review with the Milliman and Interqual clinical criteria for Quality Management and discharge planning.
- Does EMR system provide capability to aggregates data on the impact of specific evidence based/cost effectiveness intervention as compared to current population in management practice?
Public Health Research
- Does the vendor support the ability to send de-identified bio-surveillance data to population health agencies to support epidemiologic research?
- Does the vendor have the ability to send patient specific public health disease report(s) to a disease registry, as a part of a Public Health Information Network (PHIN)?
- Does the vendor have the ability to report on Quality Improvement, such as infection control measures (i.e., Hospital Acquired Infection rates) or patient safety measures?
- Does the vendor have the ability to report to the State, CDC Registry and external sources
Cost and Budget
There are also long-term costs associated with EHRs that need to be considered. Certain systems require specific hardware, interfaces, networking, training and support resources that all add on to the price. Practices should take these additional services into account to get a comprehensive picture of what the technology will cost. 
Associated Costs – in addition to EHR licensing and maintenance, studies report expenses can be expected for; over 130 hrs./physician for system training, reduction in patient volume by as much as 50% during initial implementation, and significant expenses for additional IT infrastructure (e.g. approx. $10,000 for primary care practice). 
- It is not uncommon for developers or implementers to impose lock-in features in order to protect their IP rights. How mature is the support community for the particular vendor and is there a marketplace for support and services to their product or is support limited to their own team?
- A multidisciplinary team, usually a subset of the decision team consisting of a physician, IT personnel and a senior management must conduct a site visit to one of the vendor customer sites and vendor headquarters. Informal discussions at this level were quite informative as they bridged the gap between theory and practice for actual software implementation and usage.
- An unstructured “drive-by” demonstration of the system must be arranged for, wherein the vendors conduct product demos, demonstrate product futures and distribute promotional materials for a majority of the employees to get the feel of the new systems. It facilitates building a comfort level and relationship with the vendor and the product offered.
- Can the company demonstrate tangible ways that use of the product can increase hospital revenue? If yes, can you validate this/these claims?
- Is the vendor willing to contract to go "at risk" for any part(s) of the contract?
- What other services does the company offer (especially useful for rural or small hospitals or practices)? e.g. consultants to do special projects, data transmission/claims, billing
- Can the record be accessed at home by clinicians and patients, or do they have to be on-site at the facility?
- Is the vendor capable of adapting to emerging technology such as open source programming, cloud computing, "tablet" PCs/Macs, demand for smartphones (e.g. iPhone), and Web 2.0 technology?
- Is the vendor capable of integrating its system with a personal health record and allowing more patient control?
- Can the company provide return on investment analysis?
- Vendors may not be chosen if the current managers have been convicted of or have had a civil judgment for the following: fraud, antitrust violations, embezzlement, theft, etc.
- Reduced pharmaceutical costs derived from having information available at the time it is needed
- How can this EHR be an asset for the solo practitioner who sees this technology primarily as a financial drain?
- How much extra financial cost will it be to train outpatient clinics that are a subset of the system?
- Does the vendor return loss money if their system can not complete established goal in scheduled time?
- Can the company provide a multi-background consult team from doctors, nurses, IT engineer, to attorney?
- Does the vendor qualify the organization's acquisition policies? Is the vendor CCHIT certified?
- Certification can be verified at the Certified Health IT Product List (CHPL) 
- Is the vendor active in national EMR leadership task forces, such as the Electronic Health Records Vendors Association 
- Will the vendor be supporting the organization's desired implementation strategy?
- Check the track record of vendor for operations and maintenance support?
- Will the vendor sell or monetize of our clinical data for research or any other purposes?
- Is Application Support provided by on-shore resources?
- Clarify the roles, responsibilities and costs for data migration if desired. 
- Does the Vendor demonstrate financial and management stability?
- Does the vendor have experience with implementing the product in a similar type of organizations? How many?
- Conduct a site visit 
- Ensure that the site is similar to you - similar number of beds, acuity, rural/urban setting
- Flexibility that allows for significant changes in product or order lines. Thus, this enables for businesses to modify orders, if necessary, without having to face a penalty from the vendor. Not always do things go as planned and if any modifications are needed, there should not be any consequences that inhibit this from occurring.
- The ability to provide all the products/services required and/or the complete solution. This in turn can provide the whole picture in allowing for the consumer more transparency in terms of being able to distinguish between the products/services that a particular vendor is offering. As a result, this can save time and money in terms of a consumer being able to choose a certain business that can provide everything that is needed instead of wasting time and effort in purchasing products/services from individual vendors that have a small selection available.
- The ability to have a consistent supply of products or services readily available for the business to purchase at all times. There is always a need for products and services to be provided on a regular basis; thus a business does not want to have a vendor that has supply issues. If a vendor is not reliable, then it can affect the business’s ability to in turn supply and provide for its customers, which can have a dramatic effect on the reputation and finances of the business.
- Does vendor volunteering disclose adverse events or near misses data involving vendor’s EMR system to potential buyers? Are there protocol designed to notify federally designated patient safety organizations of IT related adverse events or near misses? 
- Does the EHR have the ability to link payments and incentive rewards to the implementation of milestones and performance goals, as well as conduct data queries for support of quality improvement? 
- Attend Demonstrations with a rating form this will help you write down the important parts of what you DID and DID NOT like and help you make the best EHR decision for your practice. Sales people are awesome and so are the demonstrators but it's important to stay focused on what will work for you.
Hospital Inpatient Quality Reporting Program (IQR)
The Hospital Inpatient Quality Reporting (Hospital IQR) program was originally mandated by Section 501(b) of the Medicare Prescription Drug, Improvement, and Modernization Act (MMA) of 2003. This section of the MMA authorized CMS to pay hospitals that successfully report designated quality measures a higher annual update to their payment rates. Initially, the MMA provided for a 0.4 percentage point reduction in the annual market basket (the measure of inflation in costs of goods and services used by hospitals in treating Medicare patients) update for hospitals that did not successfully report. The Deficit Reduction Act of 2005 increased that reduction to 2.0 percentage points.
In addition to giving hospitals a financial incentive to report the quality of their services, the hospital reporting program provides CMS with data to help consumers make more informed decisions about their health care. Some of the hospital quality of care information gathered through the program is available to consumers on the Hospital Compare Website.  It is important that the EMR supports the capability to capture all required elements for different Core Measure Conditions to help facilitate compliance.
Physician Quality Reporting System (PQRS)
PQRS is a reporting program that uses a combination of incentive payments and negative payment adjustments to promote reporting of quality information by eligible professionals (EPs). The program provides an incentive payment to practices with EPs (identified on claims by their individual National Provider Identifier [NPI] and Tax Identification Number [TIN]). EPs satisfactorily report data on quality measures for covered Physician Fee Schedule (PFS) services furnished to Medicare Part B Fee-for-Service (FFS) beneficiaries (including Railroad Retirement Board and Medicare Secondary Payer).
Beginning in 2015, the program also applies a negative payment adjustment to EPs who do not satisfactorily report data on quality measures for covered professional services. This website serves as the primary and authoritative source for all publicly available information and CMS-supported educational and implementation support materials for PQRS. Stay informed for latest updates by subscribing to the [PQRS Listserv]  EMR should have PQRS data capture and reporting capability in order to meet PQRS and avoid related payment adjustments.
The measures CMS has selected for PQRS are developed and approved by organizations such as the National Quality Form (NQF). Some measures represent undesirable outcomes, while others represent desirable outcomes. New measures are added each year and measures from previous years can be updated or deleted. A list of the 267 measures for 2012 can be found here. The PQRS measures include both process quality measures and outcome measures, such as the patient’s blood pressure and HbA1c level. Examples include:
Participation in PQRS is voluntary, but there are rewards and penalties associated with participation. Currently, providers earn an incentive payment simply for reporting PQRS measures .
EHR Evaluation Resources
- American Academy of Family Physicians Vendor Rating Tool
- American Academy Of Family Physicians HR Demonstration Rating Form EHR Demonstration Rating Form 
- American Academy of Family Physicians vendor's references verification form 
- American College of Physicians EHR Feature Checklist
- Bureau of Primary Health Care. BPHC Electronic Medical Record Resources page. 
- Doctor's Office Quality - Information Technology Guidelines for Evaluating EHR Vendors
- The National Learning Consortium Vendor Evaluation Matrix Tool. 
- California Health Care Foundation, EMR Evaluation Tool and User Guide - A Guide for Small Physician Practices: 
- American Medical Association - 15 questions to ask before signing an EMR/EHR agreement 
- Health Resources and Services Administration How to Select a Certified EHR 
- Select or upgrade to a certified electronic health record vendor 
- Selecting the right EMR vendor. http://www.himss.org/files/HIMSSorg/content/files/selectingemr_flyer2.pdf
- What factors should I consider when selecting a vendor? http://www.healthit.gov/providers-professionals/faqs/what-factors-should-i-consider-when-selecting-vendor
- Weber, M. Selecting an EHR, now what???? [PDF document]. Retrieved from Lecture Notes Online Web site: https://moodle.sbmi.uth.tmc.edu/pluginfile.php/43545/mod_resource/content/1/Selecting_an_EHR_Vendor.pdf
- Selecting or Upgrading to a Certified EHR. http://www.healthit.gov/providers-professionals/ehr-implementation-steps/step-3-select-or-upgrade-certified-ehr
- HITECLA.Org Selecting the Right EHR. http://www.hitecla.org/ehr_selection_tips
- EHR/HIE: Interoperability http://interopwg.org/certification.html/
- Kannry, J, Mukani, S & K Myers. Using an Evidence-based Approach for System Selection at a Large Academic Medical Center: Lessons Learned in Selecting an Ambulatory EMR at Mount Sinai Hospital, Journal of Healthcare Information Management — Vol. 20, No. 2
- McDowell, S. W., Wahl, R., & Michelson, J. (2003). Herding cats: the challenges of EMR vendor selection. Journal of healthcare information management, 17(3), 63-71.
- The Christ Hospital http://www.thechristhospital.com/
- Blake, R.T., Massey, A.P., Bala, H., Cummings,J., Zotos,A. (2010).Driving health IT implementation success: Insights from The Christ Hospital.53(2),131-138 http://www.sciencedirect.com/science/article/pii/S0007681309001530/
- Jain, V. (2010). Evaluating EHR systems. Health Management Technology, 31(8), 22-24
- Integrating Electronic Health Records and Clinical Trials http://www.esi-bethesda.com/ncrrworkshops/clinicalresearch/pdf/MichaelKahnPaper.pdf .
- Using an evidence-based approach for system selection at a large academic medical center: lessons learned in selecting an ambulatory EMR at Mount Sinai Hospital. http://www.ncbi.nlm.nih.gov/pubmed/16669592
- McCarthy, K. 5 important considerations when choosing EHR. 2014. http://www.nuemd.com/news/2014/10/28/5-important-considerations-when-choosing-ehr-systems
- Understanding the True Costs of an EHR implementation http://go.galegroup.com.ezproxyhost.library.tmc.edu/ps/i.do?ty=as&v=2.1&u=txshracd2509&it=search&s=RELEVANCE&p=HRCA&st=T002&dblist=HRCA&qt=TI~Understanding+the+true+costs+of+an+EHR+implementation~~SP~52~~IU~14~~SN~00257206~~VO~89&sw=w&asid=a842cc467e789e3bde4b58ab9e5d0fab
- Certified Health IT Product List (CHPL) list of EMRs. http://oncchpl.force.com/ehrcert
- HIMSS Electronic Health Record Association http://www.himssehra.org/ASP/index.asp
- Better Patient Safety is Goal of Confidential EHR Error Reports http://www.amednews.com/article/20130724/profession/130729986/8/ .
- Health Resources and Services Administration http://www.hrsa.gov/healthit/toolbox/healthitimplementation/implementationtopics/selectcertifiedehr/selectacertifiedehr_7.html
- HEY, WHO DID THIS? Note: there needs to be something here
- Hospital Inpatient Quality Reporting Program. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HospitalQualityInits/HospitalRHQDAPU.html/
- Physician Quality Reporting System http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/PQRS/index.html?redirect=/PQRS/
- Bryan Dowd, Chia-hsuan Li, Tami Swenson, Robert Coulam, and Jesse Levy. Medicare’s Physician Quality Reporting System (PQRS): Quality Measurement and Beneficiary Attribution. Medicare Medicaid Res Rev. 2014; 4(2): mmrr2014.004.02.a04
- The National Learning Consortium Vendor Evaluation Matrix Tool. http://www.healthit.gov/providers-professionals/implementation-resources/vendor-evaluation-matrix-tool