Vendor Selection Criteria: Vendor assessment
Most vendors typically fall into one of the three categories:
- Vendors that develop their own software organically on a single source code, one database, single instance.
- Vendors that may operate under one name, but offer several acquired products, including some custom programs.
- Vendors that have been acquired/sold/merged as a means to stay more competitive.
The first category of vendor is less likely to run into major market conflicts to keep its software modern and compliant with regulations. Vendors that operate on a single source code have far less difficultly staying current and/or responding to rapid changes in the market and mandated IT standards.
- 1 Vendor Proposal (Request)
- 2 Regional Selection Center EHR Selection Criteria
- 3 Cost and Budget
- 4 Vendor Financing
- 5 Vendor viability
- 6 Extensive Testing of EMR Software Prior to Implementation
- 7 Face the Interfaces
- 8 References
Vendor Proposal (Request)
- Requesting a proposal is just as important as selecting the type of medication that will be used in your practice. When you ask for a proposal it is important that as a practice you are clear and concise as to what you and your practice will need. This process, though very monotonous is absolutely crucial to the flow and efficiency of your practice. It also gives you an inside knowledge to the various EHRs that are available and will give you the ability for good comparison to what is on the market 
- A clear list of specific needs (EHR system requirement) should be written
- Prepare a clear criterion of the system that will be used to make the selection
- Make request to selected vendors
Regional Selection Center EHR Selection Criteria
The Regional Selection Center suggests that medical practices compare EHR vendors using the following criteria before selecting a specific product. 
- Will the vendor’s product accomplish key practice goals?
- Clarify EHR system start-up prices before selecting a vendor including costs associated with hardware, software, maintenance and upgrades, phase payment options, lab and pharmacy interfaces, connection to health information exchange, and customized quality reports.
- What implementation support does the vendor offer?
- What are the costs, roles, and responsibilities associate with the data migration strategy?
- What are your sever options?
- What is the products ability to integrate with other products?
- What are the privacy and security capabilities of the product and what is the back-up plan?
- How does the product link payments with EHR incentive rewards, which are necessary to meet the practices EHR implementation goals and milestones?
- What is the vendor’s stability and market presence?
- What is the cost to link the product to HIE?
- What are the costs associated with legal counsel for contract review versus open sources through medical associations?
Cost and Budget
There are also long-term costs associated with EHRs that need to be considered. Certain systems require specific hardware, interfaces, networking, training and support resources that all add on to the price. Practices should take these additional services into account to get a comprehensive picture of what the technology will cost. 
Associated Costs – in addition to EHR licensing and maintenance, studies report expenses can be expected for; over 130 hrs./physician for system training, reduction in patient volume by as much as 50% during initial implementation, and significant expenses for additional IT infrastructure (e.g. approx. $10,000 for primary care practice). 
A 2010 Health management Technology article states, a number of healthcare technology vendors are now offering free financing for electronic health record (EHR) purchases, in an effort to increase adoption of electronic records. 
Cost is physicians' top concern when considering EHRs, according to a recent Ingenix survey of 1 ,000 physicians and physician-practice administrators. More than 80 percent ranked cost as one of the greatest risks to deploying technology in their practice. 
- What kind of support is included in the initial purchase price of the EHR system?
- How long will support be provided (e.g, on site, by telephone, or email)?
- Will the vendor work with contractors to install the system or just be providing with a set of network, telecommunications and electrical specifications?
- Will the vendor be working with the organization to customize software features such as the templates that will be used? 
- Will the travel expenses of the vendor be included in the implementation estimate? Are the travel expenses fixed or a percentage of the implementation fees?
- Vendor viability must be checked. Will the vendor be around in nine years (the average life span of a significant IT investment)? If not, can the organisation live without them? Evaluations by neutral third-party analysts like Gartner, Chilmark, KLAS, and The Advisory Board must be considered to see what these analysts are saying about the vendor’s prospects in the market. Questions about vendors viability must be considered such as: Is the vendor in solid financial shape? What’s their monthly burn rate vs. income? How many days cash-on-hand do they maintain? What does their sales pipeline look like? Does the vendor’s executive leadership team have a track record for jumping from one company to another or do they have a track record of longevity and success? How much is the vendor spending on sales staff in comparison to engineering and product development staff? The best products are supported by a very lean sales staff. That’s because great products sell themselves.
- EHR Goals must be set. Goals should be specific, measurable, attainable, relevant, and time bound.
- Key Decisions should be made by making a list of potential deal-breakers such as deciding whether EHR data should reside in-office, a vendor server, or in web-based storage (“cloud storage”). To help form a list of potential deal-breakers, vendor websites should be researched. Making key decisions up-front will enable a practice to effectively narrow the field..
- Asking about product experiences and user experiences are crucial before selecting a vendor. 
- An EHR evaluation tool should be utilized before selecting a vendor. EHR evaluation tools can be obtained from medical societies. 
- Online form from HRSA available for establishing EHR specifications to help in vendor section. The form's name is 2011 EHR Selection Guidelines for Health Centers and can be downloaded from www.HRSA.gov 
- Requesting a demo from the vendor keeps a potential buyer informed on prospective features an EHR possesses. 
- Asking if the vendor is committed to training the institution's in house staff. 
- Is there capability to tailor current features of EMRs to enable a smoother changeover from paper to electronic records and to facilitate meeting the criteria for meaningful use. 
- Will the EHR meet present and future requirements? How user friendly is the EHR? 
- Which category do you fall under should be one of the first questions you ask. Are you a small practice 1-15providers, a medium practice 15-100providers, or a large practice greater than 100providers ? Once size of practice has been determined including the potential to reach the size should help you narrow down as to which vendors might best suit your needs.
- Percentage of research and development reinvested into the company.
- Life cycle state or maturity of EHR system products should be asked(i.e., the occurrence of software obsolescence)..
- Frequency of software product updates..
- Customer support availability..
- Certification status of the EHR..
- Setting goals before selecting an EHR. The goals should be specific, measureable, attainable, relevant, and time bound. Having these goals will guarantee that your organization benefits from the EHR. 
- Make a rundown of potential issues and choose where you wish to have your EHR data to reside. Whether you need your EHR information to reside in a live in-office, a vendor server, or web-based storage. Making this list before selecting an EHR will help your practice to narrow the field. 
- Plan site visits. Ask vendors for a list of practices that have successfully implemented the EHR. At site visits it is important to have a list of questions for the practice during, before, and after implementation of the EHR. 
- No vendor can offer all possible services and products. Therefore, it is important to assess if an ecosystem of partners exist for the selected vendor. Is there active developer program? Does the vendor use an open strategy or single a source strategy? 
- A good vendor should have a healthy management team that keeps it growing and improving. Does the vendor have stable, visionary and efficient management team?
- We certainly do not wish to select a vendor that may bankrupt in the near future. Does the vendor have resources to compete in the market and likely to win in the future?
Extensive Testing of EMR Software Prior to Implementation
Vendors should be able to demonstrate the conduction of rigorous testing of their EMR, such as: 
- Unit Testing – testing of software codes and algorithm to detect errors which might lead to inaccuracies in patients’ medical records i.e., transposition of patient information, omission of diagnostic test results 
- Regression Testing – Testing “end to end” functionality after the introduction of an additional ‘unit’ of functionality or a “bug fix” to confirm that the newly introduced change does not “break” related processes. 
- Integration Testing – testing for integration and interface between different vendor modules i.e., faulty integration of CPOE and Pharmacy modules could result in physicians orders not being forwarded to the pharmacy department 
- Systems Testing – testing of the EMR system’s ability to run on a multiple user environment, have optimal runtime availability, have an interface that is highly usable and intuitive, with strict adherence to information security regulations such as HIPAA 
- User Acceptance Testing – testing by end users (physicians, nurses, administrative personnel) that validates whether all specified requirements and functionalities for the EMR have been met 
- Usability Testing - Realization of EMR benefits depends largely upon usability. Usability testing with representative end users validates “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” (p. 331). 
Face the Interfaces
One of an EHR’s most valuable assets is its potential to share information with other IT systems such as lab and diagnostic services. Interfaces allow the EHR system to communicate with other applications. These applications can reside outside the practice, such as lab applications, or can be another system within the practice, such as a billing system.
Interface development is completed during the implementation phase; however, interface functions and costs must be a consideration during system selection. One cannot assume that a given EHR product will automatically communicate with other electronic functions such as appointment scheduling or billing. Vendors should be asked to address the issue of integrating lab and radiology results into the electronic record.
Many vendors offer ancillary services such as automatic statement processing, billing services, consulting services, additional coding enhancements, and claim scrubbing. One may want to take advantage of these services later. These items are considered third-party or add-on applications.
Many practices are considering the implementation of portals that offer patients a secure Internet connection to the practice. The portal allows patients to schedule appointments, complete paperwork, and access test results (or even their medical record), depending on the services the practice offers. These portals have the potential for increased practice production, increased revenues, and increased patient satisfaction. 
Interfaces can be a tricky piece of system selection and the implementation that follows. Red flags during the selection process include:
- One-way or bidirectional interface not specified. The system may only accept demographic or appointment data from the practice management system but not feed data back to it.
- Data format not specified. Data can be entered into EHRs in one of two formats--as free text (unstructured data) or as defined elements (structured data) such as forms and pick lists. Each practice must weigh the pros and cons for itself.1 However, ask vendors if their products support standard vocabularies such as SNOMED, Medcin, and LOINC.
- Additional license costs not specified. Additional fees for third-party applications could apply, such as plug-ins for a drug database or CPT and ICD-9 coding, code scrubbing, database user seats, and anti-virus software.
- Interface promised but not available. A vendor may say it can interface with your billing application or a lab system, but after the contract is signed it may advise you that the interface is not available. That can place a serious strain on the practice and require you to enter data twice.
One of the more efficient methods of determining a particular EMR's ability to handle interfaces is by reviewing the history of interfaces which have already been built and established.  The following questions should be answered in order to establish if the EMR being considered meets your organization's needs:
- Which vendors (and which of their applications) have they interfaced with?
- What type of information (patient demographics, order entry, laboratory results, billing, radiology findings, etc.) was interfaced?
- How many interfaces were built, and what is the maximum the system can support?
- What is the capacity of the interface; how many messages where sent and/or received per day; and what is the theoretical maximum amount allowed?
- What, if any, additional costs were involved in creating, operating, and maintaining the interface?
- Where there additional fees depending on the amount of data being sent/received?
- Were there any costs to expand and/or modify the capabilities of the interface in the event the organization grows and requires additional functionality?
User Satisfaction : User-Centric Selection
- User satisfaction of the clinical staff plays a key role in vendor selection and can predict the success or failure of the EMR system implementation 
- Since users may have varied experiences according to Kannry, Mukani, and Myers (2006, p. 89) it is important to include a multidisciplinary team from clinical, administrative and IT personnel as part of the vetting and selection process.
- The creation and use of Scripted Scenarios representative of user workflow was helpful even when users had little technical knowledge of EMR systems. 
Check Vendor References
- Each vendor being considered should provide minimal of three (3) references, to include a) physician users, b) IT person, and c) senior manager of the facility or practice.
- Vendor provided references may be considered 'happy customers' with biased viewpoints.
- Check several references on your own, outside of the provided references from the proposed vendor.
- The chosen reference should be of a comparable size and structure
- Have a prepared list of questions to ask.
- Compare vendor satisfaction with current customers.
- Ask supplied list of references how they acquired the EHR system, eg did they purchase the system or was it provided by the vendor?* Ask each reference to rate the vendor on a scale from 1 (very dissatisfied/strongly disagree) to 5 (very satisfied/strongly agree) for each criterion
- Each vendor reference should provide reference on background information such as version of system currently being used, duration using current system, length of implementation, number of interfaces practice currently installed and estimated time to install, training provided on-site or remote.
- Each vendor reference should provide reference on current system overview such as system reliability, satisfaction with vendor, and overall satisfaction with system.
- Research references the vendor did not mention, visit facility to get an unbiased viewpoint
- Must use listservs, internet searches and networking to identify other users to interview to get more objective viewpoint.
- All interviewers should utilize the same questionnaire approved by the project manager or steering team.
- Notes must be recorded by each interviewer for each customer interviewed.
- These notes should be made part of the overall evaluation process, ideally kept by the project manager. 
- Is the vendor currently involved in or have a history of any litigation with customers?
- Does the vendor have a track record?
- Has the vendor been in business for long?
- Perform site visits to similar organizations that are currently implementing the vendor’s product and ask plenty of questions to all end users (physicians, administrative staff, nurses, etc.)  
Once you have narrowed down your search to two vendors, you’ll want to visit a practice where the system has already been implemented. Ask the vendors for a client list and contact the references yourself. If there is no local client base, keep in mind that you won’t have the support of a local colleague, and that can be challenging. Only visit practices that are similar in size and specialty to yours; if they use different templates you may not get the full idea of how the system can be applied to your specialty. During site visits, ask how engaged the vendor was in the implementation. For a truly successful implementation, you’ll need a close relationship with your vendor. Be sure that the vendor’s project manager provides face-to-face interaction with your staff. The project manager should also attend staff meetings and produce regular project updates as the implementation progresses. Conducting site visits at other institutions that have already implemented the vendor’s system may provide additional and practical insight. Below is a list of criteria that should be observed on site visits. The following are criteria established by MetaStar and can be found here. 
- Bring a diversity of positions from your committee.
- At minimum bring a physician, IT specialist, and senior management person.
- Observe during actual patient encounters.
- Observe other departments such as billing and labs using the vendor.
- Choose a site to visit based on your own research rather than one provided by the vendor.
EHR vendors may enter contracts with confidentiality or non-disclosure terms that can prevent transparency. Examples include:
- Disclosures required by law or regulation, sometimes with an obligation to give the other party advance notice and the opportunity to oppose the disclosure or seek confidential treatment.
- Disclosure of information that has been independently developed by the disclosing party
- Disclosure of information that is available to the general public or has been provided separately to the disclosing party without violation of an agreement.
- Are there any hidden fees associated with training, support, consultant costs?
The definition of confidential information may be broad and could restrict your ability to share access to the EHR technology developer’s software in order to compare different EHR technology developer systems, provide access to researchers, or even address possible patient safety concerns. You should review the confidentiality and non-disclosure language carefully to make certain it does not inhibit your ability to conduct activities you value.  back to home, Vendor Selection criteria
- Types of vendors http://www.greenwayhealth.com/wp-content/uploads/2013/03/Replacing_an_EHR-Coker_Greenway_white_paper.pdf
- Kenneth G. Adler, MD, MMM Fam Pract Manag. 2005 Feb;12(2):55-62How to Select an Electronic Health Record System http://www.aafp.org/fpm/2005/0200/p55.html
- Selecting or Upgrading to a Certified EHR. http://www.healthit.gov/providers-professionals/ehr-implementation-steps/step-3-select-or-upgrade-certified-ehr
- McCarthy, K. 5 important considerations when choosing EHR. 2014. http://www.nuemd.com/news/2014/10/28/5-important-considerations-when-choosing-ehr-systems
- Understanding the True Costs of an EHR implementation http://go.galegroup.com.ezproxyhost.library.tmc.edu/ps/i.do?ty=as&v=2.1&u=txshracd2509&it=search&s=RELEVANCE&p=HRCA&st=T002&dblist=HRCA&qt=TI~Understanding+the+true+costs+of+an+EHR+implementation~~SP~52~~IU~14~~SN~00257206~~VO~89&sw=w&asid=a842cc467e789e3bde4b58ab9e5d0fab
- EHR financing offered. (2010). Health Management Technology, 31(2), 10. Retrieved from http://ezproxyhost.library.tmc.edu/login?url=http://search.proquest.com/docview/622031107?accountid=7034
- EHR financing offered. (2010). Health Management Technology, 31(2), 10. Retrieved from http://ezproxyhost.library.tmc.edu/login?url=http://search.proquest.com/docview/622031107?accountid=7034
- How Much Will That EMR System Really Cost? http://www.aafp.org/fpm/2002/0400/p57.html.
- vendor viability https://www.healthcatalyst.com/How-to-Evaluate-a-Clinical-Analytics-Vendor/2/
- Vendor Assessment http://www.healthit.gov/providers-professionals/faqs/how-do-i-select-vendor
- - Vendor Assessment http://www.healthit.gov/providers-professionals/faqs/how-do-i-select-vendor
- - http://search.hhs.gov/search?q=EHR+evaluation+form&site=hrsa&client=hrsa&proxystylesheet=hrsa&btnG=Search&lr=lang_en&output=xml_no_dtd&sort=date%3AD%3AL%3Ad1&ie=UTF-8&ud=1&oe=UTF-8
- Weber, M. Selecting an EHR, now what???? [PDF document]. Retrieved from Lecture Notes Online Web site: https://moodle.sbmi.uth.tmc.edu/pluginfile.php/43545/mod_resource/content/1/Selecting_an_EHR_Vendor.pdf
- https://medicalmastermind.com/blog/emr-checklist-ten-product-questions-to-ask-the-vendor/ ‘’Medical Mastermind’’, EMR Vendor Checklist: Ten Important Product Questions to Ask the Vendor, July 31, 2012
- ‘’Practice Fusion Blog”, 4 questions to ask when selecting an EHR, July 7, 2014
- Vendor Selection http://library.ahima.org/xpedio/groups/public/documents/ahima/bok1_047961.hcsp?dDocName=bok1_047961
- Enterprise EHR Vendor Evaluation https://www.advisory.com/Research/Health-Care-IT-Advisor/ATC/research-notes/2013/enterprise-ehr-vendor-evaluation
- Valacich, J. S., George, J. F., & Hoffer, J. A. (2012). Essentials of systems analysis and design (5th ed.)
- Proposing Regulatory-Driven Automated Test Suites http://ieeexplore.ieee.org.ezproxyhost.library.tmc.edu/stamp/stamp.jsp?tp=&arnumber=6612874
- Harrington, L., Porch, L., Acosta, K., & Wilkens, K. (2011). Realizing electronic medical record benefits: An easy-to-do usability study. Journal of Nursing Administration, 41(7/8), 331-335.
- How to Select an Electronic Health Record System. http://www.aafp.org/fpm/2005/0200/p55.html
- Murphy E., et. al. An Electronic Medical Records System for Clinical Research and the EMR–EDC Interface http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2361387/
- Selecting An EHR, Now What? ftp://ftp.hrsa.gov/ruralhealth/jan082008tacall.pdf
- MetaStar https://www.metastar.com/web/professional/docs/DOQ-IT/Vendor/Tools/SiteVisitingTips.doc
- EHR Contracts: Key Contract Terms for Users to Understand. http://www.healthit.gov/sites/default/files/ehr_contracting_terms_final_508_compliant.pdf