Difference between revisions of "Vendor Selection Criteria"

Revision as of 16:54, 28 January 2010

Example Vendor Evaluation Criteria

Demonstrate Clinical Functionality (25%)

• How well is the vendor able to demonstrate the clinical functionality required by [your organization]?
• Is the software flexible enough to meet our current needs and allow for future changes?
• Can a demonstration of the software successfully handle a scenario you have prepared?
• Does the software have functions of supporting clinical decisions?
• Does the software have functions of assisting nurses in documentations?
• Is the vendor able to create interfaces for physicians' office EMRs to allow viewing and sharing of clinical, financial (insurance) and other data?
• How easily and quickly can the embedded clinical decision support be modified when new published guidelines or evidence-based medicine studies are published? Will they be responsible for these updates (and at what cost) or will that be our responsibility?
• Is the vendor software compliant with the Certification Commission of Healthcare Information Technology (CCHIT) criteria?
• Can the company categorize users according to their characteristic and provide user-centered usability design? Old physicians usually have vision problem and do not like to use keyboard.
• Can the company provide different levels of decision support based on user working experience? For resident, attending physician, the decision support should be different.
• What capabilities does the application provide for patients with psychiatric, psychological, or severe behavioral disorders?
• If your organization's mission includes clinical research, does the software include decision support and data query functions to support research?
• Can the medical records be easily transferred to another clinic with patient approval?
• With what other HIT software systems have you established successful interfaces? (organization may use a variety of systems for BCMA, clinical documentation, laboratory data, business functions that need to have data interchange)
• What capability does the application demonstrate to produce information for patient education?
• Does the system provide capability for patient portals (web access)?
• What wireless functions has the system supported?
• Does the system satisfy security requirements (access control and logging)?
• Does the system provide configurable workflow options?
• Does the system provide integration options with other service providers’ systems (hospitals, insurance companies, labs, …)
• Does the system provide any linkage between clinical notes and structured medical vocabularies?
• Joan Breuer, Ph.D. 01/27/2010 17:19 The Vendor product must have vocabulary standards such as SNOMED, NDC, LOINC, WHO Drug Dictionary, MeSH, CPT, DRG, and UMLS.
• Can the EHR vendor provide a standardized system for the market? And at the same time, can they customize the EHR for different clients and requirements?
• Is the EHR system Health Level Seven (HL7) ready?
• Does the EHR vendor have a procedure to integrate other record/data (e.g. personal health record, public health disease surveillance data) into the system?
• Functionality Matching: Vendor with best functional match. All product selection exercises should start with determining your organization's specific needs.
• Joan Breuer, Ph.D. 01/27/2010 17:20 There needs to be a plan so In-house developers can work with the Vendor such that all algorithms are carried out seamlessly.
• Is the system’s data entry interface intuitive for new users?
• Can the system pull from clinical documentation for billing and patient safety reporting?
• Does the system have meet specific specialty practice needs of users?
• How granular is user access to information? Is the user access hierarchy flexible to accomodate HIPPA standards?
• Does the EMR have different modules for different specialties in our healthcare facility?
• Are these modules “home-grown” at your company or have they been acquired from different companies as your EMR has grown?
• Are the module structures set or can they be used as a template which can be modified to suit or healthcare facilities needs?

Acquisition and Implementation Cost (25%)

• What is the total financial investment required to acquire and fully implement the proposed solution? Break out costs for the following and detail any time or resource constraints for each item. If additional resources or time are required, what are the additional costs per unit?
  • Application
  • Per-user licenses (and tier pricing, if offered)
  • Database (if treated as a separate item)
  • Application documentation
  • Annual maintenance agreement
  • Training
    • End-user
    • System administrators
    • In-house developers
  • Professional services:
    • Project management
    • Software development or customizations
    • Technical support
• What is the estimated time that it will take for the investment of the vendor to pay for itself based on projected savings?
• Does the system allows to track and detail log audits and transactions made by users?
• Does the system have the functionality to generate customizable reports given a determinate frequency?
• What is the cost for upgrading when new releases are available?

Hardware Platform and Technical Requirements (20%)

• Compared retail broker workstation alternatives, including proprietary and third party packages, based on user requirements and technology/operational considerations. Project included EMR development and response evaluation

• How well is the vendor able to meet our technical requirements ?
• How flexible is the vendor to change requirements and what are the cost?
• How hard is to implement new modules within the system?
• How well is the vendor able to create and maintain interfaces to our existing systems?
• How well is the vendor able to migrate our data from actual system to the new, if it is neccesary?
• Is the vendors's software created with the most adequate developing technology? Platform? Language? Databases?
• Will we do periodic updates, or will the vendor do this as part of the contract? How often are they released?
• Do we plan on-site or remote hosting for the system?
• If we plan to utilize remote hosting, how safe, sound, and accessible are these data?
• Is the hardware user friendly in all departments? (i.e. nursing, pharmacy, pulmonary,..etc.)
• Will the vendor provide a mock system for testing?
• What is the responding duration of each entry and information extraction? Is there any delay?
• How the data structure for storing data? Is it expandable? Is it easy to be extracted for future analysis?
• How often does the system need to be updated or serviced?
• As the technology is changing and the hospital decides to upgrade their hardware, will your system be able to handles these changes or will we have to buy a new program?
• What language is the vendor's systems software coded in?
• Scaling:
  • To how many patients has the system been scaled?
  • To how many users has the system been scaled?
  • Has the system been scaled to the size of the evaluating institution?
  • What are the hardware/software licensing implications of growth?
• Is it necessary to buy (possibly expensive) hardware from the vendor, or can we just install their software on our standard desktop computers (at least for the clients)?
• For already existing software functionality, does the vendor employ reliable and trustworthy software like an open-source MySQL or Apache server? Or do they develop their own system, or do we have to license a proprietary one (extra cost)?
• How the vendor estimates the total amount of users and licenses needed? Will they be concurrent user licenses or asynchronic?
• Will technical support remain active even if the Hospital is running a non upgrade system? For how long?
• Does the system use DICOM standards for the transmission of image data?

Implementability (15%)

• How much time, effort, and resources will be required to successfully implement?
• What is the vendors track record for successfully implementing its system in similar settings?
• Do the vendors provide detailed plan for implementation, training and quality control?
• Will the vendor supply on site support when we "go-live" and how long will they be available?
• Do they have a backup or alternative plan if the system or partial of the system is not working? The situation could be out of energy or computer virus attack.
• How long is the training that is required for each subgroup to fully implement the system?
• What are the training requirements for the vendor? For the clinic/hospital?
• What is the vendor's track record for successfully training a new system for your clinic/hospital size?
• What training manuals, user guides, on-line training assets, and any other training materials will be made available to the clinic/hospital?
• Will the implementation require a consulting "team" and how much will this cost?
• Is the vendor sufficiently well known that most health care consulting firms have familiarity with the system?
• How will legacy patient record data be integrated into the new system?
• Is the vendor's system compliant with the Health Insurance Portability and Accountability Act (HIPAA)?
• Is the vendor's system compliant with the Patient Safety and Quality Improvement Act (PSQIA) Patient Safety Rule?
• How is the evaluation procedure for the approval of the ongoing stages of implementation? How objective will be the defined how to proceed with the approval and payment?
• Is the vendor's system compliant with ISO standards for EHRs?

Vendor Partnership and On-going viability (10%)

• Create a list of vendor selection criteria, and evaluate the vendors and software: review responses to the RFP; schedule demonstrations; check references; and, use proven tools/templates for assessing and consistently comparing vendors.

• Research available EMR systems and vendors: determine if your jurisdiction has a "pre-approved" vendor list; develop checklists & questionnaires to assist with product comparison; and, prepare a Request for Proposals (RFP).

• What type of long-term relationship do we think we can expect from the vendor?
• How stable vs risky is the company?
  • Request that the software source code be put in escrow with specific provisions established under which it could be accessed.
  • How long has the company been in business?
  • How large is the company?
    • How much money did they bring in last quarter? year? 3-years?
    • How much is the Vendor company revenue in $M?
    • How much is the vendor company net income in $M?
    • How much are the total assets and liabilities of the company?
    • What is the growth strategy of the company?
    • Are they a private or a public company?
    • How many employees?
• How many contracts have they should to Health Care Systems like yours?
  • Can you contact and/or visit a few representative samples?
• How many new contracts have they signed in the past year?
• How many uninstall's have been done the last year?
• Does word of mouth support that the company believes in service after the sale?
• Joan Breuer, Ph.D. 01/27/2010 17:22 The HCF needs formal documentation from other HCFs who have used this Vendor’s product signed by the other HCFs’ CEOs, prepared by an attorney. Note: “Word of mouth” does not stand up in court, especially when we are talking about millions of dollars.
• What is the vendor's cost per year after implementation?
• How the vendor provides technical support? On site or remote? Package or individual project based?
• What provisions exist in the contract for termination and/or penalties if the application does not perform as specified or if the vendor fails to provide any services that are agreed to?
  • Ensure that all application specifications that can be measured or quantified are incorporated into the contract.
  • Ensure that all services verbally offered by the vendor are incorporated into the contract.
  • Does the vendor have service level agreements (SLAs) with appropriate penalties for technical support of the application?
  • Does the system’s implementation plan consider proper risks evaluation and mitigation strategies?
• An important consideration is whether the vendor or any of its industry partners have done an analysis of processes in healthcare
• In a HIPAA complaint product, one should consider the feasibility of implementing future mandates. The cost should be considered in terms of both money and time.

• Obtain references of current customers that are similar in size, have similar patient population, and have similar required functionality needs to your practice and evaluate the ease of implementation, current satisfaction, and costs.

• What kind of service level agreements are offered by the vendor and what is their track record for maintaining those SLAs?

Future Vision (5%)

• What is the vendor's five-year strategic plan?
• What percentage of revenue is being re-invested into R&D?
• How does the company capture/communicate client concerns for re-engineering? i.e. Is this a learning company?
• What "game changers" are on the drawing board? Are these related to one of our organization's core goals?
• Compared retail broker workstation alternatives, including proprietary and third party packages, based on user requirements and technology/operational considerations. Project included EMR development and response evaluation
• Lifespan and Staying Power: One should consider the track record of the vendor and viability as a company to sustain the ups and downs of the industry. One sub-criteria in this category often used is how long the vendor has been around.
• Does the vendor plan to offer interoperability options in the event of a nationally implemented CIS?

Extra Credit (optional)

• A multidisciplinary team, usually a subset of the decision team consisting of a physician, IT personnel and a senior management must conduct a site visit to one of the vendor customer sites and vendor headquarters. Informal discussions at this level were quite informative as they bridged the gap between theory and practice for actual software implementation and usage.
• An unstructured “drive-by” demonstration of the system must be arranged for, wherein the vendors conduct product demos, demonstrate product futures and distribute promotional materials for a majority of the employees to get the feel of the new systems. It facilitates building a comfort level and relationship with the vendor and the product offered.
• Can the company demonstrate tangible ways that use of the product can increase hospital revenue? If yes, can you validate this/these claims?
• Is the vendor willing to contract to go "at risk" for any part(s) of the contract?
• What other services does the company offer (especially useful for rural or small hospitals or practices)? e.g. consultants to do special projects, data transmission/claims, billing
• Can the record be accessed at home by clinicians and patients, or do they have to be on-site at the facility?
• Is the vendor capable of adapting to emerging technology such as open source programming, cloud computing, "tablet" PCs/Macs, demand for smartphones (e.g. iPhone), and Web 2.0 technology?
• Is the vendor capable of integrating it's system with a personal health record and allowing more patient control?
• Can the company provide return on investment analysis?
• Does the system have e-prescribing functionality?
• Vendors may not be chosen if the current managers have been convicted of or have had a civil judgment for the following: fraud, antitrust violations, embezzlement, theft, etc...