Vendor Selection Criteria

From Clinfowiki
Revision as of 04:49, 20 September 2010 by Jjortiz1863 (Talk | contribs)

Jump to: navigation, search

Example Vendor Evaluation Criteria

Demonstrate Clinical Functionality (25%)

  • How do EMR software implement HIPAA Privacy and Security Compliance and other regulatory requirements and Local laws.
  • How does EMR software have customer support. Do they have a local workforce, a testing plan, prioritization of issues
  • Do they have multiple note creation options like template, macros, dictation, voice recognition, hand writing recognition.
  • Does EMR software enable a user to maintain up-to-date problem list? For e.g.: Does it enable a user to electronically record, modify, and retrieve a patient's problem list for longitudinal care (i.e. over multiple visits with the same provider and using the problem list vocabulary standards.) in accordance with certification criteria defined by the ONC’s Interim Final Rule?
  • Does EMR software enable a user to electronically record, modify, and retrieve a patient's active medication list as well as medication history for longitudinal care (i.e. over multiple visits with the same provider and using the medication list vocabulary standards.) in accordance certification criteria defined by the ONC’s Interim Final Rule?
  • If EMR software provides CPOE Functionality then does it enable a user to electronically record, store, retrieve, and manage, at a minimum, the order types like 1) Medications 2) Laboratory, 3) Radiology and Imaging and 4) Provider referrals according to certification criteria defined by the ONC’s Interim Final Rule.
  • Will the system support utilization review with the Milliman and Interqual clinical criteria for Quality Management and discharge planning.
  • Will it support the Clinical pathways for the Physicians and the Residents for standardization of care processes?
  • Will it have the direct export options of the collected data to the State and Federal Health Departments and for other research sites?
  • How well is the vendor able to demonstrate the clinical functionality required by [your organization]?
  • Is the software flexible enough to meet our current needs and allow for future changes?
  • Specifically how does the system support many 24/7 functions, like medication administration changes in an environment that does not have 24/7 pharmacy or CPOE users on site?
  • Can a demonstration of the software successfully handle a scenario you have prepared?
  • Does the software have functions of supporting clinical decisions?
  • Does the software have functions of assisting nurses in documentations?
  • Is the vendor able to create interfaces for physicians' office EMRs to allow viewing and sharing of clinical, financial (insurance) and other data?
  • Does it have the capability to provide advanced reporting and analytics?
  • What is the ease of use for front-line workers? Does the user interface modern and able to adapt to different role-based requirements?
  • How easily and quickly can the embedded clinical decision support be modified when new published guidelines or evidence-based medicine studies are published? Will they be responsible for these updates (and at what cost) or will that be our responsibility?
  • Is the vendor software compliant with the Certification Commission of Healthcare Information Technology (CCHIT) criteria?
  • Can the company categorize users according to their characteristic and provide user-centered usability design? Old physicians usually have vision problem and do not like to use keyboard.
  • Can the company provide different levels of decision support based on user working experience? For resident, attending physician, the decision support should be different.
  • What capabilities does the application provide for patients with psychiatric, psychological, or severe behavioral disorders?
  • If your organization's mission includes clinical research, does the software include decision support and data query functions to support research?
  • Can the medical records be easily transferred to another clinic with patient approval?
  • With what other HIT software systems have you established successful interfaces? (organization may use a variety of systems for BCMA, clinical documentation, laboratory data, business functions that need to have data interchange)
  • What capability does the application demonstrate to produce information for patient education?
  • Does the system provide capability for patient portals (web access)?
  • What wireless functions has the system supported?
  • Does the system satisfy security requirements (access control and logging)?
  • Does the system provide configurable workflow options?
  • Does the system provide integration options with other service providers’ systems (hospitals, insurance companies, labs, …)
  • Does the system provide any linkage between clinical notes and structured medical vocabularies?
  • Joan Breuer, Ph.D. 01/27/2010 17:19 The Vendor product must have vocabulary standards such as SNOMED, NDC, LOINC, WHO Drug Dictionary, MeSH, CPT, DRG, and UMLS.
  • Can the EHR vendor provide a standardized system for the market? And at the same time, can they customize the EHR for different clients and requirements?
  • How “personalizable” is the system for individual users to eliminate screen clutter and focus their electronic work process.
  • Is the EHR system Health Level Seven (HL7) ready?
  • Does the EHR vendor have a procedure to integrate other record/data (e.g. personal health record, public health disease surveillance data) into the system?
  • Functionality Matching: Vendor with best functional match. All product selection exercises should start with determining your organization's specific needs.
  • Joan Breuer, Ph.D. 01/27/2010 17:20 There needs to be a plan so In-house developers can work with the Vendor such that all algorithms are carried out seamlessly.
  • Is the system’s data entry interface intuitive for new users?
  • Can the system pull from clinical documentation for billing and patient safety reporting?
  • Does the system have meet specific specialty practice needs of users?
  • How granular is user access to information? Is the user access hierarchy flexible to accomodate HIPPA standards?
  • Has this software shown to decreased adverse drug reactions when implemented in other facilities?
  • Does the EMR have different modules for different specialties in our healthcare facility?
  • Are these modules “home-grown” at your company or have they been acquired from different companies as your EMR has grown?
  • Are the module structures set or can they be used as a template which can be modified to suit or healthcare facilities needs?
  • Ask all staff who evaluate the system for their assessment of the strengths and weakness of the system as they perceive that the system would apply to the practice.
  • Does the software meet the JCAHO standards?
  • What makes this vendor software better than others?
  • Does the ASP (remote hosting) model provide local caching of active sessions in the event the network connection is lost briefly? If so, what functionality is guaranteed during the network outage? MikeField 20:43, 29 January 2010 (CST)
  • Does the EHR system support disease/domain specific profiles that accommodate the level of data needed to support specialty groups such as mental health, oncology and iridology?
  • Does the vendor offer a Document Imaging component as part of their solution to allow incorporation of paper records that contain previous patient histories?
  • How does the software handle continuity of care as patients transfer from inpatient to outpatient, from one department to another or even between providers? Does the system provide for the writing of discharge notes that list diagnoses, medications and other instructions?
  • Is the system easily configurable to allow customized structured data for efficient coding and revenue capture? Clinical data should not be restricted to some arbitrary minimum.
  • Does the system handle clinical reminders (i.e: immunization, drug monitoring, dosage) ?
  • Does your system handle live pharmacy stock when e-prescribing?
  • The user interace includes interoperability with PACS systems?
  • Are there tools for manually triggering simple alerts/messages between clinicians for situations that may not yet trigger CDS alerts?
  • Does the vendor’s application support external access by physicians, and if so is it part of the system or a user addition?
  • Does the vendor offer any data conversion services? At what cost? How long will it take to transfer the data? American Medical Association
  • Can the system indicate normal and abnormal lab results when a patient's lab data are reviewed?
  • Can the system display patients' lab data in flowsheets or graphical form to compare results and see trends?
  • Can the system document medication administration?
  • Does the system support the standards identified and recommended by the Health Information Technology Standards Panel in the latest version of its HITSPTP13 document?
  • Does the system support two-factor authentication in alignment with NIST 800-63 Level 3 Authentication?
  • If the system provides access to PHI using a web browser via HTML over HTTP, does the system provide data encryption capability via SSL?
  • Does the system provide the ability to display the patient's allergy list, including the date of entry? (CCHIT certified 2011 Ambulatory EHR criteria # FN 05.12)
  • Does the system provide the ability to display CCD (Continuity of Care Documents) and file them in the EHR, where the summary document must include information on - patient demographics, medication list, and medication allergy list? (CCHIT certified 2011 Ambulatory EHR criteria # IO-AM 10.10)

Acquisition and Implementation Cost (25%)

  • What is the total financial investment required to acquire and fully implement the proposed solution? Break out costs for the following and detail any time or resource constraints for each item. If additional resources or time are required, what are the additional costs per unit?
    • Application
    • Per-user licenses (and tier pricing, if offered)
    • Database (if treated as a separate item)
      • Data integration from legacy systems
    • Application documentation
    • Annual maintenance agreement
    • Training
      • End-user
      • System administrators
      • In-house developers
      • Re-current training
    • Professional services:
      • Project management
      • Software development or customizations
      • Technical support
  • What is the estimated time that it will take for the investment of the vendor to pay for itself based on projected savings?
  • Does the system allows to track and detail log audits and transactions made by users?
  • Does the system have the functionality to generate customizable reports given a determinate frequency?
  • What is the cost for upgrading when new releases are available?
  • What is the projected timeline for upgrades? What training support is included if any, and what is covered by the service fees?
  • Upon go-live, business imperatives will define a timeframe within which workflows and processes are to to be run in parallel, until agreed upon metrics/milestones are achieved and the system is signed off into full production: which system is more flexible, less complex, and least disruptive in terms of workflow and process change/adjustments?
  • Implementation costs associated with organizational changes to clinician, administrative, etc. processes should be considered in calculating ROI.
  • At what point are implemenation costs (and other factors) considered too extreme and a GO/NO decision is given serious consideration?
  • All implementation risks must be calculated for "impact and probability" and rank accordingly when analyzing EHR solutions.
  • Does the vendor provide functionality to track ROI? If not what reports can be produced that could help track usage?

Hardware Platform and Technical Requirements (20%)

  • Which vendor uses less proprietary systems and has partnerships with established industry players? Are they certified developers for the platform they are using? Is the vendor's system and devleopment lifecycle management aligned with their respective industry partners?
  • Compared retail broker workstation alternatives, including proprietary and third party packages, based on user requirements and technology/operational considerations. Project included EMR development and response evaluation
  • Identify all hardware and system software that would be required to support an implementation of our scope and scale.
  • How well is the vendor able to meet our technical requirements ?
  • How flexible is the vendor to change requirements and what are the cost?
  • How hard is to implement new modules within the system?
  • How well is the vendor able to create and maintain interfaces to our existing systems?
  • How well is the vendor able to migrate our data from actual system to the new, if it is neccesary?
  • Is the vendors's software created with the most adequate developing technology? Platform? Language? Databases?
  • Will we do periodic updates, or will the vendor do this as part of the contract? How often are they released?
  • Do we plan on-site or remote hosting for the system?
  • If we plan to utilize remote hosting, how safe, sound, and accessible are these data?
  • Is the hardware user friendly in all departments? (i.e. nursing, pharmacy, pulmonary,..etc.)
  • Will the vendor provide a mock system for testing?
  • What is the responding duration of each entry and information extraction? Is there any delay?
  • How the data structure for storing data? Is it expandable? Is it easy to be extracted for future analysis?
  • How often does the system need to be updated or serviced?
  • As the technology is changing and the hospital decides to upgrade their hardware, will your system be able to handles these changes or will we have to buy a new program?
  • What language is the vendor's systems software coded in?
  • Scaling:
    • To how many patients has the system been scaled?
    • To how many users has the system been scaled?
    • Has the system been scaled to the size of the evaluating institution?
    • What are the hardware/software licensing implications of growth?
  • Is it necessary to buy (possibly expensive) hardware from the vendor, or can we just install their software on our standard desktop computers (at least for the clients)?
  • For already existing software functionality, does the vendor employ reliable and trustworthy software like an open-source MySQL or Apache server? Or do they develop their own system, or do we have to license a proprietary one (extra cost)?
  • How the vendor estimates the total amount of users and licenses needed? Will they be concurrent user licenses or asynchronic?
  • Will technical support remain active even if the Hospital is running a non upgrade system? For how long?
  • Does the system use DICOM standards for the transmission of image data?
  • How frequently does the vendor provide patch upgrades for the product?
  • Is the system using standards such as Snomed, ICD 10; HL7 Version 2 or 3; HL7 infobutton…)
  • If this is a hosted solution, how many computing facilities does the vendor have capable of hosting the application and where are they located? What are the vendors' security and disaster recovery plans?
  • How do you handle redundancy for clinical records, like off-site backups and such?
  • For Mobile EMR implementations, how do you handle communication to remote wireless clients? security interfaces? types of devices and minimum requirements?
  • Is the database program one likely to be waning in use and are personnel available at my site with adequate skills/knowledge to provide support post implementation?
  • What database and programming tools (such as instrumented code) are in place for quick problem resolution?
  • If software licenses are sold per physician or user, how are part-time physicians, physician assistants and/or advanced nurse practitioners calculated? American Medical Association

Implementability (15%)

  • How does the vendor compare in KLAS rankings of similar systems and applications?
  • How much time, effort, and resources will be required to successfully implement?
  • What is the vendors track record for successfully implementing its system in similar settings?
  • Do the vendors provide detailed plan for implementation, training and quality control?
  • Will the vendor supply on site support when we "go-live" and how long will they be available?
  • How responsive is the vendor to emergencies? Do they have a quick and accurate response to support issues?
  • Do they have a backup or alternative plan if the system or partial of the system is not working? The situation could be out of energy or computer virus attack.
  • How long is the training that is required for each subgroup to fully implement the system?
  • What are the training requirements for the vendor? For the clinic/hospital?
  • What is the vendor's track record for successfully training a new system for your clinic/hospital size?
  • What training manuals, user guides, on-line training assets, and any other training materials will be made available to the clinic/hospital?
  • Will the implementation require a consulting "team" and how much will this cost?
  • Is the vendor sufficiently well known that most health care consulting firms have familiarity with the system?
  • How will legacy patient record data be integrated into the new system?
  • Is the vendor's system compliant with the Health Insurance Portability and Accountability Act (HIPAA)?
  • Is the vendor's system compliant with the Patient Safety and Quality Improvement Act (PSQIA) Patient Safety Rule?
  • How is the evaluation procedure for the approval of the ongoing stages of implementation? How objective will be the defined how to proceed with the approval and payment?
  • Is the vendor's system compliant with ISO standards for EHRs?
  • Does the system have a proper data recovery plan in case of an crash or any other unexpected event? Can the system be restored without any data loss? Does the vendor have a proper disaster recovery plan?
  • How does a legacy Practice Management System integrate with the EHR system? Is there a list of successful PM vendor integrations? MikeField 20:47, 29 January 2010 (CST)
  • What is the legacy Practice Management System conversion process for legacy data import into the EHR system? Time/cost and loose-ends? What reference clients are there for particular legacy PM systems? MikeField 20:47, 29 January 2010 (CST)
  • Does the vendor have a List of Lessons Learned from previous implementations?
  • Does the vendor have a legal license to essential code sets, such as the AMA Current Procedural Terminology (CPT®) file? Will the vendor maintain this license annually as part of your service or maintenance agreement?American Medical Association
  • Does the vendor offer a “test environment” in which upgrades are loaded to allow you time to test and learn their functionality without affecting your live system? American Medical Association
  • What is the vendor's rate for on time & under budget implementations?
  • Are other consulting firms certified to implent the vendor's product? If so will the vendor provide system updates and customer service if system issues (not related to implementation) arise?

Vendor Partnership and On-going viability (10%)

  • Please provide audited financial statements for the last five years, including Balance Sheet, Income Statement, and Statement of Cash Flows, as well as any accompanying footnotes.
  • Provide a list of customers who have implemented the systems and applications that you are recommending to us.
  • Create a list of vendor selection criteria, and evaluate the vendors and software: review responses to the RFP; schedule demonstrations; check references; and, use proven tools/templates for assessing and consistently comparing vendors.
  • Research available EMR systems and vendors: determine if your jurisdiction has a "pre-approved" vendor list; develop checklists & questionnaires to assist with product comparison; and, prepare a Request for Proposals (RFP).
  • What type of long-term relationship do we think we can expect from the vendor?
  • How stable vs risky is the company?
    • Request that the software source code be put in escrow with specific provisions established under which it could be accessed.
    • How long has the company been in business?
    • How large is the company?
      • How much money did they bring in last quarter? year? 3-years?
      • How much is the Vendor company revenue in $M?
      • How much is the vendor company net income in $M?
      • How much are the total assets and liabilities of the company?
      • What is the growth strategy of the company?
      • Are they a private or a public company?
      • How many employees?
  • How many contracts have they should to Health Care Systems like yours?
    • Can you contact and/or visit a few representative samples?
  • How many new contracts have they signed in the past year?
  • How many uninstall's have been done the last year?
  • Does word of mouth support that the company believes in service after the sale?
  • Joan Breuer, Ph.D. 01/27/2010 17:22 The HCF needs formal documentation from other HCFs who have used this Vendor’s product signed by the other HCFs’ CEOs, prepared by an attorney. Note: “Word of mouth” does not stand up in court, especially when we are talking about millions of dollars.
  • What is the vendor's cost per year after implementation?
  • How the vendor provides technical support? On site or remote? Package or individual project based?
  • What provisions exist in the contract for termination and/or penalties if the application does not perform as specified or if the vendor fails to provide any services that are agreed to?
    • Ensure that all application specifications that can be measured or quantified are incorporated into the contract.
    • Ensure that all services verbally offered by the vendor are incorporated into the contract.
    • Does the vendor have service level agreements (SLAs) with appropriate penalties for technical support of the application?
    • Does the system’s implementation plan consider proper risks evaluation and mitigation strategies?
  • An important consideration is whether the vendor or any of its industry partners have done an analysis of processes in healthcare
  • In a HIPAA complaint product, one should consider the feasibility of implementing future mandates. The cost should be considered in terms of both money and time.
  • Obtain references of current customers that are similar in size, have similar patient population, and have similar required functionality needs to your practice and evaluate the ease of implementation, current satisfaction, and costs.
  • What kind of service level agreements are offered by the vendor and what is their track record for maintaining those SLAs?

Future Vision (5%)

  • Does the vendor have a meaningful Product Lifecycle that defines major and minor releases, their associated costs, and delivery (push or pull) methods?
  • What is the vendor's five-year strategic plan?
  • What percentage of revenue is being re-invested into R&D?
  • How does the company capture/communicate client concerns for re-engineering? i.e. Is this a learning company?
  • What "game changers" are on the drawing board? Are these related to one of our organization's core goals?
  • Compared retail broker workstation alternatives, including proprietary and third party packages, based on user requirements and technology/operational considerations. Project included EMR development and response evaluation
  • Lifespan and Staying Power: One should consider the track record of the vendor and viability as a company to sustain the ups and downs of the industry. One sub-criteria in this category often used is how long the vendor has been around.
  • Does the vendor plan to offer interoperability options in the event of a nationally implemented CIS?
  • What is the plan to support smart phone applications to enhance interoperability and custom accessibility of EHR data while maintaining security? (DROID, IPHONE)
  • Upon the termination of a license or agreement, is there an orderly process for you to extract your data? This is applicable if you access the vendor’s software within an application service provider (ASP) model.American Medical Association

Extra Credit (optional)

  • It is not uncommon for developers or implementers to impose lock-in features in order to protect their IP rights. How mature is the support community for the particular vendor and is there a marketplace for support and services to their product or is support limited to their own team?
  • A multidisciplinary team, usually a subset of the decision team consisting of a physician, IT personnel and a senior management must conduct a site visit to one of the vendor customer sites and vendor headquarters. Informal discussions at this level were quite informative as they bridged the gap between theory and practice for actual software implementation and usage.
  • An unstructured “drive-by” demonstration of the system must be arranged for, wherein the vendors conduct product demos, demonstrate product futures and distribute promotional materials for a majority of the employees to get the feel of the new systems. It facilitates building a comfort level and relationship with the vendor and the product offered.
  • Can the company demonstrate tangible ways that use of the product can increase hospital revenue? If yes, can you validate this/these claims?
  • Is the vendor willing to contract to go "at risk" for any part(s) of the contract?
  • What other services does the company offer (especially useful for rural or small hospitals or practices)? e.g. consultants to do special projects, data transmission/claims, billing
  • Can the record be accessed at home by clinicians and patients, or do they have to be on-site at the facility?
  • Is the vendor capable of adapting to emerging technology such as open source programming, cloud computing, "tablet" PCs/Macs, demand for smartphones (e.g. iPhone), and Web 2.0 technology?
  • Is the vendor capable of integrating it's system with a personal health record and allowing more patient control?
  • Can the company provide return on investment analysis?
  • Does the system have e-prescribing functionality?
  • Vendors may not be chosen if the current managers have been convicted of or have had a civil judgment for the following: fraud, antitrust violations, embezzlement, theft, etc...
  • How can this EHR be an asset for the solo practitioner who sees this technology primarily as a financial drain? MikeField 20:54, 29 January 2010 (CST)