Difference between revisions of "Vendor Selection Criteria: Usability"

From Clinfowiki
Jump to: navigation, search
(Created new headings, created new section "Outcomes of Poor Usability", added references, rearrangement of material, fixed other minor errors such as spelling and grammar)
 
(7 intermediate revisions by 4 users not shown)
Line 1: Line 1:
−
Previous studies demonstrated how usability measurements can be applied to the evaluation of EHR systems; however, most of these studies were conducted post implementation. If significant problems have been discovered with usability, at this point in the system development
+
Previous studies demonstrated how usability measurements can be applied to the evaluation of EHR systems; however, most of these studies were conducted post implementation. If significant problems have been discovered with usability at this point in the system development life cycle, it is usually too late to make any major modifications to the EHR system. Therefore, it is important to consider the usability criteria in early stages of the EHR implementation, and particularly during the process of vendor and product selection.  <ref name="Saiku"> I. Saiku. Including usability in the procurement process of healthcare IT Systems. <http://www.soberit.hut.fi/T-121/shared/thesis/di-Inkeri-Saiku.pdf</ref>   
−
life cycle it is usually too late to make any major modifications to the EHR system. Therefore, it is important to consider the usability criteria in early stages of the EHR implementation, and particularly during the process of vendor and product selection.  <ref name="Saiku"> I. Saiku. Including usability in the procurement process of healthcare IT Systems. <http://www.soberit.hut.fi/T-121/shared/thesis/di-Inkeri-Saiku.pdf</ref>   
+
  
−
A 2015 report by Ratwani et al. reviews complaince with the US Department of Health and Human Services’ Office of the National Coordinator for Health Information Technology (ONC) requirements for EHR vendor usability practices.  Compliance with the usability standards was weak, and the authors suggest that this lack of focus on usability may contribute significantly to end user frustration.  <ref name="ratwani">Ratwani, R. M., Benda, N. C., Hettinger, A. Z., & Fairbanks, R. J. (2015). Electronic Health Record Vendor Adherence to Usability Certification Requirements and Testing Standards. JAMA, 314(10), 1070–1071. http://jama.jamanetwork.com/article.aspx?articleid=2434673</ref>
+
== Definition ==
  
 
Usability describes how easy it is for users to accurately and efficiently accomplish a task while using a system. Usability can be defined in terms of three main attributes: useful, usable, and satisfaction. Even in the presence of all necessary features, the system can still have poor reliability, mismatch between interface and tasks, and lack critical functionalities that can impact end-user ability to conduct work. <ref name="Usability">AHRQ. EHR Usability Toolkit: A Background Report
 
Usability describes how easy it is for users to accurately and efficiently accomplish a task while using a system. Usability can be defined in terms of three main attributes: useful, usable, and satisfaction. Even in the presence of all necessary features, the system can still have poor reliability, mismatch between interface and tasks, and lack critical functionalities that can impact end-user ability to conduct work. <ref name="Usability">AHRQ. EHR Usability Toolkit: A Background Report
 
on Usability and Electronic Health Records. https://healthit.ahrq.gov/sites/default/files/docs/citation/EHR_Usability_Toolkit_Background_Report.pdf</ref>
 
on Usability and Electronic Health Records. https://healthit.ahrq.gov/sites/default/files/docs/citation/EHR_Usability_Toolkit_Background_Report.pdf</ref>
 +
 +
 +
== Methods ==
  
 
The following methods can be used to measure usability:
 
The following methods can be used to measure usability:
Line 19: Line 21:
 
on Usability and Electronic Health Records. https://healthit.ahrq.gov/sites/default/files/docs/citation/EHR_Usability_Toolkit_Background_Report.pdf</ref>
 
on Usability and Electronic Health Records. https://healthit.ahrq.gov/sites/default/files/docs/citation/EHR_Usability_Toolkit_Background_Report.pdf</ref>
  
 +
 +
== Characteristics ==
 +
 +
Some key characteristics to consider when evaluating EHR usability:
 +
* Does the system have an easy to read interface with an user centered design (UCD)?<ref name="EHR usability">Ratwani, R.M., Fairbanks, R.J., Hettinger, A.Z., Benda, N.C. (2015). Electronic health record usability: analysis of the user-centered design processes of eleven electronic health record vendors. ''Journal of the American Medical Informatics Association'', 0:1–5. doi: 10.1093/jamia/ocv050</ref>
 +
* Speed: How much time does it take for the user to complete a task? How much time between switching from one screen to another?
 +
* Juxtaposition: Are similar options placed too close to one another? Is it easy for the user to click on the wrong option?
 +
* Aesthetics: Is the EHR design aesthetically pleasing? Does the design make it intuitive for first time users to work with the EHR?<ref name= " Usability Characteristics to Consider"> Ash, Berg, & Coiera (2004). Some Unintended Consequences of Information Technology in Health Care: The Nature of Patient Care Information System-related Errors. Journal of the American Medical Informatics Association (11)104-112. DOI 10.1197/jamia.M1471 </ref>
 +
* Are the icons consistent and provide intuitive meaning to the user?<ref name="EHR usability"></ref>
 +
* Will the interface fully support the user's workflow and information needs?<ref name="EHR usability"></ref>
 +
* Can it display data that makes it easy to draw insights for the user?<ref name="EHR usability"></ref>
 +
 +
 +
== TURF Usability Framework ==
 +
 +
When selecting an EMR from a Vendor, it is important to have a framework in which to compare each module of the EMR with the workflow of the users.  Usability is how useful, usable and satisfying a system is for the user and how well the user is able to complete a given task.  In a recent study, it was found the TURF analysis framework can be used to assess and create a satisfying and less complex interface.  TURF stands for
 +
 +
*'''''T'''''ask Analysis
 +
*'''''U'''''ser Analysis
 +
*'''''R'''''epresentational Analysis
 +
*'''''F'''''unctional Analysis
 +
 +
This framework can be used to leverage and compare the usability of the EMR vendor being considered.<ref name = "usability framework">Harrington,C., Wood,R., Breuer,J., Pinzon, O., Howell, R., et al. (2011).  Using a unified usability framework to dramatically improve the usability of an EMR module. American Medical Informatics Association. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3243270/</ref>
 +
 +
=== Reports on TURF ===
 +
 +
A paper written by Jiajie Zhang and Muhammad Walji (2011) describes TURF as a framework that "(1) describing, explaining, and predicting usability differences in terms of the representation effect; (2) for defining, evaluating, and measuring usability objectively; (3) for designing built-in good usability; and (4) for developing EHR usability guidelines and standards." <ref name= "Zhang & Walji"> Zhang, J., & Walji, M. (2011). TURF: toward a unified framework of EHR usability. Journal of Biomedical Informatics, 44(6), 1056-1067. doi:10.1016/j.jbi.2011.08.005</ref> They also define usability as useful, usable, and satisfying.
 +
 +
*Useful is defined as a system where users "can accomplish their goals independent of how the system is implemented." <ref name= "Zhang & Walji"> Zhang, J., & Walji, M. (2011). TURF: toward a unified framework of EHR usability. Journal of Biomedical Informatics, 44(6), 1056-1067. doi:10.1016/j.jbi.2011.08.005</ref>
 +
*Usable is defined as a system that is easy to use and learn and is tolerant of mistakes. <ref name= "Zhang & Walji"> Zhang, J., & Walji, M. (2011). TURF: toward a unified framework of EHR usability. Journal of Biomedical Informatics, 44(6), 1056-1067. doi:10.1016/j.jbi.2011.08.005</ref>
 +
*Satisfying is defined as users having good impressions concerning usefulness and usability. <ref name= "Zhang & Walji"> Zhang, J., & Walji, M. (2011). TURF: toward a unified framework of EHR usability. Journal of Biomedical Informatics, 44(6), 1056-1067. doi:10.1016/j.jbi.2011.08.005</ref>
 +
 +
 +
== Outcomes of Poor Usability ==
 +
 +
Many different articles have been released relaying information from different experts in the field of Health IT. These articles are written with a clear attempt to help vendors attain credibility for usability. One article points towards five different criteria that could be useful for creating better EHR usability; the five points are data entry, errors, metrics, training, and what the article calls “bottom line.”<ref name="insider">5 health IT insiders offer their takes on EHR usability http://www.healthcareitnews.com/news/five-health-it-insiders-offer-five-takes-ehr-usability</ref> Data entry, according to JiaJie Zhang in the article, is not optimized and can be improved.<ref name="insider"></ref> With the lack of usability, errors can be unprecedented and lead to unnecessary adverse effects, as noted by Robert Tennant.<ref name="insider"></ref> The goodness of usability can offer the vendors valuable metrics, as mentioned by Mary Kate Foley.<ref name="insider"></ref> The ability to have usability comes with the need for adequate training according to Dr. William F. Bria.<ref name="insider"></ref> And after all is said and done, the physicians would like to see good changes from the usability of EHRs, as noted by Rosemarie Nelson.<ref name="insider"></ref>
 +
 +
=== Special 2015 Report ===
 +
 +
A 2015 report by Ratwani et al. reviews compliance with the US Department of Health and Human Services’ Office of the National Coordinator for Health Information Technology (ONC) requirements for EHR vendor usability practices.  Compliance with the usability standards was weak, and the authors suggest that this lack of focus on usability may contribute significantly to end user frustration.  <ref name="ratwani">Ratwani, R. M., Benda, N. C., Hettinger, A. Z., & Fairbanks, R. J. (2015). Electronic Health Record Vendor Adherence to Usability Certification Requirements and Testing Standards. JAMA, 314(10), 1070–1071. http://jama.jamanetwork.com/article.aspx?articleid=2434673</ref>
  
  
 
== References ==
 
== References ==
 
<references/>
 
<references/>

Latest revision as of 21:23, 23 September 2015

Previous studies demonstrated how usability measurements can be applied to the evaluation of EHR systems; however, most of these studies were conducted post implementation. If significant problems have been discovered with usability at this point in the system development life cycle, it is usually too late to make any major modifications to the EHR system. Therefore, it is important to consider the usability criteria in early stages of the EHR implementation, and particularly during the process of vendor and product selection. [1]

Definition

Usability describes how easy it is for users to accurately and efficiently accomplish a task while using a system. Usability can be defined in terms of three main attributes: useful, usable, and satisfaction. Even in the presence of all necessary features, the system can still have poor reliability, mismatch between interface and tasks, and lack critical functionalities that can impact end-user ability to conduct work. [2]


Methods

The following methods can be used to measure usability:

  • Literature Search
  • Heuristic Evaluation
  • Cognitive walkthrough
  • Laboratory Testing
  • Remote Evaluation
  • Usability Questionnaires
  • Predictive Modeling
  • Risk Assessment
  • Workload Evaluation [2]


Characteristics

Some key characteristics to consider when evaluating EHR usability:

  • Does the system have an easy to read interface with an user centered design (UCD)?[3]
  • Speed: How much time does it take for the user to complete a task? How much time between switching from one screen to another?
  • Juxtaposition: Are similar options placed too close to one another? Is it easy for the user to click on the wrong option?
  • Aesthetics: Is the EHR design aesthetically pleasing? Does the design make it intuitive for first time users to work with the EHR?[4]
  • Are the icons consistent and provide intuitive meaning to the user?[3]
  • Will the interface fully support the user's workflow and information needs?[3]
  • Can it display data that makes it easy to draw insights for the user?[3]


TURF Usability Framework

When selecting an EMR from a Vendor, it is important to have a framework in which to compare each module of the EMR with the workflow of the users. Usability is how useful, usable and satisfying a system is for the user and how well the user is able to complete a given task. In a recent study, it was found the TURF analysis framework can be used to assess and create a satisfying and less complex interface. TURF stands for

  • Task Analysis
  • User Analysis
  • Representational Analysis
  • Functional Analysis

This framework can be used to leverage and compare the usability of the EMR vendor being considered.[5]

Reports on TURF

A paper written by Jiajie Zhang and Muhammad Walji (2011) describes TURF as a framework that "(1) describing, explaining, and predicting usability differences in terms of the representation effect; (2) for defining, evaluating, and measuring usability objectively; (3) for designing built-in good usability; and (4) for developing EHR usability guidelines and standards." [6] They also define usability as useful, usable, and satisfying.

  • Useful is defined as a system where users "can accomplish their goals independent of how the system is implemented." [6]
  • Usable is defined as a system that is easy to use and learn and is tolerant of mistakes. [6]
  • Satisfying is defined as users having good impressions concerning usefulness and usability. [6]


Outcomes of Poor Usability

Many different articles have been released relaying information from different experts in the field of Health IT. These articles are written with a clear attempt to help vendors attain credibility for usability. One article points towards five different criteria that could be useful for creating better EHR usability; the five points are data entry, errors, metrics, training, and what the article calls “bottom line.”[7] Data entry, according to JiaJie Zhang in the article, is not optimized and can be improved.[7] With the lack of usability, errors can be unprecedented and lead to unnecessary adverse effects, as noted by Robert Tennant.[7] The goodness of usability can offer the vendors valuable metrics, as mentioned by Mary Kate Foley.[7] The ability to have usability comes with the need for adequate training according to Dr. William F. Bria.[7] And after all is said and done, the physicians would like to see good changes from the usability of EHRs, as noted by Rosemarie Nelson.[7]

Special 2015 Report

A 2015 report by Ratwani et al. reviews compliance with the US Department of Health and Human Services’ Office of the National Coordinator for Health Information Technology (ONC) requirements for EHR vendor usability practices. Compliance with the usability standards was weak, and the authors suggest that this lack of focus on usability may contribute significantly to end user frustration. [8]


References

  1. ↑ I. Saiku. Including usability in the procurement process of healthcare IT Systems. <http://www.soberit.hut.fi/T-121/shared/thesis/di-Inkeri-Saiku.pdf
  2. ↑ 2.0 2.1 AHRQ. EHR Usability Toolkit: A Background Report on Usability and Electronic Health Records. https://healthit.ahrq.gov/sites/default/files/docs/citation/EHR_Usability_Toolkit_Background_Report.pdf
  3. ↑ 3.0 3.1 3.2 3.3 Ratwani, R.M., Fairbanks, R.J., Hettinger, A.Z., Benda, N.C. (2015). Electronic health record usability: analysis of the user-centered design processes of eleven electronic health record vendors. Journal of the American Medical Informatics Association, 0:1–5. doi: 10.1093/jamia/ocv050
  4. ↑ Ash, Berg, & Coiera (2004). Some Unintended Consequences of Information Technology in Health Care: The Nature of Patient Care Information System-related Errors. Journal of the American Medical Informatics Association (11)104-112. DOI 10.1197/jamia.M1471
  5. ↑ Harrington,C., Wood,R., Breuer,J., Pinzon, O., Howell, R., et al. (2011). Using a unified usability framework to dramatically improve the usability of an EMR module. American Medical Informatics Association. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3243270/
  6. ↑ 6.0 6.1 6.2 6.3 Zhang, J., & Walji, M. (2011). TURF: toward a unified framework of EHR usability. Journal of Biomedical Informatics, 44(6), 1056-1067. doi:10.1016/j.jbi.2011.08.005
  7. ↑ 7.0 7.1 7.2 7.3 7.4 7.5 5 health IT insiders offer their takes on EHR usability http://www.healthcareitnews.com/news/five-health-it-insiders-offer-five-takes-ehr-usability
  8. ↑ Ratwani, R. M., Benda, N. C., Hettinger, A. Z., & Fairbanks, R. J. (2015). Electronic Health Record Vendor Adherence to Usability Certification Requirements and Testing Standards. JAMA, 314(10), 1070–1071. http://jama.jamanetwork.com/article.aspx?articleid=2434673
Retrieved from "http://clinfowiki.org/wiki/index.php?title=Vendor_Selection_Criteria:_Usability&oldid=25118"