Difference between revisions of "Whole system measures"

From Clinfowiki
Jump to: navigation, search
 
(Example of Hospital Readmission Percentage)
 
(3 intermediate revisions by 2 users not shown)
Line 1: Line 1:
Whole System Measures
+
The Institute for Healthcare Improvement (IHI) '''Whole System Measures (WSM)''' approach to quality improvement and coordination of quality activities provides a framework upon which future informational systems should be structured.
16
+
 
Innovation Series 2007
+
Far too often quality improvement activities are structured with a very limited scope and produce only a marginal effect on overall quality or outcomes. QI activities are rarely coordinated and complimentary. More often QI activities are conducted within a single, isolated department or division of a healthcare organization. In many cases the reduced scope was mandated by the information systems used. Because different departments or divisions used different IT systems, the QI project was limited to the relevant IT system.  
Copyright ©2007 Institute for Healthcare Improvement
+
All rights reserved. Individuals may photocopy these materials for educational, not-for-profit uses,
+
New opportunities emerge as organizations implement enterprise wide IT systems or develop more interactive best of breed system relationships. QI activities can now be constructed across departments or divisions. Organizations now wonder how they take advantage of these new opportunities with the little resources they have available. The IHI WSM provides a useful roadmap for organizations to move forward and maximize the use of their QI resources.  
provided that the contents are not altered in any way and that proper attribution is given to IHI as
+
 
the source of the content. These materials may not be reproduced for commercial, for-profit use in
+
== Healthcare improvement ==
any form or by any means, or republished under any circumstances, without the written permission
+
 
of the Institute for Healthcare Improvement.
+
IHI used the Institute of Medicine’s 6 aims for healthcare improvement – safe, effective, efficient, timely, patient centered, and equitable – to build their 13 WSM. The 13 WSM are referenced as Big Dots up to which smaller dots (focused, limited scope QI projects). In this manner previous and ongoing QI projects can be coordinated and incorporated into an organization wide program.  
How to cite this paper:
+
Martin LA, Nelson EC, Lloyd RC, Nolan TW. Whole System Measures. IHI Innovation Series white
+
The 13 measures and associated IOM Aim are:
paper. Cambridge, Massachusetts: Institute for Healthcare Improvement; 2007.
+
 
(Available on www.IHI.org)
+
# Rate of Adverse Events :# Safety
Acknowledgements:
+
# Incidence of Nonfatal Occupational Injuries or Illnesses# Safety
The Institute for Healthcare Improvement is grateful to the many organizations and individuals that
+
# Hospital standardized Mortality Ratio (HSMR) : Effective
have contributed to this work. The Whole System Measures would not be possible without the experts
+
# Unadjusted Raw Mortality Percentage : Effective
who helped create and refine them, and the early pioneer organizations in IMPACT and Pursuing
+
# Functional Health Outcomes Scores : Effective
Perfection that tested the measures and graciously shared their experiences so that many could learn
+
# Hospital Readmission Percentage :    Effective
from their journey. IHI also acknowledges the contribution of Craig Melin, President and CEO of
+
# Reliability of Core Measures :Effective
Cooley Dickinson Hospital, for his willingness to share the hospital’s experience with the Whole
+
# Patient Satisfaction with Care Score : Patient-Centered
System Measures. We thank Donna Truesdell, MS, RN, Frank Davidoff, MD, Donald Goldmann,
+
# Patient Experience Score :Patient-Centered
MD, and Lloyd Provost, MS, for their critical review of the paper.
+
# Days to Third Next Available Appointment : Timely
IHI also thanks staff members Jane Roessner, PhD, and Val Weber for their editorial review and
+
# Hospital Days per Decedent During the Last 6 Months of Life# Efficient
assistance with this paper.
+
# Health Care Cost per Capita :Efficient
For reprints requests, please contact:
+
# Equity (stratification of WSM) :Equitable
Institute for Healthcare Improvement, 20 University Road, 7th Floor, Cambridge, MA 02138
+
 
Telephone (617) 301-4800, or visit www.IHI.org.
+
As health IT matures the WSM can guide the application of systems and projects to produce enterprise wide improvement. All 13 WSM are applicable to the outpatient environment while 10 are applicable to inpatient environment.  
The Institute for Healthcare Improvement (IHI) is a not-for-profit organization leading the
+
 
improvement of health care throughout the world. IHI helps accelerate change by cultivating
+
The IHI Executive Summar reads, The aim of this white paper is to describe and promote the use of a system of metrics, called the Whole System Measures, to measure the overall quality of a health system and to align improvement work across a hospital, group practice, or large health care system. The Institute for Healthcare Improvement and colleagues developed the Whole System Measures, a balanced set of system-level measures, to supply health care leaders and other stakeholders with data that enable them to evaluate their health systems’ overall performance on core dimensions of quality and value, and that also serve as inputs to strategic quality improvement planning. Properly constructed, the Whole System Measures should complement existing measures that organizations use to evaluate the performance of their heath care systems. The Whole System Measures, because they are intended to focus on important system-level measures, are limited to a small set of 13 measures that are not disease or condition-specific. One objective for developing the Whole System Measures was to also provide a view of performance that reflects care provided in different sites—both inpatient and outpatient—and across the continuum of care.
promising concepts for improving patient care and turning those ideas into action. Thousands
+
 
of health care providers participate in IHI’s groundbreaking work.
+
There was a time when it was mainly the providers of care who were concerned about health care quality data; this is no longer the case. Today, not only are the providers of care keenly focused on the processes and outcomes of health care delivery, but the consumers of health care—as well as managers, boards, purchasers, and policy makers—are also becoming increasingly interested in being shown that health care services are safe, effective, patient-centered, timely, efficient, and equitable. Many of the questions that drive this growing interest in health care quality measurement can only be answered with data. The WSM provide the following:
We have developed IHI’s Innovation Series white papers as one means for advancing our
+
 
mission. The ideas and findings in these white papers represent innovative work by IHI and
+
* A useful conceptual framework for organizing measures of health care quality; and
organizations with whom we collaborate. Our white papers are designed to share the problems
+
* A specific set of quality metrics that can contribute to a health care organization’s family of measures, balanced scorecard, or dashboard of strategic performance measures. A central premise of IHI’s work on the WSMs is that any family of measures should reflect a balance among structures, processes, and outcomes. A balanced set of system-level measures is
IHI is working to address, the ideas we are developing and testing to help organizations make
+
breakthrough improvements, and early results where they exist.
+
Whole System Measures
+
Innovation Series 2007
+
Authors:
+
Lindsay A. Martin, MSPH, Senior Research Associate, IHI
+
Eugene C. Nelson, DSc, MPH, Professor, Dartmouth Medical School, and Director
+
of Quality Administration, Dartmouth-Hitchcock Medical Center
+
Robert C. Lloyd, PhD, Executive Director of Performance Improvement, IHI
+
Thomas W. Nolan, PhD, Senior Fellow, IHI
+
Innovation Series: Whole System Measures
+
© 2007 Institute for Healthcare Improvement
+
1
+
Executive Summary
+
The aim of this white paper is to describe and promote the use of a system of metrics, called the
+
Whole System Measures, to measure the overall quality of a health system and to align improvement
+
work across a hospital, group practice, or large health care system. The Institute for Healthcare
+
Improvement and colleagues developed the Whole System Measures, a balanced set of system-level
+
measures, to supply health care leaders and other stakeholders with data that enable them to
+
evaluate their health systems’ overall performance on core dimensions of quality and value, and that
+
also serve as inputs to strategic quality improvement planning. Properly constructed, the Whole
+
System Measures should complement existing measures that organizations use to evaluate the performance
+
of their heath care systems. The Whole System Measures, because they are intended to
+
focus on important system-level measures, are limited to a small set of 13 measures that are not disease-
+
or condition-specific. One objective for developing the Whole System Measures was to also
+
provide a view of performance that reflects care provided in different sites—both inpatient and
+
outpatient—and across the continuum of care.
+
Context and Background
+
There was a time when it was mainly the providers of care who were concerned about health care quality
+
data; this is no longer the case. Today, not only are the providers of care keenly focused on the processes
+
and outcomes of health care delivery, but the consumers of health care—as well as managers, boards, purchasers,
+
and policy makers—are also becoming increasingly interested in being shown that health care
+
services are safe, effective, patient-centered, timely, efficient, and equitable. Many of the questions that
+
drive this growing interest in health care quality measurement can only be answered with data.
+
The Institute for Healthcare Improvement (IHI) is committed to helping health care organizations
+
develop, implement, and use measurement systems that enable them to evaluate the efficiency and
+
effectiveness of the services they provide. IHI and colleagues therefore developed the Whole System
+
Measures, a balanced set of system-level measures. Specifically, the Whole System Measures (WSMs)
+
provide the following:
+
A useful conceptual framework for organizing measures of health care quality; and
+
A specific set of quality metrics that can contribute to a health care organization’s family
+
of measures, balanced scorecard, or dashboard of strategic performance measures.
+
A central premise of IHI’s work on the WSMs is that any family of measures should reflect a
+
balance among structures, processes, and outcomes.1-3 A balanced set of system-level measures is
+
 
needed to provide leaders and other stakeholders with data that:
 
needed to provide leaders and other stakeholders with data that:
Show performance of their health care system over time;
+
* Show performance of their health care system over time;
Allow the organization to see how it is performing relative to its strategic plans
+
* Allow the organization to see how it is performing relative to its strategic plans for improvement;
for improvement;
+
* Allow comparisons to other similar organizations; and
Allow comparisons to other similar organizations; and
+
* Serve as inputs to strategic quality improvement planning.
Serve as inputs to strategic quality improvement planning.
+
Institute for Healthcare Improvement Cambridge, Massachusetts 2
+
In 2003 a group of approximately 10 people from the United Kingdom, Sweden, and the United States met to discuss the idea of developing a method for measuring the quality of care at the level of a health system. They believed that although many helpful quality measures existed and more were being rapidly created, high-level measures reflecting the overall quality of a health system were largely missing. They also believed that the important work to measure hospital quality—based on an overall mortality measure called the hospital standardized mortality ratio (HSMR)—done by Sir Brian Jarman, MD, Emeritus Professor and head of the Dr Foster Unit at Imperial College in London, could serve as a model for a high-level quality measure. Together the members of this group, including leaders from IHI, believed they could develop a small set of measures to go beyond
The aim of IHI’s WSMs initiative was to develop, test, and use a small set of measures that focuses
+
Jarman’s HSMR metric. After several months of dialogue and planning, the following health systems began to test the initial
on quality of care and is aligned with the Institute of Medicine’s (IOM’s) six dimensions of quality
+
(i.e., care that is safe, effective, patient-centered, timely, efficient, and equitable).4 Properly
+
constructed, the WSMs should complement existing measures (e.g., utilization, program growth,
+
finance, workforce satisfaction, etc.) that organizations use to evaluate the performance of their
+
heath care systems. It is important to note that not all of the measures are applicable to every health
+
care organization. Systems should modify the WSMs to reflect their own structures and strategies.
+
IHI hopes to continue learning about the application of the WSMs by working with health care
+
organizations to share best practices and results.
+
The Whole System Measures are based on the following ideas:
+
• In the WSMs, “system” can refer to an integrated health system, a multiple hospital system,
+
a free-standing hospital, or an ambulatory care organization. The WSMs operate at the
+
provider-organization level, whether or not the organization is part of a larger entity.
+
• Health system leaders (and the public) need a small set of measures that reflects a health system’s
+
overall performance on core dimensions of quality and value.
+
• To maintain a systems perspective, a small set of high-level, system-wide measures complements
+
the traditional large set of highly specific measures that reflect the performance of discrete
+
aspects (microlevel performance) of a health system.
+
• The IOM Crossing the Quality Chasm report’s six quality dimensions (i.e., care that is safe,
+
effective, patient-centered, timely, efficient, and equitable) provide a practical framework for
+
organizing the WSMs.
+
• Graphic displays of data over time are the preferred tools to show patterns and trends in each
+
health system’s quality measures. Shewhart control charts can be used with these displays to help
+
interpret the patterns of variation.
+
• Large administrative databases are a useful way to provide health systems with comparative
+
data. However, it may be necessary to enhance data from these databases if the data cannot
+
be collected frequently and in a timely way.
+
• Some organizations will either not wish to or not be able to track all the WSMs, but may still
+
find it helpful to use a subset of measures and then add or modify others as needed.
+
This white paper has six sections that describe the Whole System Measures in detail:
+
• Section One: Overview of the Whole System Measures
+
• Section Two: Implementing the Whole System Measures
+
• Section Three: Setting the “Toyota Specification” for Each Whole System Measure
+
• Section Four: Lessons Learned and Conclusions
+
• Section Five: Case Study of an Organization Using the Whole System Measures
+
• Section Six: Appendices
+
o Appendix A: Detailed Information on the Whole System Measures
+
o Appendix B: Example of One Organization Using the Toyota Specifications
+
o Appendix C: Measurement Experts for Each of the Whole System Measures
+
© 2007 Institute for Healthcare Improvement
+
3 Innovation Series: Whole System Measures
+
© 2007 Institute for Healthcare Improvement
+
Section One: Overview of the Whole System Measures
+
In 2003 a group of approximately 10 people from the United Kingdom, Sweden, and the United
+
States met to discuss the idea of developing a method for measuring the quality of care at the level
+
of a health system. They believed that although many helpful quality measures existed and more
+
were being rapidly created, high-level measures reflecting the overall quality of a health system were
+
largely missing. They also believed that the important work to measure hospital quality—based on
+
an overall mortality measure called the hospital standardized mortality ratio (HSMR)—done by Sir
+
Brian Jarman, MD, Emeritus Professor and head of the Dr Foster Unit at Imperial College in
+
London, could serve as a model for a high-level quality measure.5,6 Together the members of this
+
group, including leaders from IHI, believed they could develop a small set of measures to go beyond
+
Jarman’s HSMR metric.
+
After several months of dialogue and planning, the following health systems began to test the initial
+
 
set of prototype measures the group developed:
 
set of prototype measures the group developed:
Sweden: Jönköping County
+
* Sweden: Jönköping County
United Kingdom: East Lancashire—Blackburn Trust, Bentley Trust
+
* United Kingdom: East Lancashire—Blackburn Trust, Bentley Trust
United States: Pursuing Perfection site7—McLeod Regional Medical Center
+
* United States: Pursuing Perfection site7—McLeod Regional Medical Center
United States: IHI IMPACT network8 organizations—Geisinger Medical Center,
+
* United States: IHI IMPACT network8 organizations—Geisinger Medical Center, St. John’s Mercy, ThedaCare
St. John’s Mercy, ThedaCare
+
 
Based on their experience with the prototype measures and the associated work of collecting and
+
== Method ==
analyzing data, reporting the results, and using this information for evaluation and improvement,
+
 
the group produced Version 1.0 of the Whole System Measures. Version 1.0 contained nine
+
From the fall of 2004 through the summer of 2005, approximately 30 health systems collaborated with IHI to collect data and measure their progress using the WSMs. Lessons learned from their work and progress in the field of health care quality improvement led IHI to create a revised version of the WSMs—adding new measures where there were gaps in the system-level metrics and removing measures that were not helpful. IHI presented the WSMs to senior leaders of organizations in IHI’s IMPACT network as the proposed measurement set for their systems. Moreover, IHI’s Framework for the Leadership of Improvement calls for senior leaders and board members to focus their strategic improvement work on important measures (i.e., the “big dots”) such as mortality, harm, and patient satisfaction that reflect the quality of care delivered. Keeping in mind that the WSMs are meant to be the “big dots” at the system level, the WSMs are limited to a small set of 13 measures that are not disease- or condition-specific. One objective for developing the WSMs was to provide a view of performance that reflects care provided in different sites and across the continuum of care.  
measures that cut across the six IOM quality dimensions and represented both inpatient and
+
 
outpatient care.
+
If the system is performing well at the highest level of aggregation, then it is likely to be performing well at lower levels whose measures roll up into the high-level measures. If the best possible results are not being achieved, then it is necessary to dig deeper into the causal system to identify how and where the processes of care need to be improved.
From the fall of 2004 through the summer of 2005, approximately 30 health systems collaborated
+
 
with IHI to collect data and measure their progress using the WSMs. Lessons learned from their
+
=== Example of Hospital Readmission Percentage ===
work and progress in the field of health care quality improvement led IHI to create a revised version
+
 
of the WSMs—adding new measures where there were gaps in the system-level metrics and
+
An example of a WSM is Hospital Readmission Percentage which is defined as Readmission to the hospital is a measure of both the care received in the hospital and the coordination of care back to the outpatient setting and within the outpatient setting. The Hospital Readmission Percentage is defined as the percentage of patients discharged from the hospital who are readmitted to the hospital within 30 days. Hospital Readmission Percentage = (Number of discharged patients readmitted to the hospital
removing measures that were not helpful. IHI presented the WSMs to senior leaders of organizations
+
within 30 days of their discharge / Number of patients discharged) * 100. Exclusions:
in IHI’s IMPACT network as the proposed measurement set for their systems. Moreover, IHI’s
+
* Planned readmissions
Framework for the Leadership of Improvement calls for senior leaders and board members to focus
+
* False labor patients
their strategic improvement work on important measures (i.e., the “big dots”) such as mortality,
+
 
harm, and patient satisfaction that reflect the quality of care delivered.5,9
+
=== Frequency ===
Keeping in mind that the WSMs are meant to be the “big dots” at the system level, the WSMs are
+
 
limited to a small set of 13 measures that are not disease- or condition-specific. One objective for
+
Monthly [Note: There is a one-month delay in obtaining the required data due to the need to wait for 30 days post-discharge.] Method for Measuring: Each month, use your organization’s financial and/or admission information systems to identify patients who were discharged that month and also had a second admission within 30 days of the initial discharge date.
developing the WSMs was to provide a view of performance that reflects care provided in different
+
Background on the Measure: This is an important measure to indicate if changes to improve patient flow through the system are negatively affecting care. While some readmissions are part of the planned care and are desirable, others may be indications of a quality issue related to a shortened length of stay and premature discharge, inadequate care, or lack of patient adherence to the care regimen following discharge from the hospital.  
sites and across the continuum of care. Table 1 lists the Whole System Measures, the relevant IOM
+
 
quality dimension for each measure, and the setting(s) in which the measure applies.
+
=== The Equity WSM (Stratification of Whole System Measures) ===
Institute for Healthcare Improvement Cambridge, Massachusetts 4
+
 
Table 1. Whole System Measures, IOM Dimensions of Quality, and Care Locations
+
It is difficult to create a primary measure for equity. Equity is measured by stratifying the Whole System Measures, when possible, into subpopulations that differentiate by gender, age, income, or racial groupings, for example. Equity = The difference in outcome for a Whole System Measure stratified by different subpopulations. Frequency: Monthly [Note: If the sample is small and cannot be separated into subpopulations due to lack of adequate representation, monthly data should be aggregated and reviewed quarterly.] Method for Measuring: When possible, each Whole System Measure should be stratified by subpopulation. The goal is to drive the difference in outcomes between subpopulations to zero.
Whole System Measure IOM Outpatient Inpatient
+
 
Dimension Care Care
+
== References ==
of Quality
+
1. Rate of Adverse Events Safe X X
+
2. Incidence of Nonfatal Occupational
+
Injuries and Illnesses Safe X X
+
3. Hospital Standardized Mortality Ratio Effective
+
(HSMR) X
+
4. Unadjusted Raw Mortality Percentage Effective X
+
5. Functional Health Outcomes Score Effective X X
+
6. Hospital Readmission Percentage Effective X X
+
7. Reliability of Core Measures Effective X X
+
8. Patient Satisfaction with Care Score Patient-
+
Centered X X
+
9. Patient Experience Score Patient-
+
Centered X
+
10. Days to Third Next Available Appointment Timely X
+
11. Hospital Days per Decedent During the Efficient
+
Last Six Months of Life X
+
12. Health Care Cost per Capita Efficient X X
+
13. Equity (Stratification of Whole System Measures) Equitable X X
+
Figure 1. View of a Health System Using the Whole System Measures
+
Note that equity is not pictured in the figure. This important quality dimension is measured by stratifying the Whole System
+
Measures, when possible, into subpopulations that differentiate by gender, age, income, or racial groupings, for example.
+
Institute for Healthcare Improvement Cambridge, Massachusetts 6
+
incorporate the roles listed below into their current data support system for collecting and
+
assembling the monthly or quarterly family of measures.It is IHI’s recommendation that, rather than flooding an organization’s
+
senior leaders and board members with countless pages of data tables, the WSMs are properly
+
integrated into the organization’s existing family of measures in a balanced dashboard report.3,12
+
If the system is performing well at the highest level of aggregation, then it is likely to be performing
+
well at lower levels whose measures roll up into the high-level measures. If the best possible results
+
are not being achieved, then it is necessary to dig deeper into the causal system to identify how and
+
where the processes of care need to be improved.
+
Section Four: Lessons Learned and Conclusion
+
Since the initial testing of the Whole System Measures in 2003, IHI has learned much about the need for
+
clarity, continuity, parsimony, and utility when using the Whole System Measures. Since many of the
+
measures are collected within health care organizations, we learned that the difficulty of gathering the
+
data elements is not a planning challenge, but rather an operations issue that requires clarity of roles and
+
responsibilities. Thus, while the role of Data Coordinator for the Whole System Measures is not an independent
+
full-time job, formally assigning this role to a staff member who is part of the data team helps to
+
establish clarity about the measures being collected and the timeline on which they are reviewed.
+
Parsimony is crucial when creating Whole System Measures. Using too many measures results in too
+
little focus. It is helpful to have different levels of the measures—“big dots” and “little dots”—rather
+
than a multiplicity of unrelated measures. The WSMs are the biggest macrolevel “dots” and, therefore,
+
should be the fewest in number and should be reviewed at the highest level of the organization.
+
Finally, by working with different types of health care organizations, IHI has learned the value of
+
utility: some measures are very helpful and others need to be evaluated and replaced. Throughout
+
the testing of the WSMs, IHI removed and added measures to increase the usefulness of the
+
measure set. Similarly, it is also necessary for organizations using the WSMs to periodically review
+
their strategic plan and add or replace measures from the WSMs as their strategy evolves over time.
+
While the WSMs were developed for use at the local level within health care organizations, they
+
also could be used at the national level. IHI has begun to test moving the “big dots” with its national
+
5 Million Lives Campaign,22 using the adverse events and mortality Whole System Measures as
+
success metrics. While this effort is still at an early stage, much is being learned from the experiences
+
of the more than 3,600 hospitals enrolled in the Campaign. Further testing and refinement of the
+
WSMs is necessary to put them into a national context and to create traction for their widespread
+
use. One key challenge for national use of the WSMs is to align them with existing required measures
+
such as the Hospital Quality Measures used by the Centers for Medicare and Medicaid Services
+
and The Joint Commission, and the National Quality Forum Safe Practices. This alignment is needed
+
both to reduce the measurement burden on hospitals and to gain broad-based support from key
+
stakeholders.
+
6. Hospital Readmission Percentage
+
Definition: Readmission to the hospital is a measure of both the care received in the hospital and the
+
coordination of care back to the outpatient setting and within the outpatient setting. The Hospital
+
Readmission Percentage is defined as the percentage of patients discharged from the hospital who are
+
readmitted to the hospital within 30 days.
+
Hospital Readmission Percentage = (Number of discharged patients readmitted to the hospital
+
within 30 days of their discharge / Number of patients discharged) * 100
+
Exclusions:
+
Planned readmissions
+
False labor patients
+
Frequency: Monthly
+
[Note: There is a one-month delay in obtaining the required data due to the need to wait for 30 days post-discharge.]
+
Method for Measuring: Each month, use your organization’s financial and/or admission information
+
systems to identify patients who were discharged that month and also had a second admission within
+
30 days of the initial discharge date.
+
Background on the Measure: This is an important measure to indicate if changes to improve patient
+
flow through the system are negatively affecting care. While some readmissions are part of the
+
planned care and are desirable, others may be indications of a quality issue related to a shortened
+
length of stay and premature discharge, inadequate care, or lack of patient adherence to the care
+
regimen following discharge from the hospital.
+
Example Line Graph: Hospital Readmission Percentage (Within 30 Days of Initial Discharge) in a
+
Multispecialty Health System
+
© 2007 Institute for Healthcare Improvement
+
29 Innovation Series: Whole System Measures
+
Toyota Specification19
+
Data for this Toyota Specification is derived from the 2006 Premier Perspective™ Database.
+
Readmission is defined as patients readmitted within 30 days to the same hospital/health system
+
(link by Medical Record Number). The following patients are excluded from the readmission rate
+
calculation: skilled nursing facility patients; false labor patients (patients with principal or secondary
+
ICD Codes 644.10, 644.13); and same-day readmissions (patients who are discharged and readmitted
+
the same day).
+
7. Reliability of Core Measures
+
Definition: Reliability is defined as failure-free operation over time and is measured as the inverse of
+
the system’s failure rate. Reliability is expressed as a failure rate to demonstrate the order of
+
magnitude. Reliability of 10-1 means one failure per 10 attempts, and 10-2 means five or less failures
+
per 100 attempts.20
+
Reliability = (Number of actions that achieved the intended result / Total number of actions taken) * 100
+
Frequency: Monthly
+
Method for Measuring: Define the processes in your organization for which you will measure reliability.
+
There are many national measure sets that highlight clinical guidelines that should be followed.
+
At a minimum, your organization should monitor and track reliability for the Hospital Quality
+
Measures (also called Core Measures) used by the Centers for Medicare and Medicaid Services
+
(CMS) and The Joint Commission (TJC).27
+
Each month, use your organization’s quality improvement data and additional data collection system
+
to identify patients who were eligible for the indicated interventions or treatment. For each Core
+
Measure, note the number of patients who received all indicated components of care and the
+
number of patients who were eligible to receive care. See the IHI Innovation Series white paper,
+
Improving the Reliability of Health Care, for additional information.20
+
Background on the Measure: The principles of designing reliable systems are routinely used in many
+
industries, such as manufacturing and air travel, to improve safety and compensate for the limits of
+
human ability. Studies suggest that most US health care organizations perform below many other
+
industries and consistently only achieve a 10-1 level of reliability. Systems can be put in place to
+
increase the reliability of key processes and enhance both patient and staff safety.20
+
© 2007 Institute for Healthcare Improvement
+
Institute for Healthcare Improvement Cambridge, Massachusetts 30
+
  
 +
# Martin LA, Nelson EC, Lloyd RC, Nolan TW. Whole System Measures. IHI Innovation Series white paper. Cambridge, Massachusetts: Institute for Healthcare Improvement; 2007. Online information retrieved September 24, 2007. [http://www.ihi.org/IHI/Results/WhitePapers/ImprovingtheReliabilityofHealthCare.htm]
  
13. Equity (Stratification of Whole System Measures)
 
Definition: It is difficult to create a primary measure for equity. Equity is measured by stratifying the
 
Whole System Measures, when possible, into subpopulations that differentiate by gender, age,
 
income, or racial groupings, for example.
 
Equity = The difference in outcome for a Whole System Measure stratified by
 
different subpopulations
 
Frequency: Monthly
 
[Note: If the sample is small and cannot be separated into subpopulations due to lack of adequate representation, monthly data
 
should be aggregated and reviewed quarterly.]
 
Method for Measuring: When possible, each Whole System Measure should be stratified by subpopulation.
 
The goal is to drive the difference in outcomes between subpopulations to zero.
 
© 2007 Institute for Healthcare Improvement
 
Medicare Reimbursements per Enrollee by Hospital Referral Region (HRR)11
 
Toyota Specification
 
Health Care Cost per Capita (in US Dollars) for Selected Countries (2005)17
 
39 Innovation Series: Whole System Measures
 
Toyota Specification
 
The two graphics below depict the Functional Health Outcomes Score Whole System Measure
 
stratified by two different categories (annual household income and race/ethnicity). The goal is to
 
have all subpopulations achieve the outcome of the Toyota Specification (which is 5.1 percent).
 
[Note: The self-reported health status data for this specification is derived from the Centers for Disease Control and Prevention
 
report, Health-Related Quality of Life Surveillance—United States, 1993-2002.16]
 
Would you say that in general your health is excellent, very good, fair or poor?
 
© 2007 Institute for Healthcare Improvement
 
Would you say that in general your health is excellent, very good, fair or poor?
 
Institute for Healthcare Improvement Cambridge, Massachusetts 40
 
© 2007 Institute for Healthcare ImprovementReferences
 
1 Donabedian A. Explorations in Quality Assessment and Monitoring, Volume II: The Criteria and
 
Standards of Quality. Ann Arbor, MI: Health Administration Press; 1982.
 
2 Donabedian A. Explorations in Quality Assessment and Monitoring, Volume I: The Definition of
 
Quality and Approaches to Its Assessment and Monitoring. Ann Arbor, MI: Health Administration
 
Press; 1980.
 
3 Kaplan R, Norton D. Balanced Scorecard: Translating Strategy into Action. Cambridge, MA;
 
HBS Press; 1996.
 
4 Institute of Medicine. Committee on Quality of Health Care in America. Crossing the
 
Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies
 
Press; 2001.
 
5 Jarman B, Nolan T, Resar R. Move Your Dot™: Measuring, Evaluating, and Reducing Hospital
 
Mortality Rates. IHI Innovation Series white paper. Boston: Institute for Healthcare
 
Improvement; 2003. Online information retrieved September 24, 2007.
 
http://www.ihi.org/IHI/Results/WhitePapers/MoveYourDotMeasuringEvaluatingandReducingHos
 
pitalMortalityRates.htm.
 
6 Jarman B, Gault S, Alves B, et al. Explaining the differences in English hospital death rates using
 
routinely collected data. British Medical Journal. 1999;318(7197):1515-1520.
 
7 Pursuing Perfection, a major initiative of the Robert Wood Johnson Foundation for which IHI
 
is the National Program Office, is a multi-year program designed to create models of excellence
 
at a select number of provider organizations that are redesigning all of their major care processes.
 
Online information retrieved September 24, 2007.
 
http://www.ihi.org/IHI/Programs/StrategicInitiatives/PursuingPerfection.htm.
 
8 The IMPACT network is IHI’s “association for change,” an intensive program in which member
 
organizations work together and with expert faculty to make dramatic and measurable
 
improvements at the system level. Online information retrieved September 24, 2007.
 
http://www.ihi.org/IHI/Programs/IMPACTNetwork/.
 
9 Provost L, Miller D, Reinertsen J. A Framework for Leadership of Improvement. Cambridge,
 
MA: Institute for Healthcare Improvement; February 2006. Online information retrieved
 
September 24, 2007.
 
http://www.ihi.org/IHI/Topics/LeadingSystemImprovement/Leadership/EmergingContent/
 
AFrameworkforLeadershipofImprovement.htm.
 
10 Nelson EC, Batalden PB, Ryer JC. Clinical Improvement Action Guide. Chicago: Joint
 
Commission on Accreditation of Healthcare Organizations; 1998.
 
43 Innovation Series: Whole System Measures
 
© 2007 Institute for Healthcare Improvement
 
11 Wennberg J. The Dartmouth Atlas of Health Care. Online information retrieved June 12, 2007.
 
http://www.dartmouthatlas.org/index.shtm.
 
12 Provost L, Leddick S. How to take multiple measures to get a complete picture of organizational
 
performance. National Productivity Review. 1993;12(4):477-490.
 
13 Nelson EC, Batalden PB, Godfrey MM. Quality By Design: A Clinical Microsystems Approach.
 
San Francisco: Jossey-Bass Publishers; 2007.
 
14 Berwick D, Kabcenell A, Nolan T. No Toyota yet, but a start. A cadre of providers seeks to
 
transform an inefficient industry—before it’s too late. Modern Healthcare. 2005;Jan 31:18-19.
 
15 Email communication with John Wasson, MD. March 29, 2007.
 
16 Zahran HS, Kobau R, Moriarty D, Zack M, Holt J, Donehoo R. Health-related quality
 
of life surveillance—United States, 1993-2002. MMWR: Surveillance Summaries.
 
2005 Oct 28;54(SS04):1-35.
 
17 Organisation for Economic Co-operation and Development. OECD Health Data. Online
 
information retrieved May 24, 2007.
 
http://www.oecd.org/document/16/0,2340,en_2649_34631_2085200_1_1_1_1,00.html.
 
18 Bureau of Labor Statistics. Table 1. Incidence rates of nonfatal occupational injuries and illnesses
 
by industry and case types, 2005. Washington, DC: US Department of Labor; 2005.
 
19 Belk K. Premier Perspective™ Database Readmission Data (Calendar Year 2006). Premier, Inc.; 2007.
 
20 Nolan T, Resar R, Haraden C, Griffin F. Improving the Reliability of Health Care. IHI Innovation
 
Series white paper. Boston: Institute for Healthcare Improvement; 2004. Online information
 
retrieved September 24, 2007.
 
http://www.ihi.org/IHI/Results/WhitePapers/ImprovingtheReliabilityofHealthCare.htm.
 
21 Nolan TW. Execution of Strategic Improvement Initiatives to Produce System-Level Results. IHI
 
Innovation Series white paper. Cambridge, MA: Institute for Healthcare Improvement; 2007.
 
Online information retrieved September 24, 2007.
 
http://www.ihi.org/IHI/Results/WhitePapers/ExecutionofStrategicImprovementInitiatives
 
WhitePaper.htm.
 
22 The 5 Million Lives Campaign is a voluntary initiative to protect patients from five million
 
incidents of medical harm over a two-year period (December 2006 to December 2008). Online
 
information retrieved September 24, 2007. http://www.ihi.org/IHI/Programs/Campaign/.
 
23 Griffin F, Resar R. IHI Global Trigger Tool for Measuring Adverse Events. IHI Innovation Series
 
white paper. Cambridge, MA: Institute for Healthcare Improvement; 2007. Online information
 
retrieved September 24, 2007.
 
http://www.ihi.org/IHI/Results/WhitePapers/IHIGlobalTriggerToolWhitePaper.htm.
 
Institute for Healthcare Improvement Cambridge, Massachusetts 44
 
© 2007 Institute for Healthcare Improvement
 
24 US Department of Labor, Occupational Safety and Health Administration. Forms for Recording
 
Work-Related Injuries and Illnesses. Online information retrieved September 24, 2007.
 
http://www.osha.gov/recordkeeping/new-osha300form1-1-04.pdf.
 
25 Alcoa Health and Safety Overview. Online information retrieved September 24, 2007.
 
http://www.alcoa.com/global/en/about_alcoa/sustainability/health_overview.asp.
 
26 Institute for Healthcare Improvement. 5 Million Lives Campaign. Measurement Information
 
Form: Acute Care Inpatient Mortality. Online information retrieved September 24, 2007.
 
http://www.ihi.org/NR/rdonlyres/64543320-F284-4EC7-86C7-
 
B1B75D24F042/0/MIFHospitalAcuteCareInpatientMortalityRate.pdf.
 
27 The Joint Commission. Facts About ORYX® for Hospitals, Core Measures and Hospital Quality
 
Measures. Online information retrieved September 24, 2007.
 
http://www.jointcommission.org/NewsRoom/PressKits/AnnualReport/ar_facts_oryx.htm.
 
28 Agency for Healthcare Research and Quality. CAHPS Database Interactive Chartbook. Online
 
information retrieved September 24, 2007. https://www.cahps.ahrq.gov.
 
29 Wasson J. How’s Your Health. Online information retrieved September 24, 2007.
 
http://www.howsyourhealth.org.
 
30 Murray M, Tantau C. Must patients wait? Joint Commission Journal on Quality Improvement.
 
1998 Aug;24(8):423-425.
 
31 Wennberg J. “Understanding Practice Patterns: A Focus on What the Quality Movement Can Do
 
to Reduce Unwarranted Variation.” Paper presented at the Institute for Healthcare Improvement
 
National Forum on Quality Improvement in Health Care, Orlando, FL; December 2005.
 
32 Centers for Medicare and Medicaid Services, Office of the Actuary National Health Statistics
 
Group. National Health Expenditures. Online information retrieved September 24, 2007.
 
http://www.cms.hhs.gov/NationalHealthExpendData/downloads/tables.pdf.
 
20 University Road, 7th Floor
 
Cambridge, MA 02138
 
(617) 301-4800
 
www.ihi.org
 
 
Submitted by Dr. Terry Olson, M.D.
 
Submitted by Dr. Terry Olson, M.D.
 
[[Category:BMI512-W-10]]
 
[[Category:BMI512-W-10]]

Latest revision as of 15:14, 18 November 2011

The Institute for Healthcare Improvement (IHI) Whole System Measures (WSM) approach to quality improvement and coordination of quality activities provides a framework upon which future informational systems should be structured.

Far too often quality improvement activities are structured with a very limited scope and produce only a marginal effect on overall quality or outcomes. QI activities are rarely coordinated and complimentary. More often QI activities are conducted within a single, isolated department or division of a healthcare organization. In many cases the reduced scope was mandated by the information systems used. Because different departments or divisions used different IT systems, the QI project was limited to the relevant IT system.

New opportunities emerge as organizations implement enterprise wide IT systems or develop more interactive best of breed system relationships. QI activities can now be constructed across departments or divisions. Organizations now wonder how they take advantage of these new opportunities with the little resources they have available. The IHI WSM provides a useful roadmap for organizations to move forward and maximize the use of their QI resources.

Healthcare improvement

IHI used the Institute of Medicine’s 6 aims for healthcare improvement – safe, effective, efficient, timely, patient centered, and equitable – to build their 13 WSM. The 13 WSM are referenced as Big Dots up to which smaller dots (focused, limited scope QI projects). In this manner previous and ongoing QI projects can be coordinated and incorporated into an organization wide program.

The 13 measures and associated IOM Aim are:

  1. Rate of Adverse Events :# Safety
  2. Incidence of Nonfatal Occupational Injuries or Illnesses# Safety
  3. Hospital standardized Mortality Ratio (HSMR) : Effective
  4. Unadjusted Raw Mortality Percentage : Effective
  5. Functional Health Outcomes Scores : Effective
  6. Hospital Readmission Percentage : Effective
  7. Reliability of Core Measures :Effective
  8. Patient Satisfaction with Care Score : Patient-Centered
  9. Patient Experience Score :Patient-Centered
  10. Days to Third Next Available Appointment : Timely
  11. Hospital Days per Decedent During the Last 6 Months of Life# Efficient
  12. Health Care Cost per Capita :Efficient
  13. Equity (stratification of WSM) :Equitable

As health IT matures the WSM can guide the application of systems and projects to produce enterprise wide improvement. All 13 WSM are applicable to the outpatient environment while 10 are applicable to inpatient environment.

The IHI Executive Summar reads, The aim of this white paper is to describe and promote the use of a system of metrics, called the Whole System Measures, to measure the overall quality of a health system and to align improvement work across a hospital, group practice, or large health care system. The Institute for Healthcare Improvement and colleagues developed the Whole System Measures, a balanced set of system-level measures, to supply health care leaders and other stakeholders with data that enable them to evaluate their health systems’ overall performance on core dimensions of quality and value, and that also serve as inputs to strategic quality improvement planning. Properly constructed, the Whole System Measures should complement existing measures that organizations use to evaluate the performance of their heath care systems. The Whole System Measures, because they are intended to focus on important system-level measures, are limited to a small set of 13 measures that are not disease or condition-specific. One objective for developing the Whole System Measures was to also provide a view of performance that reflects care provided in different sites—both inpatient and outpatient—and across the continuum of care.

There was a time when it was mainly the providers of care who were concerned about health care quality data; this is no longer the case. Today, not only are the providers of care keenly focused on the processes and outcomes of health care delivery, but the consumers of health care—as well as managers, boards, purchasers, and policy makers—are also becoming increasingly interested in being shown that health care services are safe, effective, patient-centered, timely, efficient, and equitable. Many of the questions that drive this growing interest in health care quality measurement can only be answered with data. The WSM provide the following:

  • A useful conceptual framework for organizing measures of health care quality; and
  • A specific set of quality metrics that can contribute to a health care organization’s family of measures, balanced scorecard, or dashboard of strategic performance measures. A central premise of IHI’s work on the WSMs is that any family of measures should reflect a balance among structures, processes, and outcomes. A balanced set of system-level measures is

needed to provide leaders and other stakeholders with data that:

  • Show performance of their health care system over time;
  • Allow the organization to see how it is performing relative to its strategic plans for improvement;
  • Allow comparisons to other similar organizations; and
  • Serve as inputs to strategic quality improvement planning.

In 2003 a group of approximately 10 people from the United Kingdom, Sweden, and the United States met to discuss the idea of developing a method for measuring the quality of care at the level of a health system. They believed that although many helpful quality measures existed and more were being rapidly created, high-level measures reflecting the overall quality of a health system were largely missing. They also believed that the important work to measure hospital quality—based on an overall mortality measure called the hospital standardized mortality ratio (HSMR)—done by Sir Brian Jarman, MD, Emeritus Professor and head of the Dr Foster Unit at Imperial College in London, could serve as a model for a high-level quality measure. Together the members of this group, including leaders from IHI, believed they could develop a small set of measures to go beyond Jarman’s HSMR metric. After several months of dialogue and planning, the following health systems began to test the initial set of prototype measures the group developed:

  • Sweden: Jönköping County
  • United Kingdom: East Lancashire—Blackburn Trust, Bentley Trust
  • United States: Pursuing Perfection site7—McLeod Regional Medical Center
  • United States: IHI IMPACT network8 organizations—Geisinger Medical Center, St. John’s Mercy, ThedaCare

Method

From the fall of 2004 through the summer of 2005, approximately 30 health systems collaborated with IHI to collect data and measure their progress using the WSMs. Lessons learned from their work and progress in the field of health care quality improvement led IHI to create a revised version of the WSMs—adding new measures where there were gaps in the system-level metrics and removing measures that were not helpful. IHI presented the WSMs to senior leaders of organizations in IHI’s IMPACT network as the proposed measurement set for their systems. Moreover, IHI’s Framework for the Leadership of Improvement calls for senior leaders and board members to focus their strategic improvement work on important measures (i.e., the “big dots”) such as mortality, harm, and patient satisfaction that reflect the quality of care delivered. Keeping in mind that the WSMs are meant to be the “big dots” at the system level, the WSMs are limited to a small set of 13 measures that are not disease- or condition-specific. One objective for developing the WSMs was to provide a view of performance that reflects care provided in different sites and across the continuum of care.

If the system is performing well at the highest level of aggregation, then it is likely to be performing well at lower levels whose measures roll up into the high-level measures. If the best possible results are not being achieved, then it is necessary to dig deeper into the causal system to identify how and where the processes of care need to be improved.

Example of Hospital Readmission Percentage

An example of a WSM is Hospital Readmission Percentage which is defined as Readmission to the hospital is a measure of both the care received in the hospital and the coordination of care back to the outpatient setting and within the outpatient setting. The Hospital Readmission Percentage is defined as the percentage of patients discharged from the hospital who are readmitted to the hospital within 30 days. Hospital Readmission Percentage = (Number of discharged patients readmitted to the hospital within 30 days of their discharge / Number of patients discharged) * 100. Exclusions:

  • Planned readmissions
  • False labor patients

Frequency

Monthly [Note: There is a one-month delay in obtaining the required data due to the need to wait for 30 days post-discharge.] Method for Measuring: Each month, use your organization’s financial and/or admission information systems to identify patients who were discharged that month and also had a second admission within 30 days of the initial discharge date. Background on the Measure: This is an important measure to indicate if changes to improve patient flow through the system are negatively affecting care. While some readmissions are part of the planned care and are desirable, others may be indications of a quality issue related to a shortened length of stay and premature discharge, inadequate care, or lack of patient adherence to the care regimen following discharge from the hospital.

The Equity WSM (Stratification of Whole System Measures)

It is difficult to create a primary measure for equity. Equity is measured by stratifying the Whole System Measures, when possible, into subpopulations that differentiate by gender, age, income, or racial groupings, for example. Equity = The difference in outcome for a Whole System Measure stratified by different subpopulations. Frequency: Monthly [Note: If the sample is small and cannot be separated into subpopulations due to lack of adequate representation, monthly data should be aggregated and reviewed quarterly.] Method for Measuring: When possible, each Whole System Measure should be stratified by subpopulation. The goal is to drive the difference in outcomes between subpopulations to zero.

References

  1. Martin LA, Nelson EC, Lloyd RC, Nolan TW. Whole System Measures. IHI Innovation Series white paper. Cambridge, Massachusetts: Institute for Healthcare Improvement; 2007. Online information retrieved September 24, 2007. [1]

Submitted by Dr. Terry Olson, M.D.