see attached
deidraso
ha425_ch3_pp.pptxChapter 3
Measurement, Variation and CQI Tools
Contents
Introduction
Learning from measurement
The role of variation in quality improvement
Quality improvement tools
Recent trends in CQI tools
Six Sigma
Conclusion
Introduction
Measurement is a central element of CQI
Health care institutions are full of data, plus ‘factoids’, opinions, and anecdotes masquerading as data
Analytical approaches require the use of data to evaluate current contexts, analyze and improve processes, and track progress
Introduction
The quality evolution from industry to health care has included the transfer and adoption of industrial statistical tools to measure quality improvement
These tools come from biostatistics, economics, epidemiology, and health services research, but should be considered as an integrated portfolio
Learning from Measurement
The primary purpose of measurement in any quality improvement initiative is to make improvements
Risk: the application of statistical methods and a focus on results without the necessary step of thinking critically and understanding what our data tell us about the system we are trying to improve
Learning from Measurement
“Measurement is only a handmaiden to improvement but improvement cannot act without it. We speak here not of measurement for the purpose of judgment (for deciding whether or not to buy, accept or reject) but for the purpose of learning.”
(Berwick,1996 p.621)
THE ROLE OF VARIATION IN QUALITY IMPROVEMENT
What is variation?
Nature of process variation
Measurement and statistical analysis
Process capability
Interpreting process performance
Process requirements
The Role of Variation in CQI
The starting point for any QI is understanding the type and causes of system variation (Deming 1993; Nolan and Provost 1990)
Statistical control (or statistical process control) of stable or “in control” processes is the basis of CQI activities (Shewhart, 1931)
If a process exhibited variation, then the cause of that variation had to be discovered and removed
The Role of Variation in CQI
Determining variation and analyzing its causes in order to remove them is one primary function of TQM and CQI
Deming’s notion of profound knowledge relates to variation and how it interacts with other elements to lead to system improvements (1993).
In recent years the business concepts related to understanding variation have also been extended specifically to health care (Nelson, Splaine, Batalden and Plume 1998; Carey and Lloyd 2001).
Discussion Question
From your perspective:
What is the role of variation in quality improvement?
Why are measurement and statistical analysis are vital to quality improvement efforts?
What is Variation?
Variation is the extent to which a process differs from the norm
It is related to the statistical concepts of variance and standard deviation
Variation is like a band of output around the central measure of a process
Average time
to
X-ray
takes
10 mins
Variation
X-ray
takes
15 minutes
Variation
X-ray
takes
8 minutes
What is Variation?
The concept of variation in health care may be viewed from several different perspectives
From the national perspective variation highlights health care quality issues relative to access, medical errors, patient outcomes, and resource allocation
At the organizational management perspective variation provides insights on the links between variation and organization effectiveness and results
From the individual perspective may be considered from practitioner, employee, and customer points of view
Nature of Process Variation
There are two general categories of variation, first described by Deming (1986), special and common.
Special causes of process variation: unnecessary variation associated with specific material(s), machine(s), or individual(s)
Common cause variation is the inherent variance in the process that is a result of how the process is performed
It is also referred to as systemic or internal variation
Can be addressed by those working directly with the process
Nature of Process Variation
2. Common causes of variation: those associated with aspects of the system itself such as design, training, materials, machines, or working conditions.
Special cause (or externally caused) variations can be attributed to a particular source
Special cause variation may be traced to the source eliminated but common cause variation can only be reduced by improving the underlying process or system
Responsibility of management to correct, as management is responsible for correcting and preventing system problems
MEASUREMENT AND STATISTICAL ANALYSIS
Process Capability
Interpreting Process Performance
Process Requirements
Process Capability
Process capability studies are to understand the expected output of a process, or the behavior of the process
This involves plotting outputs from the process on a histogram
The aim is to answer the question “Is the process inherently predictable or dependable?”
The next two slides show just such histograms
The first plots the turnaround time for 223 STAT blood tests during a 23-hour period at one large hospital
The second shows a histogram of time spent in the dentist’s waiting room before being led to the dental chair for the exam to begin.
Figure 3.1: Process Capability
Laboratory Test Turnaround Time
Figure 3.2: Process Capability
Waiting Time at the Dentist
Process Capability
The following aspects of a process capability histogram provide valuable information about how the process is performing
The shape of the curve formed by a histogram (normal or non-normal)
Suggests which type of tools should be used for further analysis
A measure of central tendency (mean, median, or mode)
Provides information about the average level of performance
The standard deviation
Shows the range of performance that may be expected.
Process Capability
By plotting the variables over time, patterns or trends in data emerge which can signal:
a problem with the process
that it is time to identify the source of the problem
to prompt action to resolve the problem, and to monitor the impact of the solution.
Interpreting Process Performance
Variation exists in every process
The key is to determine if the average level of performance and amount of common cause variation is acceptable
Acceptable variation depends on understanding the expectations or requirements for the process
Context (turnaround time (TAT) required by an ED department vs primary care)
Specific requirements of the task (e.g. differences in time taken to apply different tests)
Customer requirements
Technical requirements
Process Requirements
Process requirements are the criteria from which the effectiveness of a process is evaluated
They function both as inputs to designing a process and outputs from executing a process
These requirements may be considered from three perspectives:
the customer
other stakeholders
market in general
The next slide illustrates the sequence of questions that should be asked in order to interpret process performance.
Figure 3.3: Process Requirements
Process Requirement Determination Process: Sequence of Questions
Process Requirements
Who are health service customers?
A customer is defined as anyone who has expectations regarding a process operation or outputs (for health care services this might be the patient, while for the community, it is the public health agency)
Internal customers are those within the organizations and are sometimes thought of as those departments or co-workers ‘downstream’ from the process (e.g. patient care units as customers of radiology departments)
Payers may be considered as external customers, that is, those outside the provider organization
Stakeholders are groups or individuals with an interest in or affected by the work health services do (for example regulatory bodies and professional associations)
Process Requirements
What do customers require of your services?
What do patients require? Access? Competent, courteous providers?
What do payers require? A certain level of clinical results delivered in a cost-effective manner?
What do regulatory bodies require? Compliance?
What do markets require? Aculturally diverse approach to delivering services?
Process Requirements
Requirements:
Are vital to determining how services should be specified
Shape how the processes comprising the services are designed and improved
Provide the basis for selecting variables or attributes that will measure the process performance
Are the measure against outputs of processes are evaluated, in order to determine if the process performance is acceptable
Are dynamic and change over time, which is why feedback is important
Process Requirements
The following slide illustrates how one health services organization makes operational the link between customer requirements, process design, and measurement. This includes:
The requirements from important stakeholder groups (i.e., regulatory, accreditation, etc.)
Key organizational processes that address the requirements of these groups are identified
Attributes or variables that the organization measures to understand the degree to which their processes are meeting stakeholder requirements are listed
The related performance goals are identified
If the process capability is not aligned with organizational goals as derived from the stakeholder requirements, then the process must be improved.
Table 3.1: Process Requirements
Links Between Customer Requirements, Process Design, and Measurement
Process Requirements
The next slide illustrates the core processes for each phase of the continuum of care
The patients’ interface with this organization follows the following path:
Admission
Care delivery-treatment
Discharge
Assessment
Table 3.2: Process Requirements
Links Between Process Stages, Requirements, and Measures
Figure 3.4: Process Requirements
Process Performance v. Process limits: (a) A process that does not have difficulty maintaining quality will have normally distributed observations over time. (b) A process that has difficulty maintaining quality may still have normally distributed observations over time but may have control limits outside the natural bounds of the process and a mean that is not at the center of the normal curve. The shaded areas in this diagram represent the areas out of specification.
Process Requirements
There are different types of process capabilities
Segment A in the previous slide shows the result of a process that is highly unpredictable
Segment B shows the QA approach of looking for outliers
Segment 3 shows the CQI or process improvement approach
In the process improvement not only is the average level of performance improved, but the width of the band has narrowed
This shows a predictable process that will deliver more consistent results
The graphs are process performance charts, also known as a statistical process control charts
A process performance chart is the most effective way to measure, document, analyze, and understand the capability of a process
Process Requirements
The graphs in Figure 3–4 are examples of process performance charts, also known as a statistical process control charts. A process performance chart is the most effective way to measure, document, analyze, and understand the capability of a process.
QUALITY IMPROVEMENT TOOLS
Process Flow Chart
Cause-and-Effect Diagram
Histogram and Pareto Diagram
Regression Analysis
Run Charts and Process Performance Charts
Quality Improvement Tools
A systematic, fact-based approach is required to provide permanent solutions to root causes of problems in health care
Different tools, techniques, and methods may be used to accomplish the purpose of each phase of the PDSA cycle
Data and analytical tools may be used throughout the entire PDSA cycle
Berwick (1996) notes that it is critical at the studying stage of a PDSA cycle to take the time to reflect and learn about the impact of improvements that have already been made
This should include evaluation of whether these changes have actually been improvements, and then decide on what further improvements to make.
Quality Improvement Tools
Activity network diagrams;
Affinity diagrams;
Brainstorming;
Cause & effect (fishbone) diagrams;
Check sheets;
Concentration diagrams;
Control charts;
Failure mode and effects analysis (FMEA);
Flowcharts (process, deployment, top-down, opportunity);
Force field analysis;
Frequency plots;
Histograms;
Interrelationship digraphs (ID);
Matrix diagrams;
Pareto charts;
Prioritization matrices;
Process capability charts;
Radar charts;
Run charts;
Scatter diagrams;
Suppliers, process steps, inputs, outputs, customers (SPIOC) diagrams;
Time plots;
Tree diagrams;
Workflow diagrams.
Process Flow Chart
Flowcharts:
Are also known as process flow diagrams, flowcharts pictorial representations of how a process works
They define, describe, and communicate clinical, administrative, and operational processes
They trace the steps that the “object” (specimen, piece of paper, patient) of a process goes through from start to finish
Often used to describe the sequence of actions that must be carried out in order to complete a particular task
Process Flow Chart
Flow diagrams are constructed by:
Defining the basic stages of a process
Breaking each stage of the process down into specific steps needed to complete the process
Following the object through the process a number of times to verify the process by observation
Reviewing the process to clarify the process and include any steps that might be missing
Figure 3.5: Process Flow Chart
Flowchart Symbols: Arrows are used to connect the symbols indicating sequencing and interrelationship.
Figure 3.7: Process Flow Chart
Flowchart of Medication Administration Source
Reprinted with permission by VHA and First Consulting Group from the VHA 2002 Research series publication, Surveillance for Adverse Drug Events: History, Methods and Current Issues by Peter Kilbridge, M.D. and David Classen, M.D., First Consulting Group.
Process Flow Chart
Once an accurate representation of the current process has been achieved, the following questions will be asked:
How effective is the process in meeting customer requirements?
Are there performance gaps or perceived opportunities for improvement?
Have the relevant stages of the process been represented? Are “owners” of each stage represented on the team? If not, what needs to be done to gather their feedback and ideas?
What are the inputs required for the process and where do they come from? Are the inputs constraining the process or not? Which ones?
Are there equipment or regulatory constraints forcing this approach?
Is this the right problem-process to be working on? To continue working on?
Cause-and-Effect Diagram
Cause-and-effect diagrams are also known as Ishikawa or fishbone diagrams because the shape resembles the skeleton of a fish
They are most useful in identifying variation once the process has already been described and document
They are a schematic means of relating the causes of variation to the effect of variation on the process
They help to organize the contributing causes to a problem in order to prioritize, select, and improve the source of the problem
Figure 3.8: Cause-and-Effect Diagram
Multilayered Process of Developing a Fishbone Chart
Cause-and-Effect Diagram
Step 1: the identified performance gap or problem is put on the right and an arrow is drawn leading to it that represents the overall causation
Step 2: spines are drawn from the arrow to represent main classifications or categories of causes, such as labor, materials, and equipment
Step 3: each major spine is labeled with specific causes, which also may occur at multiple levels
It may be necessary to stratify cause-and-effect diagrams further to achieve finer gradations of error causes and help identify corrective action
Figure 3-9: Cause-and-Effect Diagram
Cause-and-Effect Diagram of Medication Adverse Event: Root Causes of Medication Errors
Histogram and Pareto Diagram
Once the cause-and-effect diagram is generated, data is collected to quantify how often the different causes occur
The simplest way to display this is a histogram (a vertical bar chart representing the frequency distribution of set of data), which visualizes the nature of underlying statistical distribution.
The bars are arrayed on the X-axis representing equal or adjacent data intervals or discrete events
The length of the bar against the Y-axis shows the number of observations falling on that interval or event classification
Successive histograms can be used to indicate whether or not there has been a change in the variability of a process.
Figure 3.10: Histogram and Pareto Diagram
Histogram of Linen and Discard Causes
Histogram and Pareto Diagram
A Pareto diagram is a vertical bar chart with the bars arranged from the longest first on the left and moving successively toward the shortest
The vertical bars give an visual indication of the relative frequency of the contributing causes of the problem with each bar representing one cause
Concentrating on the vital few causes are likely to constitute the areas of highest payback
Concentrating on the high-volume, useful many causes should have the largest potential for reducing process variation
Figure 3.11: Histogram and Pareto Diagram
Pareto Chart: Root Causes of Adverse Drug Events
Regression Analysis
Regression analysis tests the hypothesis that one event is temporally or causally related to another by some form of correlational modeling
Negative findings about cause-and-effect relationships are not a bad outcome in CQI
They reduce the complexity of the set of cause-and-effect hypotheses to be studied by reducing the number of possible causes
Regression analysis is used to test what may turn out to be erroneous impressions about the causes of poor performance
It can also provide a way of looking for unknown or underrated associations and to verify and support any improvement programs and processes
Run Charts and Process Performance Charts
Performance data need to be monitored on an ongoing basis to identify:
What the temporal behavior of the process is;
Establish the time of process performance changes so that they can be linked to the time of other possibly related events
Figure 3–13 shows a series of run charts and some diagnostic interpretations of those data. Since the effects of health care errors tend to be asymmetrical, it is best to look at one-sided rules of thumb for process control. A process is considered under control if most of the observations are near the centerline, if there are few points near the extreme values (above the mean plus or minus three standard deviations), and there are no runs (more than eight consecutive observations to one side of the mean). Run charts are very easy to generate using spreadsheet software.
Run Charts and Process Performance Charts
Run charts are frequently used in the quality improvement process to answer the questions, “How are we doing?” and “Are we doing better since implementing the improvement intervention?”
Performance data needs to be monitored on an ongoing basis to:
Identify what the temporal behavior of the process is (does it change over time);
Establish the time of process performance changes so that they can be linked to the time of other possibly related events
Figure 3.13: Run Charts and Process Performance Charts
Three Examples of Run Charts. In (a), the data is considered to be under control—the points are apparently randomly distributed on either side of the mean, and do not go outside of the control limits. In (b), there are extreme values (outside the control limits), and thus the process is not in control. Another thing to be cautious of is too many observations on one side of the mean. In (c), there are too many values in a row (>8) below the mean.
Run Charts and Process Performance Charts
A process is considered under control if:
Most of the observations are near the centerline
There are few points near the extreme values (above the mean plus or minus three standard deviations)
There are no runs (more than eight consecutive observations to one side of the mean)
Run Charts and Process Performance Charts
There are two types of measures that can be used to develop run charts and control charts: attributes or variables
Attribute data arise from (1) classification of items, such as products or services, into categories; from (2) counts of the number of items or the proportion in a given category; and from (3) counts of the number of occurrences per unit . . . Important attributes (are): fraction defective, number of defects, number of defects per unit (Gitlow et al. 1989, pp. 78, 79, 144).
Run Charts and Process Performance Charts
There are two types of measures that can be used to develop run charts and control charts: attributes or variables
Attribute data arises from (1) classification of items, such as products or services, into categories; from (2) counts of the number of items or the proportion in a given category; and from (3) counts of the number of occurrences per unit . . . Important attributes (are): fraction defective, number of defects, number of defects per unit (Gitlow et al. 1989, pp. 78, 79, 144).
Run Charts and Process Performance Charts
Variables are either measured directly or based on direct measures only and do not result from a classification scheme
Charts often present the:
variable mean (X-bar)
process range (R), and/or
standard deviation (s) for a specific process parameter
A run chart that includes notations indicating the control limits of plus or minus three standard deviations may be referred to as a control chart or a process performance chart.
The control limits are referred to as the:
upper control limit (UCL), three standard deviations above the mean
lower control limit (LCL), three standard deviations below the mean
Run Charts and Process Performance Charts
Use of a control chart depends on the process being free of special causes of variation at the time the control limits were set
Processes have to be reviewed to see:
whether special causes were again creeping in;
whether the underlying processes has changed
The most common form of control chart used is the X-bar chart
A plot of the sample mean (X-bar) of the observations
The sample size per observation for health care is usually 1
This sample size assumes that there is no sampling error and that all observations are accurate
Run Charts and Process Performance Charts
Control charts often use simple statistics mean, standard deviation, and range
For variable data (such as time or distance) X-bar charts and R-charts are used
R-chart is created in the same manner as the X-chart
Both plot the UCL, the center line, the LCL, and the range of each group of observations
For attribute data (such as mortality rates) p-charts are used
P-charts shows the proportion of cases in which a given defect or set of defects occurs
3.4: Run Charts and Process Performance Charts
Acute Myocardial Infarction: Door-to-Needle Time
Figure 3.14: Run Charts and Process Performance Charts
X-Bar Chart—Door to Needle Time
© S.P. Johnson, F. Alemi et al: “Rapid Improvement Teams.” Joint Commission Journal on Quality Improvement, Vol. 24, No. 3, pp. 119-129, 1998. Reprinted with permission.
Table 3.5: Run Charts and Process Performance Charts
Factors to Determine Control Chart Limits
Recent Trends in CQI Tools
Measuring Medical Errors
Checklists
Measuring Medical Errors
Global Trigger Tool for Measuring Adverse Events was developed by the Institute for Health Care Improvement
It is used for tracking rates of adverse events and medical errors in hospital records over time (Griffin and Resar, 2007)
“Triggers” are defined as clues in patient records that indicate adverse events or medically induced harms
The trigger tool was found to have very high specificity, reliability and sensitivity (Office of the Inspector General, 2010; Sharek, Parry, Goldmann et al, 2010, Landrigan, Parry, Bones et al, 2010)
Checklists
Checklists are not statistical tools nor completely new tool in quality improvement, having been used extensively in aviation
They are considered to be part of an accelerated evolution into medical care which has had a greater focus on safety issues in the early part of the 21st century (Gawande 2009; Pronovost 2009).
Checklists have been found to be an effective safety tool in surgery (Haynes et al, 2009; deVries et. al 2010) and other medical specialties (Gawande 2009; Pronovost 2006)
Six Sigma
Six sigma (sigma being the Greek symbol used in statistics to measure variation) utilizes statistical methods to identify and remove errors and minimize variability in processes
It is a set of practices or strategies which start with process mapping to identify elements critical to quality and then focuses on the changes to these via the DMAIC model (Duffy et al. 2009)
P
Process mapping
C
Control
D
M
A
I
Define
Measure
Analyze
Improve
Six Sigma
Lean methodology is often used with six sigma
It is defined as a systematic approach to identifying and eliminating waste (defined as any non-valued tasks)
Lean six sigma is used in a wide range of health care applications, and provides a synergistic methodology for analyzing, and reducing or eliminating waste in health care processes
Lean and Six Sigma methods are parallel to and can be used with approaches like PDSA (e.g., as part of the “D” stage, in PDSA), as well as in conjunction with collaborative improvement strategies
Conclusion
Measurement concepts are important in continuous quality improvement
Most important is understanding:
variation and the causes of variability
the distinction between special and common causes of variation
There are a wide set of tools that are typically used to identify, measure and interpret the results of data collection in CQI
The ultimate goal of measurement and CQI tools for health care organizations is that the people within them learn how to make system improvements, to assess the impact of improvements that have been made to, in order to increase the knowledge gained and motivate further improvements.
