i have home work after 1 hour
ibr333h
chapter_3_-_fall_2016.pptxChapter 3
Measuring Performance
Outline
Measurement in Quality Management
Measurement Characteristics
Introduction
Accuracy
Usefulness
Ease of Interpretation
Consistent Reporting
Measurement Categories
Background and Definitions
Structure Measurement
Process Measurement
Outcome Measurement
Outline (2)
Selecting Performance Measures
Introduction
Measurement Priorities
Constructing Measures
Identify the Topic of Interest
Develop the Measure
Design the Data Collection System
What
Who
When
How
Measure Specifications
Outline (3)
Measures of Clinical Decision Making
Balanced Score Card Measures
Measurement in quality management (QM)
Review
Measurement is the starting point of all QM activities
Organizations use measurement to determine how it is performing and whether or not they are meeting expectations; if not, then they implement changes
Measurement Characteristics
Introduction
Performance measures are quantitative (meaning they use numerical data) tools used to evaluate an element of patient care
For reference, qualitative data is nominal (e.g. open ended questions that require a text response, such as the quiz questions)
See notes for a review of the types of statistics and an example of the statistic
In order to be effective, performance measures must be:
Accurate
Easy to interpret
Consistent
Absolute number – i.e. number of patients served in the health clinic, number of patients who fall while in the hospital, number of billing errors
Percentage – i.e. percentage of nursing home residents who develop an infection, percentage of newly hired staff who receive job training, percentage of prescription filled accurately by pharmacists
Average – i.e. average patient length of stay in the hospital, average patient wait time in the emergency department, average charges for laboratory tests
Ratio – i.e. nurse-to-patient ratio, cost-to-charge ratio, technician-to-pharmacist ratio
8
accuracy
When discussing performance measures, accuracy relates to the correctness of the numbers (e.g. if it was reported there were 37 patients discharged on Monday, but really only 35 were discharged that is inaccurate and could lead to staffing issues, lack of beds for patients, etc.)
Accuracy also relates to validity-is the measure gathering the information it is supposed to be gathering?
Think about this when constructing surveys-is the question you are asking clear? Could it have more than one interpretation?
usefulness
Measures must tell people something they want or need to know
Again, are you correcting the right data? Will it be used for anything? Is it needed? Does it answer a specific question?
Data collection and analysis is a very time consuming, difficult process-you do not want to waste time going through the process for data that will sit untouched
Ease of interpretation
Data must be easily interpreted-meaning can you easily see what the results are telling you?
Line graphs, bar charts, and pie charts are great ways to display data-most individuals can easily interpret these type of data representations
Consistent reporting
Consistent reporting is important because data used in health care is often needed to make comparisons
It is also often used to look at trends over time (e.g. customer satisfaction scores over a five year period)
A simple, minor difference in data reporting can drastically affect results of data
In healthcare, data is often portrayed in line graphs to allow for trend analysis
Measurement categories
Background
Performance is measured in three ways:
Structure
Process
Outcome
These categories were developed in 1966 by Avedis Donabedian, MD
He believed these three categories represented different characteristics of health care services
Background (2)
Structure measures look at the health care environment (hospital or facility, often reported as an absolute number)
Ex: Number of hours per day that a person skilled in reading head CT scans is available
Process measures look at the health care delivery process to ensure services are properly performed and delivered
Ex: Percentage of ED patients less than 13 years old with a current weight in kilograms document in the ED record
Outcome measures look at results of patient care
Ex: Median time from ED arrival to ED departure for patients admitted to the hospital
Structure measurement
These measures evaluate the physical and organizational resources available
These are considered indirect measures of performance
Examples: Staffing ratios, number of beds available, etc
Process measurement
Evaluates actual activities/services and whether or not they are performed or delivered properly
These are the most commonly used type of measure in health care
Outcome measurement
Evaluate the results of health care services
Typically looks at health outcomes (patient health status) following treatment or inpatient stay
Hospitals also measure patient mortality rates and complication rates in order to identify opportunities for improvement
Measuring outcomes is important, but be sure to note that they can be affected by outside factors beyond the control of a physician or hospital
Example of all three categories
Selecting performance measures
Introduction
Health care organizations use two tiers of measures to evaluate performance:
System level-think organization wide or several departments
Activity level-think one process or activity; or one department
System and activity level measures are influenced by both external and internal factors
Externally, these measures may be influenced by government regulations, accreditation standards, and purchaser requirements
Government regulations
Performance measurement requirements of the government continue to increase in response to quality improvement and cost-containment efforts
Quality measurement is a part of the HITECH Act
HITECH Act established incentives for the adoption and meaningful use of EHR
Part of “meaningful use” includes the ability to electronically report performance measurement data to CMS (Center for Medicaid and Medicare Services) or to the state.
**We will discuss HITECH further later in the course**
Government regulations (2)
State licensing regulations often require a organization or facility to evaluate structural issues, such as compliance with safety and sanitation codes
Licensing regulations may also have requirements regarding process and outcome measures
This is just another example of how external forces can influence the types of performance measures collected and reported by a health care facility
It is important to note that certain state and federal regulations apply to only specific health care organizations or facilities
Accreditation standards
Accreditation standards (regardless of the accrediting body or board) often contain performance measurement requirements
Luckily, a lot of these overlap with government mandated requirements
But, many consider these government mandated requirements to be minimum standard and have requirements that go above and beyond
HEDIS Measures
See required reading about HEDIS measures and their purpose
See RR7 – HEDIS Measures
24
Internal measures
Many times external performance measures are very different than the quality concerns a health care organization identified internally
Examples of Clinic Performance Measures are in Required Reading 8
See RR 8 – Examples of Clinic Performance Measures
25
Constructing measures
Constructing measures
Developing performance measures includes three steps:
Identifying the topic of interest
Developing the measure
Designing the data collection system
Identify the topic of interest
First step is to actually decide on what you want to know or identify a process that can be improved (think of the plan step in PDCA)
An organization can also compare a process to the six dimension of healthcare quality to help determine appropriate performance measures for their organization/facility
Develop the measure
After identifying a question or questions, the next step is to actually develop the measure
Measures are often reported as a number, percentage, average, rate or ratio
Using the radiology department as the example, the manager may want to know:
The percentage of X-ray films that cannot be located within 15 minutes
Design the data collection system
This is the process of actually collecting data and determining how data will be collected
This process looks at:
What
Who
When
How
Examples of where data can be collected include: administrative files (patient demographics, diagnosis and procedure codes, count data); patient records (outcome measures, treatments, procedures) and miscellaneous information (patient satisfaction surveys, employee surveys, research studies, etc)
Design the data collection system (2)
There are two types of data collection:
Primary-data is collected through surveys, interviews, focus groups. This also includes instances where data is not already available to answer the identified question(s). This is when the data would be collected for that purpose
Secondary-analyzing data that is already available (through existing information sources or databases)
Design the data collection system (3)
What:
This refers to the population that will be measured
Will you look at certain age groups or genders? Will you look at all patients or only the ones that have been admitted in the last 30 days?
Health care organizations should use sampling methods in order to determine sample size
For example, if only looking at female patients, do you need to pull charts for all of them? No, this is too time consuming and difficult but you results would be considered statistically significant if you have a population size of 500 or larger and sampled 70 of them (think back to biostatistics)
Who:
Refers to the data collectors or investigators
This is when it is decided if staff will be used, if someone will be hired, if a consulting firm is hired to do the data collection, etc.
Design the data collection system (4)
When:
Refers to the frequency and time frame of data collection
Will it be collected annually? Semi-annually? Quarterly?
Will patients that were admitted 3 years ago be considered? Or just those within the last FY?
How:
Refers to the process used to collect data
This goes back to primary and secondary data mentioned previously
Measure specifications
Most performance measures required by the government, licensing bodies, and accreditation groups have already gone through a stringent development, testing, and validation process
This means that managers or administrators do not have to come up with what they will measure it and the steps in developing the measure
These available measures are very detailed and work under an operational definition that outlines: the measure, how it is collected, and where the data can be found
Because secondary data is often collected by multiple people and stored in various areas and formats they require what is known as a codebook (your text refers to this as a data dictionary; but most statisticians and individuals involved in data call this a code book)
Measure specifications
Code books outline how each item of data is coded
For example:
A survey that asks gender may code the variable of gender as male, female, or other; it can also code it numerically as 1=male, 2=female, and 3=other
When we analyze data it has to be numeric, so all text is transformed into a number
Another example: 1= yes, 2= no
If you used a survey that has a likert scale (strongly agree to strongly disagree) then it would be coded as:
1=Strongly Disagree
2=Disagree
3=Neutral
4=Agree
5=Strongly Agree
So if you saw a 4, you could check the codebook to see that 4=agree. The patient agrees that they received quality care at the facility
Measures of clinical decision making
Clinical decision making
Process by which physicians and other clinicians determine which patients need what and when
Health care organizations measure both the service aspects of performance and the quality of clinical decision making
Process measures determine whether physicians/clinicians are making the right patient management choices
Outcome measures are used to evaluate the results of those choices
These measures undergo the same three steps of development as other performance measures
Clinical decision making
Measures of clinical decision making are established in clinical practice guidelines
These guidelines assist physicians in making decisions about patient care in regards to specific clinical circumstances
Measures of clinical decision making are known as evidence-based measures
These are measures that have been established through research and expert consensus
Exhibit 3.16 provides examples of evidence based measures
Balanced scorecard
Balanced scorecard
Used to evaluate achievement of operational objectives
Score card measures look at four categories:
The customer (think customer satisfaction surveys as the type of data collected)
Internal business (internal operations and processes)
Learning and growth (this documents skill sets, workforce capacity, and things such as number of professional meetings attended)
Financial (Costs, ROI, accounts receivable)
Exhibit 3.7 provides examples of scorecard measures for each category
Balanced scorecard (2)
The scorecard is built into the organization’s strategic plan
Typically the organization will focus on what they call Key Performance Indicators (KPIs) or objectives they want to meet for that fiscal year (FY)
These KPIs are built into the strategic plan as an action item, then a staff member or team will be assigned with that KPI
The scorecard provides the data to measure their performance against (think of it as a benchmark), it can also be used to look at performance overtime (trend data)
