Scenario: West Central Hospital
canmic
dashboard.pptBoards, Dashboards, and Data
From the Top: Getting the Board on Board
1-3 p.m., June 11, 2007
Boston, Massachusetts
James L. Reinertsen, M.D.
Boards ask two types of questions about quality and safety
How good is our care?
How do we compare to others like us?
Is our care getting better?
Are we on track to achieve our key quality and safety objectives?
If not, why not? Is the strategy wrong, or is it not being executed effectively?
For all of these questions…
In God we trust.
All others bring data.
Yes, but what data?
| Purpose of Measurement | Research | Comparison or Accountability | Improvement |
| Key question | “What is the truth?” | “Are we better or worse than…?” | “Are we getting better?” |
| Penalty for being wrong | Misdirection for the profession | Misdirected reward or punishment | Misdirection for an initiative |
| Measurement requirements and characteristics | Complete, accurate, controlled, glacial pace, expensive | Risk adjusted, with denominators, attributable to individuals or orgs, validity | Real time, raw counts, consistent operational definitions, utility |
| Typical displays | Comparison of control and experimental populations | Performance relative to benchmarks and standards… | Run charts, control charts, time between events… |
| Adapted from Solberg,Mosser, McDonald Jt Comm J Qual Improv. 1997 Mar;23(3):135-47. |
Example of an answer to
“How good is our care?”
Date of this report is October 24, 2006
Hospital could be “green” but still worse than median of comparison group
Compared to others
Another example of “How do we compare?”
Hospital Adverse Events per 1,000 Patient Days
Adverse Events Include (but are not limited to):
- Allergic rash
- Excessive bleeding, unintentional trauma of a blood vessel
- Respiratory depression requiring intubation due to pain medications
- Hyperkalemia as the result of overdose of potassium
- Lethargy/shakiness associated with low serum glucose
- Drug-induced renal failure
- Surgical site infection, sepsis, infected lines, other hospital-acquired infections
- Internal bleeding following the first surgery and requiring a second surgery to stop the bleeding
- Atelectasis, skin breakdown, pressure sores
- DVT or pulmonary embolism during a hospital stay
Source: Roger Resar, John Whittington, IHI Collaborative
150
Number of Adverse Events per 1,000 Patient Days
Using IHI Global Trigger Tool
0
25
50
75
100
125
Current IHI Best
IHI Average
5
40
Our Hospital, May 2007
深層靜脈栓塞症(deep vein thrombosis, DVT)的症狀、深層靜脈栓塞症(DVT)的治療
What Boards should know about data on “How good are we and how do we compare to others?”
Upside
- Often risk adjusted
- Apples to Apples
- Source of pride
- Source of energy for improvement
Downside
- Time lag (months)
- Static (no data over time)
- If you look bad, energy is wasted on “the data must be wrong”
- If you look good, you become complacent
- How you look depends on how others perform
- Standards and Benchmarks are full of defects (“The cream of the crap”)
Recommendations for Board use of
“How do we compare to others?”
Ask this question to help you set aims, and perhaps annually thereafter, but don’t use these sorts of reports to oversee and guide improvement at each meeting.
Compare to the best, not the 50th %tile
e.g. Toyota Specs
Always make sure you know how “Green” is determined
Boards ask two types of questions about quality and safety
How good is our care?
How do we compare to others like us?
Is our care getting better?
Are we on track to achieve our key quality and safety objectives?
If not, why not? Is the strategy wrong, or is it not being executed effectively?
Where dashboards and scorecards can be helpful to boards
1.1 Satisfy Our Patients
Example: Immanuel St. Joseph’s Mayo Health System Board’s answer to the question “Is our mortality rate getting better?”
Available in January 2007!
13.unknown
Is our quality and safety getting better?
Are we going to achieve our aims?
- To answer these questions for Boards…
The aims should be clearly displayed and understood
A few system-level measure(s) should be graphically displayed over time
The measures should be displayed monthly, at worst, and should be close to “real time”
Measures do not necessarily need to be risk adjusted
Measures of critical initiatives (projects that must be executed to achieve the aim) should be available if needed to answer the Board’s questions
The Board question “are we going to achieve our aims?” requires management to have a strategic theory
| Big Dots (Pillars, BSC…) | Drivers (Core Theory of Strategy) | Projects (Ops Plan) |
| What are your key strategic aims? How good must we be, by when? What are the system-level measures of those aims? | Down deep, what really has to be changed, or put in place, in order to achieve each of these goals? What are you tracking to know whether these drivers are changing? | What set of projects will move the Drivers far enough, fast enough, to achieve your aims? How will we know if the projects are being executed? |
The ideal dashboard will display a cascaded set of measures that reflect the “theory of the strategy.”
Example Dashboard for Harm
(for 5M Lives Campaign)
System Level Measure: Global Harm Trigger Tool
Drivers: Handwashing, culture of discipline, and teamwork
Projects: High alert meds, surgical complications, pressure ulcers, CHF, MRSA
Board
The full Board should review the System-level Measures (Big Dots.) The Board Quality Committee should review both the System-level Measures and the Key Drivers of those Measures. Occasionally, but not often, the Board will need to see measures of Key Projects, but these are generally the responsibility of management to oversee and execute.
Common Flaws in Dashboards
- No system-level measures or aims (so it’s possible to quality and safety to be worse, and yet to achieve “green” on all the measures the Board sees!)
- Hodge-podge of system, driver, and project measures (so the Board doesn’t know what’s important)
- Static measures (so the Board has to take management’s word that “we’re on track to achieve our aims”
- Too many measures (so the Board doesn’t understand any of them)
- Mixture of “How do we compare to others” and “are we getting better?” measures (so the Board doesn’t know what questions to ask)
- Low, unclear standards for “green” (so the Board becomes complacent despite significant opportunities for improvement!)
Can you identify the flaws in the following “dashboard?”
歐美先進國家十年來膝與髖關節置換術的手術方法、置入物的材質與設計都有長足的精進。感染率約下降到0.5%到2%。本國的文獻則甚少討論關節置換術的感染率。為瞭解台灣有關關節置換術感染的情況，我們回溯性調查了二年六個月的本院關節置換手術個案，針對膝髖關節置換術感染的危險因子、感染菌種、治療及預後等做一探討。本調查收集關節置換術病患共68人，感染率為20.6%(膝關節置換術感染率29.3%、髖關節置換術感染率7.4%)。其中57.1%是在術後門診追蹤時才發現。經個案對照研究資料顯示，有統計上差異之變項為年齡及平均住院天數。其他潛在性疾病，如糖尿病等並無關連。細菌的採檢方法不良(多用棉花棒沾取傷口處)，使得致病菌的來源判斷困難，多為coagulase-negative Staphylococci。預防性抗生素使用時間高達5.2天，與一般建議不同。建議增進手術預防性抗生素的正確使用，以期日後能降低感染率。我們也強調手術後門診追蹤應列入感染管制作業的標準流程，以確實追查到手術感染的個案
|
Measure |
Current Performance |
Goal for 2007 |
|
Acute MI Core Measures |
6h Decile National, 4h decile State |
2nd state decile or above |
|
Congestive Heart Failure Core Measures |
4th Decile National, 2nd decile State |
2nd State decile or above |
|
Pneumonia Core Measures |
3rd Decile National, 1st Decile State |
2nd State decile or above |
|
Press-Ganey Patient Satisfaction |
57% Rate us “Excellent” |
Statistically significant improvement i.e 62% “Excellent” rating |
|
OR Turnover Time |
22 minutes |
15 minutes |
|
Falls |
7 per 1000 patient days |
Less than 5 per 1000 patient days |
|
Medication Errors |
5.1 per 1000 patient days (from Nurse Variance Reports) |
Less than 7 per 1000 patient days |
|
Total Knee and Hip Infection Rates |
1.2% |
Less than 4.1 % i.e. Better (lower) than 50th %tile for NNIS |
|
Surgical Site Infection Rates for Cardiac Surgery |
4.2% |
Less than 10.4% i.e. Better (lower) than 50th %tile for NNIS |
|
Time to answer nurse call lights on all Med/Surg Units |
We are developing a standard measure, and will report in future meetings to Board on this initiative |
We are aiming to achieve significant improvement in timeliness of response to patients concerns. |
No display over time
Low standards for “Green”
Mix of system, project measures
Mostly comparison measures
|
Measure |
Current Performance |
Goal for 2007 |
|
Acute MI Core Measures |
6h Decile National, 4h decile State |
2nd state decile or above |
|
Congestive Heart Failure Core Measures |
4th Decile National, 2nd decile State |
2nd State decile or above |
|
Pneumonia Core Measures |
3rd Decile National, 1st Decile State |
2nd State decile or above |
|
Press-Ganey Patient Satisfaction |
57% Rate us “Excellent” |
Statistically significant improvement i.e 62% “Excellent” rating |
|
OR Turnover Time |
22 minutes |
15 minutes |
|
Falls |
7 per 1000 patient days |
Less than 5 per 1000 patient days |
|
Medication Errors |
5.1 per 1000 patient days (from Nurse Variance Reports) |
Less than 7 per 1000 patient days |
|
Total Knee and Hip Infection Rates |
1.2% |
Less than 4.1 % i.e. Better (lower) than 50th %tile for NNIS |
|
Surgical Site Infection Rates for Cardiac Surgery |
4.2% |
Less than 10.4% i.e. Better (lower) than 50th %tile for NNIS |
|
Time to answer nurse call lights on all Med/Surg Units |
We are developing a standard measure, and will report in future meetings to Board on this initiative |
We are aiming to achieve significant improvement in timeliness of response to patients concerns. |
Summary of Best Practices for Quality and Safety Dashboards for Boards
- Separate the two types of oversight questions
How good is our quality? How do we compare to others?
Are we getting better? Are we on track to achieve our aims?
- Ask the comparison question annually, when setting quality and safety aims. Avoid use of comparative data to track improvement.
- Frame your aims with reference to the theoretical ideal, and to the “best in the world,” not to benchmarks
- Ask the ‘improvement question’ at every meeting, and track with a dashboard that shows real-time data on system level and driver measures displayed on run charts
- Demand that management develop a “theory of the strategy to achieve the annual quality and safety aims
- Do not put project-level measures (often about one unit, disease, or department) on the Board’s dashboard
D
e
a
t
h
s
p
e
r
1
0
0
0
D
i
s
c
h
a
r
g
e
s
Y
r
M
o
n
.
2
0
0
6
2
0
0
6
2
0
0
6
2
0
0
6
2
0
0
5
2
0
0
5
2
0
0
5
2
0
0
5
2
0
0
4
2
0
0
4
2
0
0
4
2
0
0
4
2
0
0
3
2
0
0
3
2
0
0
3
2
0
0
3
D
e
c
S
e
p
J
u
n
M
a
r
D
e
c
S
e
p
J
u
n
M
a
r
D
e
c
S
e
p
J
u
n
M
a
r
D
e
c
S
e
p
J
u
n
M
a
r
4
0
3
5
3
0
2
5
2
0
1
5
1
0
M
o
n
t
h
l
y
1
2
m
o
r
o
l
l
i
n
g
r
a
t
e
V
a
r
i
a
b
l
e
I
n
p
a
t
i
e
n
t
M
o
r
t
a
l
i
t
y
B
e
n
c
h
m
a
r
k
8
/
7
/
2
0
0
6
;
P
r
e
p
a
r
e
d
b
y
I
m
m
a
n
u
e
l
S
t
.
J
o
s
e
p
h
'
s
-
M
a
y
o
H
e
a
l
t
h
S
y
s
t
e
m
Q
u
a
l
i
t
y
R
e
s
o
u
r
c
e
s
D
e
p
a
r
t
m
e
n
t
I
m
m
a
n
u
e
l
S
t
.
J
o
s
e
p
h
'
s
0
20
40
60
80
100
120
JanFebMarAprMay
Global Harm
Trigger Tool
Measure Current Performance Goal for 2007
Acute MI Core
Measures
6
h
Decile National, 4
h
decile State 2
nd
state decile or above
Congestive Heart
Failure Core Measures
4
th
Decile National, 2nd decile State 2nd State decile or above
Pneumonia Core
Measures
3rd Decile National, 1
st
Decile State 2nd State decile or above
Press-Ganey Patient
Satisfaction
57% Rate us “Excellent” Statistically significant improvement
i.e 62% “Excellent” rating
OR Turnover Time 22 minutes 15 minutes
Falls 7 per 1000 patient days Less than 5 per 1000 patient days
Medication Errors 5.1 per 1000 patient days (from Nurse
Variance Reports)
Less than 7 per 1000 patient days
Total Knee and Hip
Infection Rates
1.2% Less than 4.1 %
i.e. Better (lower) than 50
th
%tile for
NNIS
Surgical Site Infection
Rates for Cardiac
Surgery
4.2% Less than 10.4% i.e. Better (lower)
than 50
th
%tile for NNIS
Time to answer nurse
call lights on all
Med/Surg Units
We are developing a standard measure,
and will report in future meetings to
Board on this initiative
We are aiming to achieve significant
improvement in timeliness of
response to patients concerns.
42
44
46
48
50
52
54
56
58
JanFebMarAprMay
Culture of
discipline
on safety
rules
0
10
20
30
40
50
60
70
JanFebMarAprMay
Handwashin
g
0
10
20
30
40
50
60
70
JanFebMarAprMay
Teamwork
0
20
40
60
80
100
120
JanFebMarAprMay
Harm from
high alert
meds
0
2
4
6
8
10
12
14
16
18
20
JanFebMarAprMay
Surgical
complicatio
ns
0
5
10
15
20
25
30
35
JanFebMarAprMay
Pressure
Ulcers
0
2
4
6
8
10
12
JanFebMarAprMay
MRSA
0
2
4
6
8
10
12
14
16
JanFebMarAprMay
CHF
Readmissio
ns
