CreatingQualityImprovementCultureinPublicHealthAgencies1.pdf
Creating Quality Improvement Culture in Public Health Agencies Mary V. Davis, DrPH, MSPH, Elizabeth Mahanna, MPH, Brenda Joly, PhD, Michael Zelek, MPH, William Riley, PhD, Pooja Verma, MPH, and Jessica Solomon Fisher, MCP
Increasingly, local public health agencies are implementing quality improvement projects.1---6
According to results from the 2010 National Association of County and City Health Officials (NACCHO) profile survey, 84% of local public health agencies reported implementing some form of quality improvement effort,7 with 15% conducting agency-wide quality improvement. These efforts build on foundational initiatives designed to improve agency performance and create a culture of continuous quality improve- ment.8---11
However, agency efforts to move from spo- radic projects to creating a quality improve- ment culture and sustaining the improvement of performance have been challenging.12 Bar- riers include perceptions about lack of rele- vance, time, and financial resources to conduct quality improvement activities; inexperience and insufficient training about the use quality improvement tools and concepts; lack of lead- ership commitment to quality improvement; the need to create a manageable scope and appropriate measures for a quality improve- ment project; and public health crises.2---4,6,13
Furthermore, agencies that serve larger popu- lations are more likely to conduct quality improvement projects, and small agencies may be more likely to consider quality improve- ment an add-on activity.14,15
Facilitators to conducting quality improve- ment and creating a quality improvement culture include a commitment from senior managers who empower employees closest to the issue to make changes, the creation of activities with clear performance criteria, and the institutionalization of continuous improvement into everything the organization does.12 Specific strategies include involving more staff in quality improvement efforts and providing training to spread quality improvement competence. Quality improvement becomes part of the agency culture through a process of repetition, saturation, and spread. Agency development of advanced quality
improvement maturity occurs when administra- tors take active roles and agencies use a specific quality improvement framework and perfor- mance data.6 Experience from health care sup- ports these findings, particularly the need for leadership support to transfer quality improve- ment efforts to the whole organization.16,17
Building on the literature, we examine why, in what respects, and under what cir- cumstances select agencies develop a quality improvement culture. We explore the factors that support or hinder development of such a culture and propose a logic model for quality improvement culture development in public health agencies.
METHODS
This qualitative study is the second part of a mixed-method, iterative evaluation examining
the effectiveness of quality improvement trainings provided by NACCHO and the impact of staff and agency participation in quality improvement training on advancing quality improvement culture in an agency. The first phase18 included a survey of agency staff who participated in NACCHO-sponsored quality improvement trainings (Webcasts, 1-day workshops, and demonstration site opportu- nities) to evaluate which training type had the greatest impact. Findings revealed that dem- onstration site participants reported greater gains in knowledge, skills, and perceived abil- ity to conduct a quality improvement project. Demonstration site activities included a quality improvement project to address a gap identi- fied through an agency self-assessment. Project examples included improving HIV testing rates and streamlining animal bite---reporting processes.3 We used the survey results to
Objectives. We conducted case studies of 10 agencies that participated in early
quality improvement efforts.
Methods. The agencies participated in a project conducted by the National
Association of County and City Health Officials (2007–2008). Case study partic-
ipants included health directors and quality improvement team leaders and
members. We implemented multiple qualitative analysis processes, including
cross-case analysis and logic modeling. We categorized agencies according to
the extent to which they had developed a quality improvement culture.
Results. Agencies were conducting informal quality improvement projects
(n = 4), conducting formal quality improvement projects (n = 3), or creating a
quality improvement culture (n = 4). Agencies conducting formal quality improve-
ment and creating a quality improvement culture had leadership support for
quality improvement, participated in national quality improvement initiatives, had
a greater number of staff trained in quality improvement and quality improvement
teams that met regularly with decision-making authority. Agencies conducting
informal quality improvement were likely to report that accreditation is the major
driver for quality improvement work. Agencies creating a quality improvement
culture were more likely to have a history of evidence-based decision-making and
use quality improvement to address emerging issues.
Conclusions. Our findings support previous research and add the roles of national
public health accreditation and emerging issues as factors in agencies’ ability to
create and sustain a quality improvement culture. (Am J Public Health. 2014;104:
e98–e104. doi:10.2105/AJPH.2013.301413)
RESEARCH AND PRACTICE
e98 | Research and Practice | Peer Reviewed | Davis et al. American Journal of Public Health | January 2014, Vol 104, No. 1
refine the study hypotheses for the second phase described in this article.
In the second phase, we sequentially con- ducted key informant interviews and case studies with demonstration site agencies. In- terviews explored how participation in the demonstration site project affected the initia- tion and management of quality improvement projects. Case studies explored the extent to which demonstration site agencies could de- velop a quality improvement culture. This article presents case study results and illus- trates how these agencies are progressing to develop quality improvement cultures.
Study Sample
We selected 10 case study sites from agencies that participated in a NACCHO dem- onstration site project and achieved most of their specified aims (NACCHO, unpublished data, 2008), and where multiple staff in these agencies responded to the phase 1 survey and reported that the agency had implemented at least 1 quality improvement effort following the demonstration site project (n = 14).
Within the pool, we chose agencies with a mix of characteristics to minimize the impact of characteristics that could affect quality improvement culture; these were population size (small, < 50 000; medium, 50 000---499 999; large, ‡ 500 000) and governance mecha- nisms (local, state, and mixed).14 We also purposively chose agencies that had and had not participated in other quality improvement efforts, such as the Multi-State Learning Col- laborative (MLC).
Data Collection Protocols
We created multiple interview protocols for the case studies to be administered as part of a site visit. We conducted individual in- terviews with agency administrators and quality improvement leaders. We conducted group interviews with quality improvement staff teams. Protocols explored barriers, facili- tators, and practice-based evidence from the MLC regarding agency progress toward a quality improvement culture,2 as well as additional factors identified in our interviews. Protocol items examined leadership support for and commitment to quality improvement, the general organizational culture and context, participation in quality improvement trainings,
participation in other quality improvement efforts, implementation of quality improve- ment initiatives following the demonstration site project, agency structural support for quality improvement (such as a quality im- provement team or council), the influence of accreditation and other external drivers for quality improvement, and the impact of emerging issues, such as the H1N1 outbreak and the economic downturn, on an agency’s ability to sustain quality improvement. We also collected available, relevant documents on quality improvement projects implemented after the demonstration site project. Case study data collection occurred between Feb- ruary and May 2011.
Analysis
We recorded and transcribed case study interviews. On the basis of their initial obser- vations, interviewers ranked each agency on a scale from 1 (lowest quality improvement implementation) to 5 (highest quality im- provement implementation and development of a quality improvement culture) and recorded factors that differentiated the agency ratings. Case study analysis included creating a code- book of themes derived from the variables of interest. We combined transcripts from all participants in a case study site to create 1 transcript for each site. Three team members piloted the theme codebook on 3 site tran- scripts. Two research team members double- coded the majority of transcripts to reach complete agreement on coding. A single team
member coded the remaining site transcripts and a second member validated them.
We implemented multiple qualitative anal- ysis processes, including cross-case analysis and logic modeling.19,20 Iterative cross-case analysis included development of analytic matrices to identify and test features of agencies that achieved varying levels of qual- ity improvement culture. The evaluation team analyzed site transcripts with Atlas.ti version 6 qualitative data software (Atlas.ti Scientific Software Development GmbH, Berlin, Ger- many), and again ranked the agencies on the same scale of lowest to highest quality im- provement culture. The second set of ratings agreed with the initial ratings. We compared site features within categories and across categories20 to clarify features that differenti- ated levels of quality improvement culture.21
We reviewed agency documentation of quality improvement projects to validate ratings and distinguishing features between categories.
RESULTS
Table 1 presents characteristics of partici- pating agencies. One site served a small pop- ulation, 7 served medium populations, and 2 served large populations. Eight had local governance structures, and 1 each had a mixed and a state governance structure; 8 were single-county jurisdictions and 2 were district agencies. Four participated in the MLC, and 2 had used National Public Health Per- formance Standards Program tools. Three
TABLE 1—Characteristics of Case Study Agencies and Their Participation in Other Quality
Improvement Efforts
Site Population Size Served Governance
Multi-State Learning
Collaborative Participation
National Public Health Performance
Standards Participation
C1 50 000–499 999 Local No No
C2 < 50 000 Local No No
C3 50 000–499 999 Local No No
C4 50 000–499 999 Mixed No No
C5 ‡ 500 000 Local No Yes C6 50 000–499 999 State No Yes
C7 ‡ 500 000 Local Yes No C8 50 000–499 999 Local Yes No
C9 50 000–499 999 Local Yes No
C10 50 000–499 999 Local Yes No
RESEARCH AND PRACTICE
January 2014, Vol 104, No. 1 | American Journal of Public Health Davis et al. | Peer Reviewed | Research and Practice | e99
agencies were in states with state-based ac- creditation programs.
Seventy-nine individuals participated in the case study site interviews, with 4 to 14 partic- ipants at each site. At all sites, the agency administrator, the quality improvement team lead, and quality improvement team members participated in the interviews. Other participants were management team members and staff working in various public health programs.
Following analysis, we collapsed the initial rating scheme from 5 to 3 categories for explanatory purposes. These categories are combinations of well-defined quality im- provement culture development categories described in the NACCHO Roadmap to an Organization-Wide Culture of Quality Im- provement.22 The roadmap outlines common organizational characteristics for 6 phases of quality improvement culture development. Using the quality improvement roadmap for definitional foundations, we found that 4 case study agencies had features of an informal quality improvement culture, 3 had features of a formal quality improvement culture in specific areas, and 3 had features of creating a quality improvement culture but had not achieved all elements of such a culture.12
Categories were cumulative; in other words, agencies creating a quality improvement cul- ture had all the features of the previous 2 categories.
Table 2 presents category definitions and features that differentiate agencies conducting informal quality improvement, conducting formal quality improvement, and creating a quality improvement culture. The research team noted that there were greater differences in the features of the agencies conducting informal quality improvement compared with the other 2 categories. The commitment of the agency administrator and leadership to quality improvement was the key feature of agencies conducting formal quality im- provement and creating a quality improve- ment culture. Among these agencies, the administrator was more likely to be a strong, vocal proponent of quality improvement and effective at driving efforts, establishing a quality improvement culture, and garner- ing support.
An agency quality improvement team leader stated,
I really think that [the agency administrator] is the one who really keeps the ball rolling because I remember when a lot of the quality improve- ment stuff started, people came kicking and screaming, “It’s another thing we have to do,” and they didn’t understand how it was going to benefit them and it’s quite complex. I think a lot of administration is on board now and they’re in their individual work groups and so they can see progress and it’s not so much a thing I have to do but something they can see a benefit in.
Both the administrator and agency staff, especially senior staff, exhibited high levels of commitment to quality improvement and were more likely to be involved in state and national quality improvement initiatives. An agency quality improvement team leader said, “I think overall [senior management is] very committed. . . . They come with ideas. I’ve never had any resistance from them along the way.”
Agencies conducting formal quality im- provement and creating a quality improve- ment culture also tended to have a strong organizational culture with aligned mission and goals, low staff turnover, and a strong commitment to quality improvement training with a sizable percentage of staff trained in quality improvement practices. These agencies exhibited a well-developed quality improve- ment infrastructure characterized by teams that conduct quality improvement projects with cross-divisional representation, staff au- thority in quality improvement decision-mak- ing, and activities informed by a strategic or quality improvement plan. In addition, these agencies were more successful at leveraging outside support from the MLC, NACCHO, county governance, and other entities to fa- cilitate their quality improvement work. For example, boards of health and, in one case, the mayor, helped further quality improvement efforts by approving funding and staff posi- tions or leveraging other assistance.
Although staff in these agencies supported national public health accreditation, they did not view accreditation as driving quality im- provement initiatives. Instead, they viewed quality improvement as a key component of agency functioning in and of itself. An agency quality improvement team member said, “I don’t think [the agency administrator] just de- cided to write a policy on that just because of national accreditation. It’s because she ex- pects us to use it.”
Staff in agencies conducting informal quality improvement were likely to report that ac- creditation was the major driver for quality improvement work. A quality improvement team leader described how quality improve- ment was presented to staff in the agency:
When they hear about quality improvement, we always try to associate it to accreditation because we know moving forward in accreditation we’re going to have to have some formal quality improvement process.
Although agencies conducting formal quality improvement and creating a quality improve- ment culture were similar on most features, there were sufficient differences on some features that warrant discussion. Agencies creating a quality improvement culture were more likely to have a history of evidence- based decision-making and performance mea- surement. Quality improvement seemed to be a natural fit for these agencies, and they typically had data collection systems and methods and a performance-monitoring mindset already in place. Also, these agencies were more likely to manage emerging issues such as the H1N1 pandemic and budget cuts with quality improvement processes rather than letting these barriers overwhelm or stall their quality improvement work. In general, staff in these agencies viewed barriers as motivators for quality improvement. A team member said,
I know we get excited when there’s a chance for quality improvement to come up because . . . you have less and less staff, and you still have the same amount of public, or more, that you’re serving, so it gives us a chance to really stream- line our processes, be as efficient as we can and still keep that customer satisfaction up.
By contrast, agencies conducting informal quality improvement saw barriers to doing quality improvement work as insurmountable. In agencies conducting formal quality im- provement, quality improvement sustainability could be affected by the loss of a key admin- istrator or outside support. Staff members in agencies creating a quality improvement cul- ture were more likely to view quality im- provement as being sustainable, in large part because they had established quality improve- ment as part of their organizational culture. Quality improvement was not viewed as a burden but rather as a way of working
RESEARCH AND PRACTICE
e100 | Research and Practice | Peer Reviewed | Davis et al. American Journal of Public Health | January 2014, Vol 104, No. 1
TA B LE
2 — Fe a tu re s D if fe re n ti a ti n g A ge n c ie s C o n d u c ti n g In fo rm a l Q u a li ty Im p ro ve m e n t (Q I) , C o n d u c ti n g Fo rm
a l Q I, a n d C re a ti n g a Q I C u lt u re
D iff er en tia tin g Fe at ur es
Fa ct or Af fe ct in g Q I
Ag en ci es Co nd uc tin g In fo rm al Q Ia
Ag en ci es Co nd uc tin g Fo rm al Q Ib
Ag en ci es Cr ea tin g a Q I Cu ltu re c
Ag en cy ad m in is tr at or
D oe s no t pr io rit iz e Q I; st af f te nd
to
or ga ni ze an d dr iv e Q I ef fo rt s.
Vi ew ed as “c oa ch ” or “q ua rt er ba ck ” of ag en cy Q I; of te n pa rt of st at e or
na tio na l Q I in iti at iv es ; st ro ng
vi si on
an d pa ss io na te ab ou t Q I.
Vi ew ed
as “c oa ch ” or “q ua rt er ba ck ” of ag en cy Q I; of te n pa rt of st at e or na tio na l
Q I in iti at iv es ; st ro ng
vi si on
an d pa ss io na te ab ou t Q I.
Se ni or m an ag em en t co m m itm
en t
A m in or ity
of se ni or m an ag er s ar e
co m m itt ed
to Q I.
Th e m aj or ity
of se ni or m an ag em en t ar e co m m itt ed
an d ha ve tr ai ni ng
an d su pp or t to le ad
Q I ef fo rt s.
Th e m aj or ity of se ni or m an ag em en ta re co m m itt ed an d ha ve tr ai ni ng an d su pp or t
to le ad
Q I ef fo rt s.
O rg an iz at io na l cu ltu re
Fr ag m en te d; ag en cy cu ltu re is no t a st ro ng
fa ci lit at or of Q I w or k.
Ch ar ac te riz ed
by st ro ng
te am
or ie nt at io n an d sh ar ed
vi si on
an d go al s;
cu ltu re of no -b la m e ac co un ta bi lit y en fo rc ed
by pe er s as
w el l as
le ad er s; ch an ge
an d in iti at iv e em br ac ed ; st ro ng
co m m itm
en t to
w or kf or ce
de ve lo pm en t tr ai ni ng .
Ch ar ac te riz ed
by st ro ng te am
or ie nt at io n an d sh ar ed
vi si on
an d go al s; cu ltu re of
no -b la m e ac co un ta bi lit y en fo rc ed
by pe er s as
w el l as
le ad er s; ch an ge
an d
in iti at iv e em br ac ed ; st ro ng
co m m itm
en t to w or kf or ce
de ve lo pm en t tr ai ni ng .
Q I tr ai ni ng
M or e tr ai ni ng ne ed ed
fo r st af fa t al ll ev el s.
M aj or ity
of st af f ha ve pa rt ic ip at ed
in Q I tr ai ni ng .
M aj or ity
of st af f ha ve pa rt ic ip at ed
in Q I tr ai ni ng .
Q I in fr as tr uc tu re
Q I sp or ad ic al ly pr ac tic ed ; Q I m ee tin gs
in fr eq ue nt ; m ay no t ha ve es ta bl is he d
da ta co lle ct io n sy st em s.
D es ig na te d Q It ea m an d re gu la rm
ee tin gs w ith
re pr es en ta tio n ac ro ss al l
di vi si on s; lik el y to ha ve st ra te gi c pl an
in fo rm in g Q Ia ct iv iti es an d da ta
co lle ct io n an d an al ys is in fr as tr uc tu re .
D es ig na te d Q I te am
an d re gu la r m ee tin gs w ith
re pr es en ta tio n ac ro ss al l
di vi si on s; lik el y to ha ve st ra te gi c pl an in fo rm in g Q Ia ct iv iti es an d da ta co lle ct io n
an d an al ys is in fr as tr uc tu re .
Re la tio ns hi p w ith
bo ar d of he al th
an d ci ty or co un ty go ve rn an ce
M ay no t be
st ro ng ly in vo lv ed
or su pp or tiv e
of ag en cy Q I ac tiv iti es .
Li ke ly to be
ke y su pp or te rs of Q I in iti at iv es an d w el l in fo rm ed
by
ag en cy st af f.
Li ke ly to be
ke y su pp or te rs of Q I in iti at iv es an d w el l in fo rm ed
by ag en cy st af f.
In flu en ce
of ac cr ed ita tio n
Li ke ly to be
a st ro ng
dr iv er of Q I w or k.
Q Ia ct iv iti es te nd
to be
dr iv en by a be lie fi n th e im po rt an ce of Q Ir at he r
th an
by th e ex te rn al in flu en ce
of ac cr ed ita tio n.
Q I ac tiv iti es te nd
to be
dr iv en
by a be lie f in th e im po rt an ce of Q I ra th er th an
by
th e ex te rn al in flu en ce
of ac cr ed ita tio n.
Ex te rn al Q I re so ur ce s (s uc h as
M LC , N AC CH O , gr an ts )
N ot ab le to ga th er Q Im
om en tu m fr om
us e
of ou ts id e re so ur ce s.
Re so ur ce s ar e ke y pa rt s of bu ild in g Q I; cr ea tiv e ab ou t fin di ng an d us in g
re so ur ce s; so m e ag en ci es ga in ed
ex pe rt is e an d su pp or t by w or ki ng
w ith
st at e or na tio na l in iti at iv es .
Re so ur ce s ar e ke y pa rt s of bu ild in g Q I; cr ea tiv e ab ou t fin di ng
an d us in g
re so ur ce s; so m e ag en ci es ga in ed ex pe rt is e an d su pp or t by w or ki ng w ith
st at e or
na tio na l in iti at iv es .
Au th or ity
St af f ha ve lim ite d in pu t.
St af f ar e in co rp or at ed
in to Q I de ci si on -m ak in g pr oc es se s.
St af f ar e in co rp or at ed
in to Q I de ci si on -m ak in g pr oc es se s.
Ba rr ie rs (t im e, fu nd in g, st af f,
bu dg et cu ts , em er ge nc ie s, et c. )
Ba rr ie rs te nd
to st al l Q I ac tiv iti es .
Ab le to w ith st an d so m e ba rr ie rs .
Ba rr ie rs ar e us ed
as m ot iv at or s fo r Q I.
Em er gi ng
is su es (s uc h as
H 1N 1
or bu dg et cu ts )
Te nd
to st al l Q I w or k.
Sl ow
Q I w or k, w hi ch
is ev en tu al ly ab le to re su m e.
Af fe ct Q I w or k, bu t ag en ci es te nd
to us e em er gi ng
is su es as a pl at fo rm
to he lp
m an ag e th e ev en t w ith
Q I.
Su st ai na bi lit y
Q I w or k cu rr en tly
is no t se en
as
su st ai na bl e un le ss do ne in a lim ite d w ay .
Su st ai na bi lit y is m or e lik el y, bu t m ay no t w ith st an d lo ss of ke y
pe rs on ne l.
Li ke ly be ca us e of st af f an d le ad er sh ip co m m itm
en t; Q I is no t vi ew ed
as an
ex tr a
bu rd en
bu t as
a w ay of w or ki ng
sm ar te r an d m ak in g jo bs
ea si er .
Ev id en ce -b as ed
de ci si on -m ak in g
an d pe rf or m an ce
m ea su re m en t
Le ss lik el y to ha ve a tr ad iti on
of
pe rf or m an ce
m on ito rin g in pl ac e w ith
es ta bl is he d da ta co lle ct io n sy st em s.
Le ss lik el y to ha ve a tr ad iti on
of pe rf or m an ce m on ito rin g in pl ac e w ith
es ta bl is he d da ta co lle ct io n sy st em s.
Co m m itt ed
to ev id en ce -b as ed
de ci si on -m ak in g an d pe rf or m an ce
m on ito rin g,
w hi ch
tr an sl at es
w el l to Q I.
N ot e. M LC = M ul ti- St at e Le ar ni ng
Co lla bo ra tiv e.
a Co nd uc tin g in fo rm al Q I (N AC CH O ro ad m ap
ca te go rie s 2 an d 3) : le ad er sh ip un de rs ta nd s an d di sc us se s Q I w ith
st af f bu t do es no t en fo rc e th e im pl em en ta tio n of Q I, al lo w su ffi ci en t st af f tim
e an d re so ur ce s to de vo te to it, or ho ld st af f
ac co un ta bl e to its
us e. D is cr et e Q I ef fo rt s ar e pr ac tic ed
in is ol at ed
in st an ce s th ro ug ho ut ag en cy , of te n w ith ou t co ns is te nt us e of da ta or al ig nm en t w ith
th e st ep s in a fo rm al Q I pr oc es s.
b Co nd uc tin g fo rm al Q I( N AC CH O ro ad m ap
ca te go rie s 4 an d 5) :a
fo rm al Q Im
od el is fo llo w ed ,w ith
Q I im pl em en te d in sp ec ifi c pr og ra m ar ea s w he re Q I ch am pi on s ex is t. Th e Q I pl an
m ay be
in te gr at ed
in to ov er al la ge nc y po lic ie s an d pl an s,
in cl ud in g th e st ra te gi c pl an . Po lic ie s an d pr oc ed ur es ar e in pl ac e, an d da ta ar e co m m on ly us ed
fo r pr ob le m -s ol vi ng
an d de ci si on -m ak in g.
c C re at in g a Q I cu ltu re (N AC CH O ro ad m ap
ca te go rie s 5 an d 6) : Q I is st ro ng ly em be dd ed
in to th e w ay th e ag en cy do es bu si ne ss ac ro ss le ve ls an d se rv ic es . Le ad er sh ip an d st af f ar e fu lly co m m itt ed
to qu al ity , an d re su lts
of Q I ef fo rt s ar e
co m m un ic at ed
in te rn al ly an d ex te rn al ly . Ev en
if le ad er sh ip ch an ge s, th e ba si cs of Q I ar e in gr ai ne d in st af f.
RESEARCH AND PRACTICE
January 2014, Vol 104, No. 1 | American Journal of Public Health Davis et al. | Peer Reviewed | Research and Practice | e101
smarter and making jobs easier. An agency quality improvement team member noted,
Just changing our culture just a little bit so that we do a project a little bit differently that in- corporates some of the quality improvement into it. If you can make it part of your daily activity, you’re not really doing more but you’re making an improvement.
Agency documentation of quality im- provement projects (conducted in 2009 and 2010) validated quality improvement culture category assignments. Among agencies con- ducting informal quality improvement, only one provided specific documentation dem- onstrating implementation of quality im- provement projects with specific aims and results. Among agencies conducting formal quality improvement, 2 agencies had con- ducted multiple quality improvement projects
with aim statements and results; 1 had com- pleted a quality improvement project and had another one in process, but these projects had not achieved results connected with the aim statements. Among the agencies creating a quality improvement culture, 2 had numer- ous examples of quality improvement projects with aim statements and results; the third did not provide documents for review.
Quality improvement projects included re- ducing chart errors, improving clinic show rates, improving immunization rates, and cre- ating staff development and health improve- ment plans. Agencies conducting informal quality improvement activities primarily used the Plan-Do-Study-Act process (required for the NACCHO demonstration site project) and reported being unable to complete multiple process cycles. Agencies conducting formal
quality improvement and creating a quality improvement culture were more likely to complete Plan-Do-Study-Act cycles and to em- ploy a range of quality improvement methods and resources (lean enterprise practices, scorecards, performance management, etc.).
Figure 1 illustrates how the features de- scribed throughout Results work together to depict the development of agency quality improvement culture. There are 5 key in- gredients needed to build a sustainable cul- ture of quality improvement. First, leadership and staff commitment to quality improvement is essential. Second, agencies that value in- novation and align quality improvement practices with their strategic goals and mission are more likely to build and sustain a culture of quality. Third, agencies with strong expe- rience in performance management, quality
Inputs Strategies and
Developments Outputs
Short-Term
Outcomes
Long-Term
Outcomes
Intermediate
Outcomes
Agency administrator commitment to QI
Agency culture of innovation: mission, goals, committed staff and leadership
Agency experience in QI, performance management, evidence-based decision-making
Board of health and county government support
Emerging issues: H1N1,
economic downturn
Agencies must reconsider priorities
Agency uses QI to address issues; issues are seen as motivators for QI
Sustainable QI culture
Improved population-
based health outcomes
Enhanced delivery and quality of public
health services
Increased LHD efficiency and effectiveness
Increased initiation and
implementation of QI initiatives
(team authority to make changes)
Increased spread of QI projects;
Saturation of QI in agency
Increased workforce QI knowledge and
skills
Increased agency capacity for conducting QI, including organizational structures (QI council) and training
Agencies use standards and performance measures to identify areas for QI
Accrediation standards available for
use
Select agencies also participate in NPHPSP, MLC, and national or state based QI
Agencies participate in NACCHO
demonstration site project and other QI
trainings
Available QI trainings from
NACCHO, other organizations
Other QI and performance management
resources (NPHPSP, MAPP, MLC)
Development of PHAB, state-based
programs
Agency, staff, and community characteristics, QI barriers (time and money)
Note. LHD = local health department; MAPP = Mobilizing for Action through Planning and Partnerships; MLC = Multi-State Learning Collaborative; NACCHO = National Association of County and City
Health Officials; NPHPSP = National Public Health Performance Standards Program; PHAB = Public Health Accreditation Board.
FIGURE 1—Logic model for the development of a quality improvement (QI) culture.
RESEARCH AND PRACTICE
e102 | Research and Practice | Peer Reviewed | Davis et al. American Journal of Public Health | January 2014, Vol 104, No. 1
improvement, and evidence-based decision- making are more likely to have core capacity to lead improvement efforts and encourage adoption by others. Fourth, agencies that are held accountable for the quality of their services, programs, and outcomes by govern- ing bodies, partners, funders, and others are likely to monitor their efforts and seek op- portunities for improvement. Finally, agencies with core infrastructure and resources that support ongoing quality improvement initia- tives (e.g., available data, quality improvement teams) are better positioned to sustain a cul- ture of quality improvement. Agencies adop- ted a number of strategies that influence quality improvement culture, including skill- building activities and leveraging resources to advance quality improvement efforts, identi- fying gaps related to accreditation, and addressing emerging issues. These strategies led to an increased perception of the value of quality improvement, an increase in orga- nizational structures and supports, and an increase in the ability of practitioners to apply quality improvement concepts, tools, and approaches.
This logic model hypothesizes that quality improvement efforts in agencies with a quality improvement culture will spread and become part of routine practice, resulting in improved quality of service, programs, and operations. In turn, a sustainable quality improvement culture results in long-term outcomes of increased agency efficiency and effectiveness, enhanced delivery and quality of public health services, and improved population-based outcomes. According to the literature, contextual factors such as agency and community characteristics and quality improvement barriers of time and money affect the development of a quality improvement culture. Our study supports this hypothesis and refines it. Agencies that were developing a quality improvement culture were able to withstand barriers. Furthermore, community characteristics, such as population size, were not a factor in the ability of these agencies to conduct quality improvement.
DISCUSSION
This is one of the first studies to examine the circumstances under which agencies develop a quality improvement culture. Study findings
validate previous public health and health care literature regarding quality improvement cul- ture facilitators and barriers.3,4,6,12,13,17 Most importantly, the agency administrator plays the key role as a champion for quality improve- ment activities and culture. This mirrors what has been found in health care systems, where leaders who support quality improvement mobilize change through the organization’s structure and demonstrate authentic passion for and commitment to quality through in- spiring and motivating staff.17
Additional facilitators identified in this study include support of county governance and experience with other national quality im- provement efforts. We have observed the importance of support by county managers and commissioners in other studies (North Carolina Institute for Public Health, unpublished data, 2010). Future research should examine how— and the extent to which—this support facilitates development of a quality improvement culture. Administrators in agencies creating a quality improvement culture and, in some respects, conducting formal quality improvement, reported seeking out national and state quality improvement offerings as a learning opportu- nity for the agency, an opportunity for recog- nition as an innovative agency, or both. The opportunity to practice quality improvement and leadership support for quality improve- ment as an approach to everyday work were mutually reinforcing facilitators for developing a quality improvement culture. All agencies were able to obtain resources to continue quality improvement work through grants, participation in national efforts, or local funding. Leveraging these resources helped continue quality improvement efforts, but a quality improvement culture was created through leadership commitment to quality improvement.
Barriers commonly cited in the literature, including lack of time and resources and relevance of quality improvement to daily work, were seen as insurmountable in agencies conducting informal quality im- provement and in some agencies conducting formal quality improvement. Although agencies creating a quality improvement cul- ture experienced these barriers, staff used quality improvement to overcome them and viewed these as opportunities to use quality
improvement. Nevertheless, the greatest barrier to creating a quality improvement culture and sustained performance improve- ment may reside with the leadership and teamwork within an organization.12 Our find- ings support the key role of the leadership to break through these barriers.
This study was initiated as the national public health voluntary accreditation program, or Public Health Accreditation Board, was under development. Some agencies conduct- ing informal quality improvement reported shifting the focus of improvement work from quality improvement to accreditation prepa- ration. Agencies conducting formal quality improvement and creating a quality improve- ment culture were supportive of accreditation and were actively preparing to apply, but they saw improvement, not accreditation, as the driver of quality improvement efforts. This finding is particularly encouraging in light of the fact that the Public Health Accreditation Board emphasizes health department im- provement as the overriding goal of that program.23
The literature suggests that more work is needed to better understand the culture and practice of quality improvement in relation to agency characteristics (e.g., urban vs rural, centralized vs decentralized) and based on what public health services are provided and how they are financed.2 Results from the 2008 NACCHO profile demonstrated that agencies that serve medium or large popula- tions were more likely to implement quality improvement.14 Additionally, results from the MLC evaluation revealed that agencies with a reported higher level of quality improve- ment sophistication were often more likely to be larger and have more resources.24 In this study, we included agencies that varied by population size and governance structures, although most agencies served medium pop- ulations and had local governance structures. Future research should continue to explore whether agency characteristics affect use of quality improvement and how agencies, espe- cially those that serve small populations, can be encouraged to implement quality im- provement efforts and develop a quality improvement culture.
We found NACCHO’s quality improvement roadmap to be a useful framework for defining
RESEARCH AND PRACTICE
January 2014, Vol 104, No. 1 | American Journal of Public Health Davis et al. | Peer Reviewed | Research and Practice | e103
the types of quality improvement that are occurring in agencies. Our findings support the framework and have informed roadmap revisions. We found that the 10 case study agencies grouped together in explanatory clusters rather than in the fine-grained cate- gories the roadmap defines. Additional re- search among a larger set of agencies should further explore these definitions.
We note the following limitations. Although the research team categorized agencies into quality improvement categories twice using criteria from the research hypotheses, the ratings were subjective and not verified or discussed with agency staff to gain their per- spective.21 Furthermore, the categorization scale was based on participating agencies and is not necessarily representative of a quality improvement culture continuum. In addition, findings are based on a snapshot approach and limited to agencies with characteristics similar to those included in this study. Although we purposively sampled agencies that served var- ious size populations and governance struc- tures, the agencies in this study were quite different from agencies nationwide. Finally, given the nature of the study design, the findings do not include considerations of causation.
This study provides insights as to how agencies develop a quality improvement cul- ture. We validated much of the existing literature and, in addition, we discussed the impact that emerging issues and experience with quality improvement projects can have on an agency’s ability to develop such a cul- ture. Importantly, the role of accreditation preparation as a driver for quality improve- ment appears to diminish as an agency de- velops a quality improvement culture. Future research that examines diffusion of quality improvement and development of quality improvement cultures in a variety of health department governance structures will further advance this literature. j
About the Authors At the time of the study, Mary V. Davis, Elizabeth Mahanna, and Michael Zelek were with the North Carolina Institute for Public Health, Gillings School of Global Public Health, University of North Carolina, Chapel Hill. Brenda Joly is with the Muskie School of Public Service, University of Southern Maine, Portland. William Riley is with the School of Public Health, University of Minnesota,
Minneapolis. Pooja Verma and Jessica Solomon Fisher are with the National Association of County and City Health Officials, Washington, DC. Correspondence should be sent to Mary V. Davis, DrPH,
MSPH, North Carolina Institute for Public Health, Campus Box 8165, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 (e-mail: [email protected]). Reprints can be ordered at http://www.ajph.org by clicking the “Reprints” link. This article was accepted April 21, 2013.
Contributors M. V. Davis originated and directed the evaluation research study and led article writing and editing. E. Mahanna led the data analysis, participated in data collection, and contributed to writing and editing the article. B. Joly contributed to the design of the study, research activities, and writing and editing the article. M. Zelek contributed to the data analysis. W. Riley contributed to the design of the study, research activities, and editing the article. P. Verma and J. S. Fisher con- tributed to research activities and editing the article.
Acknowledgments This research was supported by the Robert Wood Johnson Foundation.
We thank Amy Vincus, MPH, and Matthew Eggers, MPH, for their invaluable assistance with research activ- ities; the local health departments who generously shared their staff time to discuss quality improvement practices; and the National Association of County and City Health Officials for their partnership on this effort.
Human Participant Protection This evaluation study was declared exempt from review by the non-biomedical institutional review board at the University of North Carolina. Participant confidentiality was ensured throughout the study.
References 1. Davis MV. Opportunities to advance quality im- provement in public health. J Public Health Manag Pract. 2010;16(1):8---10.
2. Joly BM, Booth M, Shaler G, Mittal P. Assessing quality improvement in local health departments: results from the Multi-State Learning Collaborative. J Public Health Manag Pract. 2012;18(1):79---86.
3. Davis PB, Solomon J, Gorenflo G. Driving quality improvement in local public health practice. J Public Health Manag Pract. 2010;16(1):67---71.
4. Seid M, Lotstein D, Williams VL, et al. Quality improvement in public health emergency preparedness. Annu Rev Public Health. 2007;28:19---31.
5. Corso L, Lenaway D, Beitsch L, Landrum L, Deutsch H. The national public health performance standards: driving quality improvement in public health systems. J Public Health Manag Pract. 2010;16(1): 19---23.
6. Joly BM, Booth M, Shaler G, Conway A. Quality improvement learning collaboratives in public health: findings from a multisite case study. J Public Health Manag Pract. 2012;18(1):87---94.
7. 2010 Profile of Local Health Departments. Wash- ington, DC: National Association of County and City Health Officials; 2011.
8. Nicola R. Turning point’s national excellence col- laboratives: assessing a new model for policy and system
capacity development. J Public Health Manag Pract. 2005;11(2):101---108.
9. Halverson PK. Embracing the strength of the public health system: why strong government public health agencies are vitally necessary but insufficient. J Public Health Manag Pract. 2002;8(1):98---100.
10. Bakes-Martin R, Corso L, Landrum L, Fisher VS, Halverson PK. Developing national performance stan- dards for local public health systems. J Public Health Manag Pract. 2005;11(5):418---421.
11. Gillen SM, McKeever J, Edwards KF, Thielen L. Promoting quality improvement and achieving measur- able change: the lead states initiative. J Public Health Manag Pract. 2010;16(1):55---60.
12. Riley W, Parsons HM, Duffy GL, Moran JW, Henry B. Realizing transformational change through quality improvement in public health. J Public Health Manag Pract. 2010;16(1):72---78.
13. Riley WJ, Brewer R. Review and analysis of quality improvement techniques in police departments: applica- tion for public health. J Public Health Manag Pract. 2009;15(2):139---149.
14. Beitsch L, Leep C, Shah G, Brooks RG, Pestronk RM. Quality improvement in local health departments: results of the NACCHO 2008 survey. J Public Health Manag Pract. 2010;16(1):49---54.
15. Davis M, Cannon M, Stone D, Wood B, Reed J, Baker E. Informing the national public health accredita- tion movement: lessons from North Carolina’s accredited local health departments. Am J Public Health. 2011;101 (9):1543---1548.
16. Needleman J, Pearson M, Parketon P, et al. Trans- forming care at the bedside: lessons learned from phase II. 2007. Available at: http://www.rwjf.org/ files/research/20071015tcabphase2lessonsfinal.pdf. Accessed February 5, 2013.
17. Lukas CV, Holmes SK, Cohen AB, et al. Transforma- tional change in health care systems: an organizational model. Health Care Manage Rev. 2007;32(4):309---320.
18. Davis MV, Vincus A, Eggers M, et al.. Effectiveness of public health quality improvement training ap- proaches: application, application, application. J Public Health Manag Pract. 2012;18(1):E1---E7.
19. Yin RK. Case Study Research: Design and Methods. 4th ed. Los Angeles, CA: Sage Publications; 2009.
20. Miles MB, Huberman AM. Qualitative Data Analysis: An Expanded Sourcebook. Thousand Oaks, CA: Sage; 1994.
21. Joly BM. Multi-State Learning Collaborative Case Studies Round III: Lead States in Public Health Quality Improvement. Portland, ME: University of Southern Maine; 2011.
22. National Association of County and City Health Officials (NACCHO). Roadmap to an organization-wide culture of quality improvement. Available at: http:// www.naccho.org/topics/infrastructure/accreditation/ upload/quality improvement-Roadmap-11-16-11.pdf. Accessed February 5, 2013.
23. Riley WJ, Bender K, Lownik E. Public health department accreditation implementation: transforming public health department performance. Am J Public Health. 2012;102(2):237---242.
24. Joly BM, Shaler G, Booth M, Conway A, Mittal P. Evaluating the multi-state learning collaborative. J Public Health Manag Pract. 2010;16(1):61---66.
RESEARCH AND PRACTICE
e104 | Research and Practice | Peer Reviewed | Davis et al. American Journal of Public Health | January 2014, Vol 104, No. 1
Copyright of American Journal of Public Health is the property of American Public Health Association and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
