Applied Statistics Review Statistics
Leading for Quality in Healthcare: Development and Validation of a Competenqr Model Andrew Garman, PsyD, MS, CEO, National Center for Healthcare Leadership, and professor, health systems management. Rush University; and Linda Scribner, BA, CPHQ, director of quality and clinical outcomes management, Methodist Dallas Medical Center
E X E C U T I V E S U M M A R Y Increased attention to healthcare quality and impending changes due to health reform are calling for healthcare leaders at all levels to strengthen their skills in leading quality improvement initiatives. To address this need, the National Asso- ciation for Healthcare Quality spearheaded the development and validation of a competency model to support healthcare leaders in assessing their strengths and planning appropriate steps for development. Initial development took place over the course of several days of meetings by an advisory panel of quality profession- als. The draft model was then validated via electronic survey of a national sample of 883 quality professionals. Follow-up analyses indicated that the model was content valid for each of the target samples and also distinguished differing levels of job scope and experience. The resulting model contains six domains spanning three organizational levels.
For more information on the concepts in this article, please contact Dr. Carman at Andy_N_Carman@rush.edu or agarman@nchl.org.
373
JOURNAL OF HEALTHCARE MANAGEMENT 5 6 : 6 N O V E M B E R / D E G E M B E R 2 0 1 1
I N T R O D U C T I O N As delivery of high-quality healthcare continues to grow more complex, so do the roles of the professionals lead- ing these efforts. Recent years have seen increased focus on the leadership ele- ments ofthe quality professionals' roles; initiatives such as the Comprehensive Unit-Based Safety Program (CUSP; Pro- novost et al. 2005), crew resource man- agement. Lean Six Sigma, and Malcolm Baldrige emphasize the key elements of leadership and management needed for success (see Carman et al. 2011). Civen the impending changes associated with the Affordable Care Act, leaders are likely to be charged with implementing these quality improvement initiatives within a context of increasing emphasis on resource efficiency. While the oppor- tunities to improve may be tremendous, threading the value needle will likely test the mettle of all leaders of quality efforts in the years to come.
In preparation for this new era for the quality professional, the National Association of Healthcare Quality (NAHQ) began an initiative to investi- gate the leadership development needs ofthe profession. Their efforts yielded a competency model that is specific to leadership in quality and holds implica- tions for professionals across the career path. Development and validation of this model are described in the follow- ing section.
METHOD Development of the competency model began in lune 2008. Members ofthe NAHQ board agreed to serve as the advisory panel for developing a leader- ship model and convened a two-day
series of meetings to develop the draft. The meetings proceeded through three phases: clarification of goals, definition of scope, and competency identifica- tion. In the competency identification phase, subcommittees were formed to represent the perspectives of present- state, future-state, and senior leadership. Using the Health Administrators Leader- ship model (HAL) (Carman, Tyler, and Darnall 2003) as their seed model, the subcommittees reviewed this and other published models against the goals and scope criteria developed during the phase one and two meetings. These sep- arate reviews were then compiled during a large group meeting to form the first draft. These results were summarized overnight and disseminated the follow- ing morning for further discussion. The revised model contained 21 competen- cies organized into six domains. This model was circulated to the commit- tee a third time several weeks after the original meeting, and feedback was col- lected via teleconference. At that point the board agreed by consensus that the model reñected the elements they believed would be essential to quality leadership moving forward.
With the draft of the model final- ized, the next step was to conduct a con- struct validation study. To accomplish this, an electronic survey was developed for distribution to quality professionals, using methods adapted from Williams and Crafts (1997). The survey asked respondents to review each of the com- petency descriptions and rate it accord- ing to level of perceived importance to their quality leadership role using a five-point scale of relative importance ( 1 = No importance, 5 = Extremely
374
LEADING FOR QUALITY: DEVELOPMENT AND VALIDATION OF A COMPETENCY MODEL
important). An open-ended question followed each competency, to allow respondents to comment on clarity, word choice, or other concerns with the way the competency was defined. Respondents were also asked to answer a set of demographic questions concern- ing the nature of their positions and their institutional settings.
Invitations to complete the survey were sent to all members of NAHQ (n = 4,445), using a list maintained by this organization and a list of non- NAHQ members holding the Certified Professional in Healthcare Quality (CPHQ) credential (n = 2,704), which was provided by the Healthcare Quality Certification Board.
RESULTS A total of 883 quality professionals responded to the survey, for a response rate of 12 percent. Respondent demo- graphics are provided in Exhibit 1. For respondent role, four categories were created to collapse roles to a manage- able number: senior executive leader- ship (CEO, president, chief of staff, chief or VP of medical affairs, chief nursing executive, chief operating or operations officer, chief financial officer); senior quality leadership (president or vice president for quality services, quality improvement, patient safety divisions/ departments, physician quality leader (department or service line); mid-level quality leadership (director or manager of quality, patient safety, and/or risk management); and direct contributor (quality coordinator, quality specialist, data analyst, data abstractor).
To determine the validity of the model and its component competencies.
a series of content validity ratios (CVR) (Lawshe 1975) were calculated. CVRs indicate the proportion of respondents who agree that a particular competency is very or extremely important. This analysis indicated that all competencies were above the recommended threshold value of 0.49 (range: 0.66 for "Finan- cial Acumen" to 0.97 for "Professional Ethics").
Model Structure The association between competen- cies and domains was examined using principal components factor analysis, a statistical technique that tests the extent to which items on a survey tend to be rated similarly by respondents (higher levels of association suggest that the items hang together more closely, form- ing associations). To test whether the domains originally defined would map to the competencies we originally speci- fied, the analysis was set up to fit the data to a six-domain solution. Results of this analysis, shown in Exhibit 2, supported the original structure for 17 of the 21 competencies and identified four areas in need of reconciliation. First, the systems thinking competency, which was originally in the organiza- tional awareness domain, loaded more heavily onto the fosters positive change domain. Second, the lifelong learning competency, originally in the profes- sionalism domain, loaded relatively equally onto the self-management and professionalism domains and slightly higher onto self-management. Third, the professional ethics competency, which was originally under the pro- fessionalism domain, loaded more strongly onto the self-management
375
JOURNAL OF HEALTHCARE MANAGEMENT 5 6 : 6 N O V E M B E R / D E C E M B E R 2 0 1 1
E X H I B I T 1 Respondent Demographics
Professional
NAHQ member
Certified professional in Health Quality (CPHQ)
Both
Years working in quality
Less than 1 year
1-2 years
3-5 years
6-10 years
11-15 years
More than 15 years
Hospital bed size (hospital-based professionals)
Less than 50
50-99
100-199
200-299
300-399
400-499
500 or more
Job level (by categorizable role, respondents in hospital settings)
Senior executive leadership
Senior quality leadership
Mid-level quality leadership
Direct contributor level
N
254
160
469
14
46
117
169
137
400
604
61
53
110
102
83
46
149
521
14
44
251
212
(%)
(29)
(18)
(53)
(2)
(5) (13)
(19)
(16)
(45)
(68)
(10)
(9)
(18)
(17)
(14)
(8)
(25)
(59)
(3)
(8) (48)
(41)
domain. Fourth, drive for results, originally under the passion for positive change domain, loaded equally strongly onto two other domains: performance improvement and self-management. These four findings were vetted with the original advisory panel, who by consen- sus agreed that the survey results made conceptual sense and should be used to guide revision ofthe model. These final
revisions yielded the model shown in Exhibit 3.
Subgroup Analyses Our next step was to develop a more refined understanding of how the competencies may differ in their rela- tive importance as a function of orga- nization setting, organization size, experience level, and position level.
376
LEADING FOR QUALITY: DEVELOPMENT AND VALIDATION OF A COMPETENCY MODEL
CO
CO
c:
0 1
U
'u c
O ai
u CO
09 ' S
i E E S 3 S £ E
o £ 01
u CO
.| s ' E CO CO a
o CL O O
eo c o • o e¡ u> CO
I !E M 01
£ g"
d 00 r̂ m Ö Ö Ó
0 0 CO
d va (N d
00 CO
d
1-H
ro d d
00 va d
va d
CO
d
tN fO (N
i n (N (N u-i fs. va va • ^
d d d d
CO
d
CO ( N • *
00 00 va Ö Ö Ö
(N CO
CN
d
<N i n 0 0 CO 00 f>. iri i n
Ö Ö Ö Ö CO
d <N d
oCO d
CO v a v a 0 0 »—1
Ö Ö Ö. d Ö (N CN c\CN
CO (N
CO (N mCN
o o
OQ
a. ro
-O ro -a ro
S
I
ro
oo.a
00 ro
u
i ° 2 '-3P ro
113 - 3 32 •= ro ë
U J^
« i ' i ûj ro
> -P
£
ro XI
'S 00
• T 3
I
I ^ I .S c S I 1 g 2 Ê 'S g c c p _ >2 r̂ " ^ ^ C Q.
00
J 3 _̂ c .-¿'S -3 "ob '5b 'O y g Si A¿ C
" ig
3 -̂
00 vua» C 00 a> ro -^
IC/5 C/0 tL, I I io
va CN
00
ro
E o
i n (N
(U
s c
T3
00
I C
u S g 1
a o
I D
377
JOURNAL OF HEALTHCARE MANAGEMENT 5 6 : 6 N O V E M B E R / D E C E M B E R 2 0 1 1
E X H I B I T 3 Leadership in Quality: Final Model Structure and Associations with Respondent Demographics
Significant Associations
Domains/ Drganization Interaction Competencies Experience Joh Level Size Terms
I. Fosters positive change * * * * * *
Advocates and adapts to change
Partners for change * * * *
Cultivates a quality-supportive climate * *
Drives for results * * * * * *
JJ. Communicating
Verbal communication skills
Written communication skills
Listening and receiving feedback * *
Educating
JJJ. Qrganizational awareness * * * * * *
Strategic planning * * * *
Strategic thinking and alignment ** ** ** **
Financial acumen * *
Systems thinking * * * * * *
IV. Self-management * *
Professional ethics
Manages personal limits
Resilience & self-restraint
V. Professionalism / Professional values
Consumer advocacy
Future focus * * * *
Lifelong learning
VJ. Performance improvement
Managing data
Analytic thinking / knowledge-based * * * * * *
decision-making
Develops a knowledge-rich environment
Note: Complete and current descriptions of each ofthe competencies are available online: www.nahq.org/membership/ leadership/devmodel.html
* * Group differences statistically significant at p < 0.05
378
LEADING FOR QUALITY: DEVELOPMENT AND VALIDATION OF A COMPETENCY MODEL
For organization setting, although the survey contained a question about primary practice setting, within-category response counts were too small for meaningful separate analysis of these results. For this reason, we focused only on respondents who provided a bed count for their organizations in the demographics section of the sur- vey. A total of 267 (31%) respondents indicated "not applicable" and were categorized as nonhospital settings. The remaining 69 percent were cat- egorized as hospital settings. A series of statistical analyses (ANOVAs) was conducted to determine whether there were differences between hospital and nonhospital groups on the importance ofthe competencies and competency domains. These analyses yielded no sta- tistically significant differences between these groups, providing some evidence for generalizability across organiza- tional settings.
To examine the influence of orga- nization size and experience level, we followed a similar approach. For organi- zation size, we focused only on hospi- tal settings and only considered those respondents who reported bed counts associated with their organizations. These analyses suggested significant differences as a function of organization size for four competencies within the organizational awareness and fosters positive change domains (strategic planning, strategic thinking and align- ment, partners for change, and drives for results), as shown in Exhibit 3. Level of experience was also significantly asso- ciated with five competencies (strate- gic thinking and alignment, financial acumen, systems thinking, analytic
thinking/knowledge-based decision making, and listening and receiving feedback). For all significant effects, the direction of the effect was for larger organizations and higher experience lev- els to be associated with higher levels of importance for the competency/domain.
To analyze competencies accord- ing to position level, subgroups were formed according to responses to the demographic question "The primary functional role (not necessarily the title) of my current position related to qual- ity or patient safety is: ." Responses were categorized into the four role levels described previously (senior execu- tive leadership, senior quality leader- ship, mid-level quality leadership, and direct contributor). For the competency analysis, respondents were selected only if they reported a bed count in the demographic section (indicating they worked in a hospital setting). This analysis yielded statistically significant associations for 5 ofthe 21 competen- cies: one from the revised organizational awareness domain, two from the revised fosters positive change domain, and one each from the performance improve- ment and professionalism/professional values domains. Those five competen- cies were strategic thinking and align- ment, drive for results, systems thinking, analytic thinking/knowledge-based decision making, and future focus. In all cases, higher organizational level was associated with greater perceived impor- tance of the competency, suggesting that these five competencies would be particularly appropriate foci for leader- ship development programs that center on career progression to higher organi- zational levels.
379
JOURNAL OF HEALTHCARE MANAGEMENT 5 6 : 6 N O V E M B E R / D E C E M B E R 2 0 1 1
D I S C U S S I O N A N D C O N C L U S I O N Several important limitations of this work are important to keep in mind when interpreting the results. First, although the sample size obtained for the content validity study was large, the small response rate means the responses could differ in important ways from the population from which they were drawn. Second, because the focus of this projert was on leadership in quality, the model should not be considered a com- plete description of all critical elements of a quality professional's job. Effective- ness in a quality leadership role will typically also require mastery of a con- siderable knowledge base and technical competencies that are beyond the scope ofthe present effort. Third, competency models are only as useful as the leader- ship-development efforts they support. When they are incorporated into devel- opment programs based on sound adult learning principles, competency models can be powerful facilitators of indi- vidual change, but by themselves these models do little to help people develop. Last, the effectiveness of even the most skilled quality professionals ultimately will be bounded by the level of collabo- ration they experience from the other leaders and clinicians they work with. Attention to leadership as a team sport and the need for a continuous learning focus will continue to be essential in supporting quality improvement gains in healthcare organizations.
These limitations notwithstanding, results of this study suggest that the Quality Leadership model and its com- ponent competencies can be considered content-valid descriptions ofthe areas
quality leaders need to master to be effective in their roles and thus represent a useful model for leadership develop- ment within the quality profession. This appears to hold true regardless of setting (hospital or other) or of other profes- sional chararteristics such as experience, job level, or organization size. Addition- ally, the relative importance of a num- ber ofthe competencies and domains increased as a flinrtion of higher posi- tion level and larger, more complex organizational settings, suggesting spe- cific areas that may be particularly useful foci for leadership-development efforts.
Based on the pattems of results described previously, we constructed the graphic model depicted in Exhibit 4. The pyramid shape captures the hier- archy of the competencies according to the analyses described in the prior sections. In particular, professional- ism/professional values appears as the base to indicate the critical role that this domain (and professional ethics in particular) plays across all quality posi- tions. The next level contains founda- tional skills that, while still relevant to all positions, start to look different at different organizational levels. The top level, which contains organizational awareness and fostering positive change, is the most closely associated with higher organizational levels.
Taken together, these competen- cies illustrate several important ways in which successful leadership of quality efforts may differ from a more general definition of leadership effectiveness. In comparison to other widely recog- nized healthcare leadership competency models (for a review, see Carman and Johnson 2006), there appears to be a
380
LEADING FOR QUALITY: DEVELOPMENT AND VALIDATION OF A COMPETENCY MODEL
E X H I B I T 4 Graphic Representation of the Quality Leadership Developmental Competency Modei
Professionalism / Professional Values
Senior-level roles
Roles at all levels
greater relative emphasis on developing and maintaining a culture of continuous process improvement (partnering for change, cultivating a quality-supportive climate) and the competencies necessary for the analytic work associated with data-driven decision making (manag- ing data, analytic thinking, developing a knowledge-rich environment). It is perhaps not surprising that both empha- ses map closely to areas focused on by the Malcolm Baldrige award criteria (National Institute of Standards and Technology 2010).
Conversely, competencies with an external focus (e.g., community relations, board relations) or longer- term focus (e.g., flindraising, talent development), or a focus on day-to- day operations (e.g., human resource management, information technology management) are absent or deempha- sized. So the model does appear to trade
a narrower focus for greater depth of focus on quality leadership specifically.
In terms of developing current and future quality leaders, competency mod- els can support these efforts in a variety of ways. Competency models are par- ticularly helpful for identifying areas in which further development may provide the greatest relative payoff. For example, the competency definitions can be used as a template for creating developmen- tal 360-degree feedback programs in preparation for on-the-job develop- ment. Similarly, the model can be used as a framework for a development planning discussion between leaders and their direct reports, using templates such as those provided by Carman and Dye (2009). Models that are adopted by a hospital or health system's senior leaders can be particularly useful for aligning the developmental agenda of the entire organization. For example.
381
J O U R N A L O F H E A L T H C A R E M A N A G E M E N T 5 6 : 6 N O V E M B E R / D E C E M B E R 2 0 1 1
an employee and organizational devel- opment department can cross-walk an organization-endorsed competency model against internally provided train- ing and development programs to better communicate these opportunities to employees and to identify potential gaps in their offerings. Additionally, assess- ments such as 360-degree feedback can be aggregated to the organization levei to identify and begin to address broader skill gaps.
In conclusion, this model appears relevant and usefijl for leaders and edu- cators interested in developing capacity in the area of quality leadership. In sup- port of open dissemination, and in the spirit of viewing quality as everyone's responsibility, all competency descrip- tions and supplementary material have been made available on the NAHQ website. Our hope is that the availability of this model will help organizations attend to the leadership development needs of their quality professionals.
R E F E R E N C E S Garman, A. N., and C. F. Dye. 2009. The
Healthcare C-Suite; Leadership Development at the Top. Chicago: Health Administration Press.
Garman, A. N., and M. P. Johnson. 2006.
"Leadership Competencies: An Introduc- tion." Journal of Healthcare Management 51 (1): 13-17.
Garman, A. N., A. S. McAlearney, J. Robbins, P. Song, M. McHugh, and M. 1. Harrison. 2011. "High-Performance Work Systems in Health Gare Management, Part 1: Development of an Evidence-Informed Model." Health Care Management Revietu 36 (3): 201-13. http://journals.lww.com/ hcmrjournal/Fulltext/2011/07000/High_ performance_work_systems_in_health_. care.l.aspx*.
Garman, A. N., J. L Tyler, and J. S. Darnall. 2003. "Development and Validation of a 360-Degree Feedback Instrument for Healthcare Administrators." Journal of Healthcare Management 49 (5): 311-25.
Lawshe, G. H. 1975. "A Quantitative Approach to Content Validity." Personnel Psychology 38 (4): 563-75.
National Institute of Standards and Technol- ogy. 2010. 2010-2011 Health Care Criteria for Performance Excellence. Accessed May 30, 2011. www.nist.gov/baldrige/pub lications/upload/2011_2012_Health_ Gare_Griteria.pdf.
Pronovost, P., B. Weast, B. Rosenstein, J. B. Sex- ton, G. G. Holzmueller, L. Paine, R. Davis, and R. Hava. 2005. "Implementing and Validating a Comprehensive Unit-Based Safety Program." Journal of Patient Safet}' 1 (1): 33-40.
Williams, K. M., and J. L. Crafts. 1997. "Induc- tive Job Analysis: The Job/Task Inventory Method." In Applied Measurement Methods in Industrial Psychology, edited by D. L. Whetzel and G. R. Wheaton, 51-88. Palo Alto, GA: Davies-Black Publishing.
P R A C T I T I O N E R A P P L I C A T I O
Janet Holdych, PharmD, CPHQ, director of quality. Catholic Healthcare West, San Francisco, California
T en years ago the Institute of Medicine's landmark report Crossing the QualityChasm called out the US healthcare delivery system for not consistently pro- viding high quality care to all people. The report concluded that fundamental changes were needed in order to provide safe, effective, patient-centered, timely, efficient, and equitable care (IOM 2001). Operationalizing this vision has led to
382
Copyright of Journal of Healthcare Management is the property of American College of Healthcare Executives
and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright
holder's express written permission. However, users may print, download, or email articles for individual use.