WK6 DQ 1
gascie4x2001
leadershiptrainingmeta-analysis2017AP.pdf
Leadership Training Design, Delivery, and Implementation: A Meta-Analysis
Christina N. Lacerenza, Denise L. Reyes, and Shannon L. Marlow
Rice University
Dana L. Joseph University of Central Florida
Eduardo Salas Rice University
Recent estimates suggest that although a majority of funds in organizational training budgets tend to be allocated to leadership training (Ho, 2016; O’Leonard, 2014), only a small minority of organizations believe their leadership training programs are highly effective (Schwartz, Bersin, & Pelster, 2014), calling into question the effectiveness of current leadership development initiatives. To help address this issue, this meta-analysis estimates the extent to which leadership training is effective and identifies the conditions under which these programs are most effective. In doing so, we estimate the effectiveness of leadership training across four criteria (reactions, learning, transfer, and results; Kirkpatrick, 1959) using only employee data and we examine 15 moderators of training design and delivery to determine which elements are associated with the most effective leadership training interventions. Data from 335 independent samples suggest that leadership training is substantially more effective than previously thought, leading to improvements in reactions (� � .63), learning (� � .73), transfer (� � .82), and results (� � .72), the strength of these effects differs based on various design, delivery, and implementation characteristics. Moderator analyses support the use of needs analysis, feedback, multiple delivery methods (especially practice), spaced training sessions, a location that is on-site, and face-to-face delivery that is not self-administered. Results also suggest that the content of training, attendance policy, and duration influence the effectiveness of the training program. Practical implications for training devel- opment and theoretical implications for leadership and training literatures are discussed.
Keywords: leadership training, leadership development, management, development, meta-analysis
Supplemental materials: http://dx.doi.org/10.1037/apl0000241.supp
“Leadership and learning are indispensable to each other” –(John F. Kennedy, 35th President of the United States).
In 2015, organizations in the United States spent an average of $1,252 per employee on training, and the largest share of this training budget was allocated to leadership training, making lead- ership the greatest training focus for today’s organizations (Ho, 2016). As such, leadership development is an essential strategic priority for organizations. Despite the number of organizations devoted to leadership training (e.g., Harvard Business Publishing
Corporate Learning, Dale Carnegie Training, Wilson Learning) and evidence suggesting that organizational funds spent on lead- ership training are increasing over time (Gibler, Carter, & Gold- smith, 2000), organizations continue to report a lack of leadership skills among their employees; only 13% of organizations believe they have done a quality job training their leaders (Schwartz et al., 2014). Similarly, some have pointed out a substantial leadership deficit (Leslie, 2009) and have noted that organizations are “. . . not developing enough leaders” and “. . . not equipping the leaders they are building with the critical capabilities and skills they need to succeed” (Schwartz et al., 2014, p. 26). This calls into question the general utility of current leadership development initiatives.
As Wakefield, Abbatiello, Agarwal, Pastakia, and van Berkel (2016) note “simply spending more money on leadership programs is unlikely to be enough. To deliver a superior return on investment (ROI), leadership spending must be far more focused on and targeted at what works . . . with a focus on evidence and results” (p. 32). In response to this call for a focused investigation of leadership training, the purpose of the current study is to provide scientists and practitioners with data-driven recommendations for effective leadership training programs that are based on a meta- analytic investigation of 335 leadership training evaluation studies. In doing so, we attempt to unpack the black box of what works in
This article was published Online First July 27, 2017. Christina N. Lacerenza, Denise L. Reyes, and Shannon L. Marlow,
Department of Psychology, Rice University; Dana L. Joseph, Department of Management, University of Central Florida; Eduardo Salas, Department of Psychology, Rice University.
This work was supported, in part, by research grants from the Ann and John Doerr Institute for New Leaders at Rice University. We also thank Fred Oswald for his helpful comments on an earlier version of this article.
Correspondence concerning this article should be addressed to Christina N. Lacerenza, who is now at the Leeds School of Business, University of Colorado Boulder, 995 Regent Dr, Boulder, CO 80309. E-mail: [email protected]
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
Journal of Applied Psychology © 2017 American Psychological Association 2017, Vol. 102, No. 12, 1686 –1718 0021-9010/17/$12.00 http://dx.doi.org/10.1037/apl0000241
1686
the design, delivery, and implementation of leadership training. That is, we attempt to answer the questions (a) How effective are leadership training programs? and (b) How should one design, deliver, and implement a leadership training program to maximize effectiveness?
The current study addresses these two questions by meta- analytically summarizing leadership training research. In our ex- amination of the factors that contribute to leadership training program effectiveness, we offer several contributions to the sci- ence of leadership development and training. First, the current study provides a meta-analytic estimate of the effectiveness of leadership training across a wide span of years (1951–2014) and organizations. We note that this literature has been previously meta-analyzed (i.e., Avolio, Reichard, Hannah, Walumbwa, & Chan, 2009; Burke & Day, 1986; Collins & Holton, 2004; Powell & Yalcin, 2010; Taylor, Russ-Eft, & Taylor, 2009b); however, the Burke and Day (1986) meta-analysis, which is arguably the most comprehensive meta-analytic investigation of leadership training to date, only included studies published through 1982, which excludes the majority of available leadership training studies. Relatedly, Collins and Holton (2004) only added 13 additional studies to Burke and Day’s (1986) meta-analytic database after conducting their own literature search. Powell and Yalcin’s (2010) meta-analytic investigation only included private sector employ- ees, thereby limiting the ability to generalize findings to other populations. Moreover, Avolio, Reichard, et al. (2009) meta- analysis was limited to 37 primary studies, and Taylor, Russ-Eft, and Taylor’s (2009b) meta-analysis only included studies that assessed training transfer.1 It is clear that a plethora of research within this area has yet to be meta-analyzed; thus, we have included more recent publications to obtain an accurate account of the current state of the field (i.e., our meta-analysis includes over three times the amount of primary studies than reported in the largest previously published meta-analysis).
Second, we empirically test moderators of leadership training that have yet to be investigated in order to identify characteristics of the most effective leadership training programs. Although ex- isting work has investigated as many as eight moderators of leadership training program effectiveness, the current study exam- ines 15 moderators of leadership training program effectiveness that will provide those who develop leadership training programs a comprehensive understanding of how to design, deliver, and implement effective programs.
Lastly, the current study makes use of updated meta-analytic techniques when combining across study designs to accommo- date different types of primary study designs (Morris & De- Shon, 2002). Such methods were not used in previous meta- analyses; past authors either excluded certain design types (Burke & Day, 1986), or conducted separate meta-analytic investigations for each design type (Avolio, Reichard, et al., 2009; Collins & Holton, 2001; Powell & Yalcin, 2010). One exception to this is the meta-analytic investigation by Taylor et al. (2009b); however, this study only evaluated training transfer and did not include trainee reactions, learning, or results as outcomes. Using these updated techniques results in a more accurate estimate of the effect of leadership training, and also allows for stronger causal inferences than typical, cross- sectional meta-analyses (because all the studies included in the
meta-analysis are either repeated measures or experimental designs).
Leadership Training Defined
To begin, we define leadership training programs as programs that have been systematically designed to enhance leader knowl- edge, skills, abilities, and other components (Day, 2000). Parallel to previous investigations, we include all forms of leader, mana- gerial, and supervisory training/development programs and/or workshops in our definition of leadership training programs (e.g., Burke & Day, 1986; Collins & Holton, 2004). Although we use the term “leadership training” as an umbrella term to refer to many forms of leader development/training, we discuss potential differ- ences among various forms of leadership training below.
Leadership training is traditionally focused on developing “. . . the collective capacity of organizational members to engage effec- tively in leadership roles and processes” (Day, 2000, p. 582). Roles refer to both formal and informal authority positions, and pro- cesses represent those that facilitate successful group and organi- zational performance (Day, 2000). Beyond this approach, recent research has begun to distinguish between leadership development and leader development (Day, 2000). Leader development repre- sents training initiatives aimed at individual-level concepts, whereas leadership development takes a more integrated approach that involves the interplay between leaders and followers and socially based concepts (Iles & Preece, 2006; Riggio, 2008). Although this difference is recognized, it is often the case that the terms are used interchangeably, and because of this, the current study incorporates leader and leadership training/development evaluation studies.
It is also important to address the distinction between manage- rial training and leadership development. Managerial training and development has been described as “the process by which people acquire various skills and knowledge that increases their effective- ness in a number of ways, which include leading and leadership, guiding, organizing, and influencing others to name a few” (Klein & Ziegert, 2004, p. 228). The objective of a managerial training or development program involves teaching or enhancing managerial skills with the purpose of improving job performance (Goldstein, 1980). Although, theoretically, there may be a distinction between managerial training and leadership training, the terms are often used interchangeably and in the current investigation, managerial training programs are included within our examination of leader- ship training programs. Executive coaching programs are also included in the current meta-analysis because these programs aid executives (who are leaders) in learning specific skills or behaviors (Witherspoon & White, 1996), which is consistent with our defi- nition of leadership training. In summary, the current study takes a similar approach to prior work by including managerial, execu- tive, leader, and leadership training/development programs in our meta-analytic summary.
1 See Table A in the online supplementary material for more information regarding previous meta-analyses.
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1687LEADERSHIP TRAINING: A META-ANALYSIS
The Design, Delivery, and Implementation of Leadership Training
According to Kirkpatrick (1959), when evaluating training ef- fectiveness, outcomes of training can be categorized into one of four criteria: reactions, learning, transfer, and results. This frame- work has been adopted in previous leadership training meta- analyses (e.g., Burke & Day, 1986) and other training meta- analyses (e.g., Arthur, Bennett, Edens, & Bell, 2003), and is used in the current study to evaluate training effectiveness. Below, we develop hypotheses based on the extant training, learning, and leadership literature.
Reactions
Reactions reflect the attitudinal component of effectiveness and consist of trainee attitudes toward the training (e.g., training utility and satisfaction with the training/instructors). As an example, Kohn and Parker (1972) evaluated the reactions of trainees to a management development meeting by asking trainees to rate the extent to which they felt the meeting was of value. According to Patel (2010), 91% of organizational training evaluations collect reaction data, although this is not necessarily reported in published literature nearly as often as it is used in practice. In addition to the popularity of reaction data, this evaluation technique is important to consider when evaluating training effectiveness because it can be a precursor to other desired training outcomes (Hughes et al., 2016; Sitzmann, Brown, Casper, Ely, & Zimmerman, 2008). Ac- cording to social learning theory (Bandura, 1977; Bandura & Wood, 1989), an individual must be motivated to learn for actual learning to occur, and trainee reactions may serve as an indicator of motivation (i.e., if a trainee does not find a program to be useful, s/he may not be motivated to learn). Similarly, Hughes et al. (2016) tested a sequential model of general training outcomes using meta-analytic data and found support that reactions set the stage for more distal outcomes (i.e., transfer, results). Therefore, reactions may be an important component of a training evaluation because they signal trainee satisfaction, serve as indicators of trainee motivation to learn, and can lead to additional outcomes.
Given the popularity and importance of trainee reactions, it is critical to evaluate whether leadership training elicits positive changes in employee reactions (i.e., Does leadership training im- prove trainees’ perceptions of satisfaction and utility?). Although the idea that employees typically dislike training has been preva- lent in popular media (e.g., Kelly, 2012), training literature sug- gests training generally produces positive reactions (e.g., Brown, 2005) that may stem from employees perceiving training as a form of organizational support. Sitzmann, Brown, Casper, Ely, and Zimmerman (2008) suggest that this may translate into motivation and interest such that employees who perceive a high degree of organizational support will exhibit increased motivation and inter- est in training, as they believe the organization will subsequently provide the support they need to apply training to the job. There- fore, we argue that although employees’ pretraining perceptions of training utility and satisfaction may be lower given that employees tend to think they will dislike training (Kelly, 2012), these per- ceptions will increase during training because of the organizational support that is reflected in training, resulting in positive prepost change in training reactions. Although primary studies have begun
to investigate the extent to which leadership training results in positive trainee reactions, meta-analytic work has yet to provide an estimate of this effect. As such, we examine this in the current effort and hypothesize:
Hypothesis 1a: Leadership training programs have a positive effect on trainee reactions.
Learning
Learning is “a relatively permanent change in knowledge or skill produced by experience” (Weiss, 1990, p. 172) and it repre- sents what trainees can do following training. According to Kraiger, Ford, and Salas (1993), learning outcomes can be cate- gorized as affective-, cognitive-, or skill-based. Affective learning reflects the acquisition or change in internally based states. Cog- nitive learning reflects a developmental change in intellectual or mental-based skills. Skill-based, or psychomotor learning, refers to the acquisition of technical or motor-skills.
By definition, leadership development programs are designed to produce changes in the ability of trainees to engage in leadership roles and processes by presenting new information (Day, 2000). According to adult learning theory, knowledge acquisition and learning during training may occur because training transforms preexisting schemas, or mental constructions of the world, and challenges assumptions (Mezirow & Taylor, 2009; see also Chen, 2014). For example, the leadership training program described in a study conducted by Unsworth and Mason (2012) involved iden- tifying dysfunctional thinking biases; this strategy encouraged leaders to use more constructive thinking patterns to contend with management challenges and was ultimately found to enhance learning. Given the prior meta-analytic evidence supporting learn- ing as an outcome of leadership training (i.e., Burke & Day, 1986; Collins & Holton, 2001; Powell & Yalcin, 2010), we hypothesize:
Hypothesis 1b: Leadership training programs have a positive effect on affective-, cognitive-, and skill-based learning outcomes.
Transfer
Transfer (behavior) outcomes represent what the trainee will do, and can be conceptualized as the extent to which trainees utilize the skills and abilities taught during training on-the-job, including job performance (Alliger et al., 1997; Baldwin & Ford, 1988; Kirkpatrick, 1959). For instance, Russell, Wexley, and Hunter (1984) evaluated the transfer of trained behaviors in supervisors by collecting ratings on a behaviorally anchored rating scale that involved the quality of the supervisor’s work and how organized s/he was. An obvious primary goal of leadership training is to create a positive behavioral change in leaders on-the-job (Day, 2000). As such, transfer evaluation is critical for the assessment of leadership training effectiveness. Interestingly, some scholars have identified a “transfer problem” (Baldwin & Ford, 1988, p. 63) which refers to the tendency for targeted behaviors to fail to transfer to the work environment (Goldstein, 1986). Indeed, some studies have found that training in and of itself does not inherently lead to transfer (e.g., May & Kahnweiler, 2000). However, some degree of transfer is generally expected to occur as a function of training, and the extent to which trained behavior fully transfers to
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1688 LACERENZA, REYES, MARLOW, JOSEPH, AND SALAS
on-the-job behaviors is argued to be contingent upon various training factors (e.g., the moderators discussed below; Baldwin & Ford, 1988). Moreover, empirical and meta-analytic evidence in- dicates that leadership training does, to some extent, generally evoke transfer of training (Avolio, Rotundo, & Walumbwa, 2009; Burke & Day, 1986). In line with this research, we hypothesize that:
Hypothesis 1c: Leadership training programs lead to the trans- fer of trained affective-, cognitive-, and skill-based concepts.
Results
According to Kirkpatrick (1959), results are evaluative methods that reflect the training program’s effect on achieving organiza- tional objectives, including costs, company profits, turnover, and absenteeism. Results are also often defined in terms of the benefit of the training compared to the program cost (e.g., ROI; Arthur et al., 2003). For the current investigation, we categorize results as either organizational outcomes or subordinate outcomes. For ex- ample, DiPietro (2006) analyzed the ROI of a leadership training program, which is an organizational result, whereas Kawakami, Takao, Kobayashi, and Tsutsumi (2006) evaluated the degree to which subordinates perceived the work environment as supportive following the implementation of leadership training, which is a subordinate result.
It is important to note that results represent the most distal of the Kirkpatrick (1959, 1994) criteria and it has been suggested that “most training efforts are incapable of directly affecting results level criteria” (Alliger et al., 1997, p. 6). Some studies have, in accordance with this suggestion, found no improvement in results criteria following leadership training. For instance, Lee and col- leagues (2010) found that the self-reported emotional exhaustion of subordinates did not change following leadership training. Yet, this study is in the minority, as meta-analytic evidence indicates that leadership training has a positive effect on results (Burke & Day, 1986). Theoretically, scholars have long suggested that re- sults are a by-product of improvements in learning and transfer (Kirkpatrick, 1959; Tharenou, Saks, & Moore, 2007; Wright, McCormick, Sherman, & McMahan, 1999) because positive changes in employee knowledge and behavior may trickle-down to affect subordinate performance and/or trickle-up to change orga- nizational norms (e.g., a sales leader who undergoes training may increase his or her subordinate’s performance and may provide other leaders with normative examples of effective performance behaviors to cause revenue increases in other sales leaders as well). Therefore, given our expectation that learning and transfer occur as a result of leadership training, we expect results to improve after training.
Hypothesis 1d: Leadership training programs positively influ- ence organizational and subordinate outcomes.
Training Design, Delivery, and Implementation: Moderator Analyses
In 2010, Barling, Christie, and Hoption (2010) called for a rapprochement between the training and leadership development sciences. They noted that a lost opportunity for leadership devel- opment practitioners and scientists involves advancements within
the training domain that are not necessarily implemented within leadership literature and practice. The current study responds to this call by drawing on the sciences of learning and training to aid in the explanation of leadership training effectiveness. Below, we identify several design, delivery, and implementation features that have strong theoretical and empirical support as moderators of training effectiveness.
Training Design Characteristics
Needs analysis. A needs analysis is the process of identifying organizational, group, or individual training needs and aligning a program with these needs (Arthur et al., 2003). By conducting a thorough needs analysis, developers are better able to provide trainees with a program that parallels their training needs, thereby increasing the appeal of the training to the trainee and subse- quently enhancing results. However, training developers may ne- glect to conduct a needs analysis because they feel as if it is a waste of time or that it will not reveal any new information (Goldstein & Ford, 2002). For example, the U.S. Merit Systems Protection Board (2015) reported that only 16 out of 23 surveyed federal government agencies conducted a needs analysis for the training and development of their senior executives.
Despite the infrequent use of needs analyses, the benefits of conducting a needs analysis have long been discussed within the training literature (e.g., Goldstein & Ford, 2002). Collins and Holton (2004) mention that a lack of a needs analysis can lead to a generic training program that may not be suitable for the orga- nization. For example, a training program might emphasize trans- actional leadership style behaviors, which may not fit an organi- zational culture that values a transformational leadership style. In such a situation, trainees may feel the training program is not relevant to their job (i.e., reduced reactions), and subsequently, they may be less motivated to learn and transfer the training to the job. Although no previous meta-analysis has examined the use of a needs analysis as a moderator of leadership training effective- ness, given the aforementioned theoretical support for needs anal- yses, we hypothesize that:
Hypothesis 2: Leadership training programs that are based on a needs analysis exhibit greater improvements in trainee re- actions (H2a), learning (H2b), transfer (H2c), and results (H2d) than programs that are not based on a needs analysis.
Training attendance policy. Generic training literature indi- cates that trainees who exhibit high motivation and perceive value in the training program are more likely to implement trained concepts on-the-job (Blume, Ford, Baldwin, & Huang, 2010; Chia- buru & Marinova, 2005; Tziner, Fisher, Senior, & Weisberg, 2007), thereby increasing training utility and effectiveness. To increase trainees’ motivation to transfer, some researchers have suggested creating voluntary training programs (Curado, Hen- riques, & Ribeiro, 2015). For example, in a cross-sectional study conducted on employees within an insurance company, results suggested that voluntary training programs enhanced transfer mo- tivation to a greater degree than mandatory programs (Curado et al., 2015). These results may partially be explained by self- determination theory (Ryan & Deci, 2000), which suggests that autonomy fosters motivation; by providing trainees with the choice to participate in training, the need for autonomy is satisfied,
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1689LEADERSHIP TRAINING: A META-ANALYSIS
thereby increasing trainee motivation to learn and transfer trained concepts (Cohen, 1990). Despite the empirically validated benefits of voluntary training programs, some researchers argue that train- ing programs should be mandatory, thereby signaling to trainees that the training is valued by their organization (Salas, Tannen- baum, Kraiger, & Smith-Jentsch, 2012). Although attendance pol- icy has not been meta-analyzed in the leadership training literature, Blume, Ford, Baldwin, and Huang (2010) shed some light on this issue in the general training literature and found a positive meta- analytic correlation between transfer and voluntary attendance. The current study assesses these effects within the realm of lead- ership training, and we hypothesize:
Hypothesis 3: Voluntary leadership training programs en- hance trainee reactions (H3a), learning (H3b), transfer (H3c), and results (H3d) to a greater degree than involuntary programs.
Spacing effect. Cognitive load theory (CLT) is a learning efficiency theory (e.g., Paas, Renkl, & Sweller, 2004; Sweller, van Merrienboer, & Paas, 1998; van Merrienboer & Sweller, 2005) positing that learners have a finite working memory capacity, and once this is met, processing and learning abilities are hindered or lost entirely. If an excessive amount of information is presented to a learner, although the information may enter working memory, it may not be processed into long-term memory, thus inhibiting the learners’ ability to access the information in the future (van Mer- rienboer & Sweller, 2005). CLT highlights the need for training programs that are designed to reduce extraneous cognitive load while increasing learners’ ability to process salient information and still presenting all of the relevant information. One way to do so is to temporally space training sessions, a technique known as spacing (Hintzman, 1974). For example, evidence suggests infor- mation may be remembered at an increased rate (increasing learn- ing and transfer) if the stimulus presentation sessions are tempo- rally spaced rather than presented at once (Janiszewski, Noel, & Sawyer, 2003). The consensus of research from the generic train- ing literature shows that spaced training is superior to massed training (Lee & Genovese, 1988). Further, meta-analytic evidence also suggests that task performance is greater when individuals practice in spaced intervals as compared to a single massed prac- tice session (Donovan & Radosevich, 1999). The current meta- analysis is the first to present a direct evaluation of how spaced training sessions can affect leadership training program effective- ness. We hypothesize:
Hypothesis 4: Leadership training programs spanning multiple training sessions result in greater effects on reactions (H4a), learning (H4b), transfer (H4c), and results (H4d) in compar- ison with training programs with one massed training session.
Trainees’ level of leadership. Leadership training can be administered to low-, middle-, or high-level leaders. It is possible that the level of the leader can influence how receptive the indi- vidual is to training. Middle- and high-level leaders may be more resistant to change because they may feel that change is disruptive (Hall, 1986). Similarly, because they have higher status, these leaders might feel as if they do not require further development because they have already succeeded as a leader (Guinn, 1999). It has also been argued that leadership experience fosters leadership
skills (Arvey, Rotundo, Johnson, Zhang, & McGue, 2006; Arvey, Zhang, Avolio, & Krueger, 2007; Riggio, 2008); as such, it could be the case that low-level leaders who lack leadership experience enter training with fewer leadership skills, allowing greater room for improvement. Because of their reduced leadership skills, it might be easier to garner desired outcomes in low-level leaders, compared with high-level leaders who may experience a ceiling effect during leadership training. In line with this theory, Avolio, Reichard, et al.’s (2009) meta-analysis of leadership training con- ducted a post hoc analysis on leader level and found that leadership training had a greater effect on low-level leaders compared to middle- and high-level leaders. We aim to replicate this finding using Kirkpatrick’s (1959) evaluation criteria, and hypothesize:
Hypothesis 5: Leadership training programs administered to low-level leaders will exhibit greater effects on reactions (H5a), learning (H5b), transfer (H5c), and results (H5d) than programs administered to middle- or high-level leaders.
Training instructor. According to Kalinoski et al. (2013), the trainer’s background can influence trainee motivation such that a program with a trainer from the trainee’s organization (i.e., internal trainer) will result in increased levels of trainee motivation in comparison to a program with a trainer outside of the organization (i.e., external trainer), especially if the trainer is a direct manager of the trainee. When participating in a leadership training program that is facilitated by an internal trainer, trainees may perceive the organization’s support for the training to be greater because they have a dedicated person on staff who is responsible for the training program. Conversely, trainees participating in a leadership training program facilitated by an external trainer might also perceive the organization as valuing training because they have paid to bring in an expert (or paid to send the employee to a leadership center). Empirical support for the effectiveness of both internal (May & Dubois, 1963; Mccormick, 2000; Teckchandani & Schultz, 2014) and external (e.g., Alsamani, 1997; Culpin, Eichenberg, Hayward, & Abraham, 2014; Jorgensen & Els, 2013) instructors exists; thus, internal and external trainers may be equally effective.
On the contrary, self-administered leadership training programs might signify to trainees that the organization does not fully support their training because they might perceive that fewer resources are needed in comparison to training programs with an instructor. Because trainees are required to complete the leadership training on their own, they may be less motivated to exert effort as they might believe the training is not valued by the organization, leading to a reduction in positive training outcomes (Blume et al., 2010). As such, we hypothesize that:
Hypothesis 6: Self-administered leadership training programs exhibit weaker effects on reactions (H6a), learning (H6b), transfer (H6c), and results (H6d) than programs facilitated by an internal or external trainer.
Training Delivery and Implementation Characteristics
Delivery method. Training delivery methods can be catego- rized into three broad categories based on their purpose: (a) to deliver information (i.e., information-based); (b) to demonstrate skills and abilities being trained (i.e., demonstration-based); or (c) to offer practice opportunities (i.e., practice-based; Salas &
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1690 LACERENZA, REYES, MARLOW, JOSEPH, AND SALAS
Cannon-Bowers, 2000; Weaver, Rosen, Salas, Baum, & King, 2010). For example, lectures, presentations, and most text-based training materials are considered information-based methods, whereas demonstration-based methods provide trainees with either negative or positive examples of the trained competency via in- person, audio, video, or simulated mediums. Finally, practice- based methods include role-play, simulations, in-basket exercises, guided practice, and others.
Of the three methods, practice-based training methods are con- sidered to be the most critical when influencing training outcomes as they enable trainees to fully conceptualize the material and implement it within a realistic environment (Weaver et al., 2010). Not only does generic training literature support the use of prac- tice, but traditional learning theory also supports this delivery method. Specifically, constructivist learning theory (Piaget, 1952) suggests learning is enhanced when the learner develops construc- tions of the world through his or her experiences and reflects on these experiences (i.e., constructivism reflects learning by doing). For example, when leaders are provided with opportunities to practice certain leadership competencies, they are able to actively reflect on their leadership experience, encounter and solve prob- lems within the training environment, and participate in the learn- ing process, which accelerates the rate at which they learn from their experience (McCauley & Van Velsor, 2004).
Although training scientists tend to agree that practice-based methods are perhaps the most effective delivery method, it is important to note that meta-analytic research indicates information (e.g., lectures) can also be effective (Arthur et al., 2003). However, we argue training programs that solely involve practice-based methods are more effective than those that only utilize information-based methods because practice-based methods are more appropriate for leadership skills (e.g., given that leadership training often involves skills related to interacting with others, a training program that allows one to practice interacting with others should be the most effective; Adair, 1983; Wexley & Latham, 2002; see also Arthur et al., 2003). Supporting this notion, a meta-analysis conducted by Burke and Day (1986) found that the leadership training programs were more effective when they in- corporated lecture/discussion and practice/role play when predict- ing objective learning criteria in comparison to those incorporating lecture alone. In line with research and theory, we hypothesize that:
Hypothesis 7: Leadership training programs incorporating only a practice-based method lead to greater effects on trainee reactions (H7a), learning (H7b), transfer (H7c), and results (H7d), as compared with programs incorporating only information- or demonstration-based methods.
However, researchers have suggested that the most effective training programs incorporate all three delivery methods (e.g., Salas et al., 2012). We posit that any disadvantages associated with one particular training technique may be overcome by utilizing multiple techniques. For example, demonstration does not gener- ally provide a conceptual understanding of why certain behaviors are occurring whereas information can provide a foundation from which to understand demonstration (Salas & Cannon-Bowers, 2000), and practice allows for active experimental and reflection. Meta-analytic research investigating training effectiveness in gen-
eral has shown that most training programs incorporate multiple delivery methods and the effectiveness of training varies as a function of the training delivery method specified (Arthur et al., 2003). Thus, we hypothesize:
Hypothesis 8: Leadership training programs incorporating information-, demonstration-, and practice-based methods demonstrate greater effects on reactions (H8a), learning (H8b), transfer (H8c), and results (H8d) in comparison with programs implementing only one (e.g., information only) or two methods (e.g., demonstration and information).
Feedback. According to feedback theory, feedback outlines successes and failures and how to correct unsuccessful behavior (Kluger & DeNisi, 1996; Powers, 1973). Feedback is valuable to trainees because it allows them to gain insight into their current ability (Maurer, 2002), thereby signaling whether there is a dis- crepancy between actual and intended performance (Nadler, 1977). Additionally, feedback aids in learning and transfer because it encourages trainees to participate in metacognitive activities (i.e., planning, monitoring, and revising behavior; Brown, Brans- ford, Ferrara, & Campione, 1983) during training. Trainees that engage in such activities learn at an accelerated rate because they adjust their learning and behavior after determining problem areas, thereby leading to increased transfer (Ford, Smith, Weissbein, Gully, & Salas, 1998). Feedback also engenders affective re- sponses (Kluger & DeNisi, 1996), and the provision of feedback may increase trainees’ perceptions of utility, thereby increasing positive reactions toward the training program (Giangreco, Sebas- tiano, & Peccei, 2009). As an example of how feedback can enhance leadership training effectiveness, May and Kahnweiler (2000) incorporated feedback in their interpersonal skills training program for supervisors by asking participants to discuss their performance with a coach using a behavioral checklist after par- ticipating in a role-play exercise. Interestingly, although feedback is a popular training tool in leadership training programs, it has yet to be meta-analytically investigated within this area. The current study offers this investigation, hypothesizing:
Hypothesis 9: Leadership training programs reporting the use of feedback display a greater effect on trainee reactions (H9a), learning (H9b), transfer (H9c), and results (H9d), in compar- ison with programs that do not report the use of feedback.
Source of feedback. One approach to delivering feedback that is often utilized within leadership training programs is 360-degree feedback (Goldsmith, Lyons, & Freas, 2000). Compared with single-source feedback, 360-degree feedback involves the collec- tion of information about the focal individual from several sources (e.g., supervisors, subordinates, customers; Wexley & Latham, 2002). Typically, all sources complete the same development instrument in reference to the leader, and then the leader is pro- vided with a report summarizing the ratings. This process allows the leader to formulate comparisons among various rating sources, and provides the leader with a more holistic depiction of his or her areas for improvement because the results are not based on a single-source (Wexley & Latham, 2002) and because the results may be perceived as more reliable (Atkins & Wood, 2002; Gre- guras & Robie, 1998).
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1691LEADERSHIP TRAINING: A META-ANALYSIS
Although some researchers have raised questions about the validity of 360-degree feedback, believing that simply collecting feedback from multiple sources may not provide any additional information (Borman, 1998; DeNisi & Kluger, 2000), organiza- tions utilizing 360-degree feedback tend to report increases in productivity and favorable reactions (Hazucha, Hezlett, & Sch- neider, 1993; Wexley & Latham, 2002). Because this specific type of feedback is quite popular (around 90% of large organizations report the use of 360-degree feedback; ETS, 2012), it is important to investigate its effectiveness via meta-analysis (existing meta- analyses have yet to compare 360-degree feedback to single- source feedback). Thus, we hypothesize:
Hypothesis 10: Leadership training programs reporting the use of 360-degree feedback compared to single-source feedback, display a greater effect on trainee reactions (H10a), learning (H10b), transfer (H10c), and results (H10d).
Training location. According to Baldwin and Ford’s (1988) theory of identical elements, training transfer is maximized when training stimuli align with the actual work environment (Baldwin & Ford, 1988). This alignment can be assessed through fidelity, or the extent to which a training program accurately depicts reality by mirroring the real-world system (Alessi, 2000; Meyer, Wong, Timson, Perfect, & White, 2012). According to Rehmann, Mit- man, and Reynolds (1995), fidelity is composed of three parts: equipment (i.e., alignment between tools, technology, and other system features utilized in training and those used on-the-job), environment (i.e., replication of the actual task environment in regard to sensory information, motion cues, and other features within the training program), and psychological fidelity (i.e., the degree to which task cues and consequences mirror those experi- enced on-the-job), all of which are important for simulating a trainee’s job environment during a training program.
Within an on-site leadership training program, trainees are typ- ically immersed within an environment that is similar, if not identical, to their work environment; as such, on-site training programs display high equipment, environment, and psychological fidelity. For example, during on-the-job leadership training (an on-site leadership training method), leaders participate in training and practice trained concepts during normal working conditions and while working on required work tasks (House & Tosi, 1963; Santrić Milicevic, Bjegovic-Mikanovic, Terzic-Supić, & Vasic, 2011). In contrast, off-site leadership training programs are housed within a facility other than the trainee’s organization; as such, the level of equipment and environment fidelity is reduced, leading to a potential reduction in training outcomes. In addition, because on-site programs are conveniently located, we believe that stake- holders are more likely to be involved in on-site programs, which is posited to enhance motivation in both trainers and trainees (Salas et al., 2015). Further, on-site leadership training methods likely result in a greater ROI as compared with off-site training programs (Arthur et al., 2003; Avolio, Avey, & Quisenberry, 2010) because resource requirements associated with off-site train- ing programs may constrain implementation of the training (i.e., the expense of off-site programs can be substantial, forcing some to choose shorter and/or less comprehensive off-site programs). Although some researchers and practitioners might note that on- site work can distract leaders attending the program (e.g., trainees
might be more tempted to leave training for work emergencies) and trainees may not have the opportunity to learn from individ- uals outside of their organization during on-site training, the ben- efits previously mentioned likely outweigh these concerns.
Though it has been assumed that on-site training methods have the “ability to minimize costs, facilitate and enhance training transfer” (Arthur et al., 2003, p. 243), no existing meta-analytic evidence has tested this assumption. As such, some have called for research on the effectiveness of on-site training methods in com- parison with off-site training programs (e.g., Wexley & Latham, 1991). The current study responds to these calls, and we hypoth- esize:
Hypothesis 11: Programs hosted on-site display a greater effect on trainee reactions (H11a), learning (H11b), transfer (H11c), and results (H11d) compared with off-site programs.
Training setting. The setting of the leadership training pro- gram and whether it is face-to-face or virtually based might also play a key role in contributing to training effectiveness. We define virtually based training programs as interventions delivered via a computer or similar device with no facilitators physically present. To compare virtual and face-to-face training, we borrow theory from education on the advantages and disadvantages of e-learning (Garrison, 2011), which is quickly becoming a popular method of replacing traditional face-to-face instruction. E-learning is pur- ported to offer several advantages over face-to-face instruction such as the capacity to provide self-paced and learner-centered experiences and archival access to learning materials (Zhang, Zhao, Zhou, & Nunamaker, 2004). In spite of these advantages, e-learning is more likely to involve a lack of immediate feedback unless it is embedded within the training program, discomfort experienced by users with limited technology experience, and the potential for increased frustration, anxiety, and confusion as a result of poorly equipped e-learning systems (e.g., technology issues; Zhang et al., 2004). We suggest these same problems may also plague virtually based leadership training programs, making such programs less effective than face-to-face programs. We fur- ther posit that a lack of fidelity may also contribute to virtually based programs being less effective than face-to-face interven- tions. For example, the high fidelity of face-to-face programs can incorporate dyadic interactions such as role-play to demonstrate and practice interpersonal leadership skills, whereas virtual envi- ronments make interpersonal interaction difficult.
Magerko, Wray, Holt, and Stensrud (2005) also note that face- to-face training can allow trainers to oversee each trainee’s indi- vidual progress and adjust the learning experience as necessary. In contrast, virtual training programs are not typically designed in this manner (virtual training is more likely to involve asynchronous communication between the trainer and trainee), although recent initiatives are seeking to address this problem (Magerko et al., 2005). Because of this, we argue that trainees in a face-to-face environment may be more likely to receive an optimal level of training difficulty, as they have the benefit of interacting with a trainer who can modify the content or provide specific guidance that a virtual environment may lack. Although a meta-analysis has yet to formally compare virtual and face-to-face training environ- ments, in line with the above theory and evidence we hypothesize:
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1692 LACERENZA, REYES, MARLOW, JOSEPH, AND SALAS
Hypothesis 12: Face-to-face leadership training programs in- crease positive trainee reactions (H12a), learning (H12b), transfer (H12c), and results (H12d) to a greater degree than virtual programs.
Exploratory Moderators
There are several additional training design, delivery, and im- plementation features that might impact training effectiveness, despite a lack of theoretical support for how the moderator will affect program effectiveness. To provide scientists and practitio- ners with a robust meta-analytic investigation of leadership train- ing programs, we incorporate the following moderators within our exploratory analyses: (a) training content, (b) program evaluator’s affiliation, (c) training duration, and (d) publication date.
Training content. There are multiple ways to succeed as a leader, yet it is unclear which knowledge, skills, and abilities should be included in the content of a training program to maxi- mize leadership training effectiveness. In a meta-analysis of 70 leadership training programs conducted by Burke and Day (1986), training that involved human relations content produced the largest effect on learning, and self-awareness training content had the greatest effect on transfer criteria; however, these analyses were based on a small number of primary studies (k � 8 and 7, respectively), and only begin to scratch the surface as to what content should be incorporated within a leadership training pro- gram to maximize effects. The current study aims to examine the effect of training content in a greater pool of primary studies using Hogan and Warrenfeltz’s (2003) domain model of managerial education. This theoretical model organizes leadership competen- cies into four competency domains: intrapersonal skills, interper- sonal skills, leadership skills, and business skills. Ultimately, this exploratory analysis allows us to examine whether each type of training content is effective (e.g., Is interpersonal skills training effective?) and whether certain training content is more successful in producing improvements in reactions, learning, transfer, and results than other training content, giving decision makers knowl- edge of which type of training may be most valuable if a needs analysis does not specify a content focus.
Training evaluator affiliation. Additionally, we sought to identify whether evaluator affiliation (i.e., academic primary study authors only, practitioner authors only, or a mix of academics and practitioners) played a role in the effectiveness of the program. Although the author of a primary study is not always involved in the design, delivery, and evaluation of the training program, in many cases, the authors are involved, and thus, we examine whether a combination of academics and practitioners is more likely to produce highly effective training programs because of their diverse backgrounds that can rely on both science and prac- tice. This has yet to be investigated in the training literature and poses an interesting question, as well as a test of the revered scientist-practitioner model (Jones & Mehr, 2007).
Training duration. Researchers have suggested that perma- nent cognitive and schematic changes are needed in order for leaders to fully understand how to transform and develop their followers over time (Lord & Brown, 2004; Wofford, Goodwin, & Whittington, 1998), and this process takes time. Thus, longer training programs may be more effective. Contrastingly, longer leadership training programs may result in cognitive overload
(Paas, et al., 2004) and thus less effective training. Taylor et al.’s (2009b) meta-analysis of leadership training also found no support for length of training as a continuous moderator of training effectiveness (i.e., transfer). As such, the current effort will shed light on this issue by meta-analytically testing the impact of training duration on leadership training effectiveness.
Publication date. The design, delivery, and implementation of leadership training programs has most likely changed over the years due to technological advancements and the dynamic nature of organizations today. We investigate whether this has led to a change in the effectiveness of training. Although Powell and Yalcin (2010) concluded that leadership training programs evalu- ated in the 1960s and 1970s produced greater effect sizes than other decades, this finding has yet to be corroborated while exam- ining publication date as a continuous moderator using an updated meta-analytic database.
Method
Literature Search and Inclusion Criteria
Relevant empirical studies were extracted in several ways. First, literature searches in the databases PsycINFO (1887–December 2014), Business Source Premiere (1886–December 2014), and ProQuest Dissertations and Theses (1861–December 2014) were conducted. Although the search goes as far back as the late 1800s, the earliest usable study for the current meta-analysis was pub- lished in 1951. Published and unpublished studies were included in order to reduce the potential for publication bias. The following keywords were included in this initial search: leadership, leader, manag� (utilizing the asterisk within searches allows for extraction of all keywords beginning with the root, e.g., management, and manager), executive, supervisory, training, and development. Google Scholar was also searched using the same keywords, with the addition of method (e.g., leadership training AND method). Lastly, reference lists from previous meta-analyses on pertinent topics (Arthur et al., 2003; Avolio, Reichard, et al., 2009; Burke & Day, 1986; Collins & Holton, 2001; Keith & Frese, 2008; Powell & Yalcin, 2010; Taylor et al., 2009b) were cross-checked. This initial search returned over 20,742 articles.
Each study was reviewed and included if the following con- ditions were met: (a) it included an empirical evaluation of a leadership, leader, managerial, supervisory, or executive train- ing (development or coaching) program; (b) the study used a repeated measures design, independent groups design, or an independent groups design with repeated measures; (c) it in- volved an adult sample (i.e., over 18-years-old); (d) it was written in English; (e) it provided the sample size and enough information to calculate an effect size; and (f) the participants were employees (e.g., not MBAs/undergraduate students). Ap- plying the above criteria resulted in a final sample of 335 independent studies (N � 26,573).
Coding Procedures
Three of the authors coded the studies, and coding discrepancies were resolved through discussion after each article was indepen-
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1693LEADERSHIP TRAINING: A META-ANALYSIS
dently coded.2 Overall rater agreement was 96.2%. Studies were coded for sample size, evaluation measure utilized, reliability (i.e., Cronbach’s alpha), experimental design (i.e., repeated measures, same subjects are assessed before and after training; independent groups, an experimental and control group are measured; and independent groups with repeated measures, an experimental and control group are tested before and after training), and effect size. A Cohen’s d effect size was coded for each study, or the statistical information needed to compute a Cohen’s d effect size (e.g., means and standard deviations, t-value). If a primary study reported multiple, nonindependent effect sizes (e.g., two separate measures of transfer were used), they were combined using the intercorre- lations among the measures to create a linear composite (see the formula provided in Nunnally, 1978). If the information necessary for calculating a linear composite was not reported (i.e., the inter- correlations were not available), the effect sizes were averaged. The following definitions and descriptions were implemented when coding for additional moderators.
Evaluation criteria. To categorize effect sizes by training evaluation criteria, we followed the framework developed by Kirk- patrick (1959). Reactions represent trainees’ perception of the training program as related to its utility and general likability (Alliger et al., 1997; Kirkpatrick, 1959, 1996). Learning measures represent quantifiable evidence that trainees’ learned the knowl- edge, skills, and/or abilities represented during training (Alliger et al., 1997; Kirkpatrick, 1959, 1996). Transfer represents measures evaluating on-the-job behavior, as assessed by trainees, supervi- sors, peers, or subordinates (Alliger et al., 1997; Kirkpatrick, 1959, 1996). Both learning and transfer were further categorized as cognitive learning/transfer (i.e., verbal knowledge, knowledge or- ganization, and cognitive strategies), affective learning/transfer (i.e., attitudinal and motivational elements), or skill-based learn- ing/transfer (i.e., compilation elements and automaticity) based upon the framework identified by Kraiger et al. (1993). In addition to these three subcategories, transfer also included a job perfor- mance category reflecting studies evaluating training with regard to on-the-job performance. Results included changes in organiza- tional objectives because of the training program (Alliger et al., 1997; Kirkpatrick, 1959, 1996). Effect sizes were further catego- rized as organizational results (e.g., turnover, absenteeism, ROI, profit) or subordinate results (e.g., subordinate job satisfaction, performance ratings of the leader’s subordinates).
Delivery method. Training delivery method was classified as information-based (e.g., lectures, presentations, advanced organizers, text-based training materials), demonstration-based (e.g., case studies, in-person modeling, computer-generated avatars), practice-based (e.g., role-play, simulations, in-basket exercises; Salas & Cannon- Bowers, 2000; Weaver et al., 2010), or a combination of the above listed methods if more than one delivery method was implemented (i.e., information and demonstration, information and practice, dem- onstration and practice, or information, demonstration, and practice).
Training content. In order to determine the content trained within each leadership training program, we utilized the compe- tency domain model developed by Hogan and Warrenfeltz (2003). Each study was coded as either: intrapersonal (i.e., internal, indi- vidual behaviors such as coping with stress, goal-setting, time management), interpersonal (i.e., behaviors associated with build- ing relationships such as active listening and communication), leadership (i.e., behaviors concerned with team building, produc-
ing results, and influencing others), or business (i.e., behaviors related to content or technical skills such as analytical skills, financial savvy, decision-making, and strategic thinking).
Trainees’ level of leadership. We coded each primary study according to the trainees’ level of leadership. Specifically, each study was coded as either: low-level (i.e., those who interact directly with first-level employees), middle-level (i.e., managers in charge of multiple groups of subordinates and serve as a liaison between subordinate groups and other levels of the organization), or high-level (i.e., those that do not have any managers above them except for executives). This categorization scheme was based on that utilized by previous researchers (DeChurch, Hiller, Murase, Doty, & Salas, 2010).
Additional codes. Each primary study was also coded with regard to the presence of feedback, source of feedback (i.e., single-source feedback or 360-degree feedback), publication status (i.e., published or unpublished), training length (in hours), pres- ence of spacing effect (i.e., training sessions were temporally spaced daily/weekly/yearly/monthly, or massed), presence of a formal needs analysis, training setting (i.e., face-to-face or virtual), location (i.e., on-site or off-site; self-administered programs were not included in this moderator analysis), trainer’s background (i.e., internal, external, or self-administered), attendance policy (volun- tary or involuntary), label of training program (i.e., leadership training, leadership development, management training, manage- ment development, supervisor training, supervisor development, executive training, or executive development), evaluator affiliation (i.e., academic, practitioner, or a mix of academics and practitio- ners), and publication year.
Analyses
The current meta-analysis incorporated studies that had three experimental design types (i.e., repeated measures, independent groups, independent groups with repeated measures). Because estimated population parameters are dependent upon the design type of each study (Ray & Shadish, 1996), appropriate statistical adjustments were made when aggregating across effect sizes (Glass, McGaw, & Smith, 1981; Morris & DeShon, 1997; Morris & DeShon, 2002) using procedures outlined in Morris and DeShon (2002). This is warranted, in part, because the overall corrected meta-analytic effect sizes were not practically or significantly different across study designs (t � 0.33, p � .05; see Table 1). These procedures required the calculation of rpre.Post which is the correlation between pre- and posttraining scores. The majority of studies did not report this correlation; therefore, the inverse sam- pling error variance-weighted average rpre.Post across repeated measures studies was used, per recommendations by Morris and DeShon (2002). For the current meta-analytic investigation, r̄pre.Post � .52. After conversions, a repeated-measures d effect size was estimated by conducting a random effects meta-analysis that accounted for sampling error variance (Hedges & Olkin, 1985) and corrected for criterion-related unreliability (Hunter & Schmidt, 1990, 2004).
T tests of the mean effect sizes were used to compare moderator conditions, following Hunter and Schmidt (2004). The subgroup
2 Detailed coding information for each primary study is located in Table B of the online supplementary material.
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1694 LACERENZA, REYES, MARLOW, JOSEPH, AND SALAS
meta-analytic technique was also used to test for moderators (Arthur, Bennett, & Huffcutt, 2001), and a separate artifact distri- bution was applied for each criterion (i.e., reactions, learning, results, transfer, and overall). In line with recommendations from previous researchers, the meta-analytic effect sizes were corrected for unreliability using artifact distributions (Hunter & Schmidt, 2004); the artifact distributions were created from averaging the internal consistency estimates reported in the primary studies. The mean reliability for the overall analysis was .97, and the mean reliabilities for reaction, learning, transfer, and results were as follows: .92, .95, .96, and .93, respectively.
Results
Results are presented in Tables 1–7. Table 1 presents the overall meta-analytic d combined across evaluation criteria, and Tables 2, 3, 4, and 5 present specific outcome effect sizes (reactions, learn- ing, transfer, and results). Effect sizes are presented as both the corrected average � value (corrected for unreliability in the crite- rion and sampling error variance; Hunter & Schmidt, 2004) and the observed d value. To test for the statistical significance of the effect size, 95% confidence intervals were reported. Continuous
moderator analyses (i.e., training duration and publication year) are reported in Tables 6 and 7.
Multiple techniques were implemented to test for publication bias. First, we visually inspected a funnel plot for asymmetry constructed on the overall effect (Sterne, Becker, & Egger, 2005). Second, we conducted a trim and fill analysis based on procedures identified by Duval and Tweedie (2000); results from a fixed effects model on the overall effect suggest that zero studies were imputed to the left of the mean, suggesting that publication bias is not present. Lastly, a moderator analysis was conducted by com- paring published and unpublished studies (see Table 1), and results indicated that there were no significant differences between meta- analytic effect sizes, t � 1.08, p � .05. Taken together, these results suggest that there is no evidence of publication bias.
In support of Hypothesis 1, results suggest leadership training programs are effective. Specifically, the overall effect size was significant and positive (� � .76, 95% CI [.64, .89]). Comparing across criteria, the strongest effect was found for transfer (� � .82, 95% CI [.58, 1.06]), followed by learning (� � .73, 95% CI [.62, .85]), results (� � .72, 95% CI [.60, .84]), and reactions (� � .63, 95% CI [.12, 1.15]). However, these values were not significantly
Table 1 Meta-Analytic Results: Overall
Meta-analysis k N d � SD %Var
95% CI 80% CR
LL UL LL UL
Overall 335 26,573 .73 .76 1.17 .20 .64 .89 �.74 2.26 Published 214 17,219 .78 .81 1.40 .10 .63 1.00 �.98 2.60 Unpublished 121 9,354 .66 .69 .66 .96 .57 .80 �.16 1.53
Study design Repeated measures 208 18,182 .75 .78 1.19 .17 .62 .94 �.74 2.30 Independent groups 62 3,901 .74 .76 1.13 .26 .48 1.05 �.69 2.22 Independent groups and
Repeated measures 58 4,490 .46 .48 .90 .99 .25 .71 �.67 1.63
Note. k � number of independent studies; N � sample size; d � repeated measures Cohen’s d; � � Cohen’s d corrected for unreliability in the criterion; SD � corrected standard deviation; %Var � percent of variance accounted for by sampling error; CI � confidence interval; LL � lower limit; UL � upper limit; CR � credibility interval.
Table 2 Meta-Analytic Results: Reactions Criteria
Meta-analysis k N d � SD %Var
95% CI 80% CR
LL UL LL UL
Reactions 7 620 .58 .63 .73 1.05 .12 1.15 �.30 1.57 Training method
Information and practice 2 115 1.31 1.47 1.57 .40 �.58 3.52 �.54 3.48 Information, demonstration, and practice 2 257 .11 .12 .28 10.03 �.27 .50 �.24 .48
Feedback Feedback 3 145 1.02 1.13 1.28 .00 �.24 2.50 �.51 2.77 No feedback 4 475 .52 .57 .57 1.56 .04 1.09 �.17 1.30
Evaluator’s affiliation Academic 3 363 .16 .17 .32 8.21 �.18 .53 �.23 .58 Practitioner 2 118 .59 .65 .00 100.00 .64 .66 .65 .65 Mix 2 139 1.42 1.61 .73 3.08 .64 2.59 �.68 2.54
Note. k � number of independent studies; N � sample size; d � repeated measures Cohen’s d; � � Cohen’s d corrected for unreliability in the criterion; SD � corrected standard deviation; %Var � percent of variance accounted for by sampling error; CI � confidence interval; LL � lower limit; UL � upper limit; CR � credibility interval.
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1695LEADERSHIP TRAINING: A META-ANALYSIS
different from one another, indicating that leadership training programs tend to be equally effective across criteria. Positive effect sizes were also found across more specific evaluation cri- teria (i.e., cognitive learning/transfer, affective learning/transfer, skill-based learning/transfer, job performance, organizational re- sults, and subordinate results; see Tables 2–5). Upon comparing these more specific criteria, we found that training programs resulted in significantly greater cognitive learning gains (� � 1.05) than affective learning gains (� � .54, t � 3.91, p � .05) and skill-based learning outcomes (� � .51, t � 3.84, p � .05; other
comparisons among specific learning outcomes were not signifi- cant). For transfer of training, training programs resulted in sig- nificantly greater skill-based transfer outcomes (� � .89) than affective transfer gains (� � .24, t � 4.49, p � .05) and job performance gains (� � .56, t � 2.02, p � .05). Further, affective transfer (� � .24) was also significantly lower than cognitive transfer (� � .68, t � �2.70, p � .05) and job performance gains (� � 56, t � �3.26, p � .05).Training programs also displayed significantly greater improvements in organizational outcomes (� � .75) than subordinate outcomes (� � .22, t � 6.08, p � .05).
Table 3 Meta-Analytic Results: Learning Criteria
Meta-analysis k N d � SD %Var
95% CI 80% CR
LL UL LL UL
Learning 153 9,716 .69 .73 .74 .83 .62 .85 �.22 1.68 Affective learning 55 4,630 .51 .54 .39 4.16 .44 .65 .04 1.04 Cognitive learning 48 3,206 .99 1.05 .93 .00 .79 1.30 �.14 2.24 Skill-based learning 103 5,437 .49 .51 .72 2.25 .37 .65 �.41 1.43
Training method Information 21 1,568 .57 .60 .66 1.37 .31 .89 �.24 1.44 Practice 10 302 .27 .28 .13 47.21 .13 .44 .11 .45 Information and demonstration 5 177 1.07 1.14 .78 .10 .44 1.85 .14 2.15 Information and practice 46 2,164 .57 .60 .68 2.18 .40 .79 �.28 1.47 Information, demonstration, and practice 29 1,913 1.16 1.24 .74 .35 .97 1.51 .29 2.19
Feedback Feedback 44 2,437 .75 .79 .69 .78 .59 .99 �.09 1.67 No feedback 108 7,279 .68 .71 .75 .83 .57 .85 �.25 1.68 Single-source feedback 6 363 .63 .66 .28 6.53 .42 .91 .31 1.02 360-feedback 8 430 .54 .57 .49 3.48 .20 .93 �.06 1.20
Needs analysis Needs analysis 30 1,218 1.05 1.12 .98 .03 .76 1.47 �.14 2.38 No needs analysis 123 8,498 .64 .68 .68 1.15 .56 .80 �.19 1.54
Spacing effect Spaced 116 7,526 .70 .74 .76 .73 .61 .88 �.23 1.72 Massed 17 1,344 .86 .92 .74 1.16 .57 1.26 �.03 1.86
Setting Virtual 10 620 .52 .55 .31 7.45 .34 .76 .16 .94 Face-to-face 116 6,916 .74 .78 .82 .55 .63 .93 �.27 1.83
Location On-site 27 2,001 .95 1.01 1.09 .01 .62 1.41 �.38 2.40 Off-site 31 2,493 .80 .84 .39 1.07 .70 .98 .35 1.34
Attendance Voluntary 47 3,038 .65 .69 .64 1.29 .50 .87 �.13 1.50 Involuntary 25 2,323 .88 .94 .76 .10 .64 1.23 �.04 1.91
Leader level High-level 16 627 .49 .52 .72 2.91 .17 .87 �.40 1.45 Middle-level 32 1,784 .63 .67 .82 1.02 .39 .95 �.38 1.71 Low-level 25 1,692 .93 .99 .83 .04 .66 1.31 �.08 2.05
Evaluator’s affiliation Academic 106 6,692 .65 .68 .65 1.33 .56 .81 �.15 1.51 Practitioner 19 785 .25 .26 .62 5.84 �.02 .53 �.53 1.05 Mix 24 1,774 1.14 1.21 .90 .15 .86 1.56 .06 2.36
Content Intrapersonal 4 265 .69 .73 .18 8.21 .50 .95 .50 .95 Interpersonal 9 263 .36 .38 .29 17.92 .13 .64 .02 .75 Leadership 21 1,232 .62 .66 .57 2.03 .42 .90 �.06 1.38 Business 16 1,370 .91 .97 .34 .27 .79 1.14 .53 1.41
Trainer External 51 2,744 .70 .74 .55 1.57 .58 .89 .03 1.45 Internal 23 2,396 .92 .98 .98 .02 .60 1.36 �.27 2.23 Self-administered 9 604 .50 .53 .30 7.91 .31 .74 .15 .91
Note. k � number of independent studies; N � sample size; d � repeated measures Cohen’s d; � � Cohen’s d corrected for unreliability in the criterion; SD � corrected standard deviation; %Var � percent of variance accounted for by sampling error; CI � confidence interval; LL � lower limit; UL � upper limit; CR � credibility interval.
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1696 LACERENZA, REYES, MARLOW, JOSEPH, AND SALAS
Concerning Hypothesis 2, programs that included a needs anal- ysis displayed significantly stronger effects for both learning and transfer than those reporting no needs analyses (t � 2.57, p � .05; t � 16.14, p � .05, respectively). However, this pattern was not found for results, as there was no significant difference between programs that did and did not conduct a needs analysis, t � �1.59, p � .05, and this could not be tested for reactions due to a small amount of available primary studies.3 Thus, Hypothesis 2 was supported for two of the three outcomes investigated.
Hypothesis 3c was supported such that voluntary training pro- grams displayed a significantly stronger effect for transfer than involuntary programs, t � 6.26, p � .05. However, contrary to
3 Because there were so few studies involving reactions as a criterion, many moderators could not be tested on this criterion. In the analyses that follow, we only discuss moderators of reactions when there were enough primary studies available to conduct the moderator analysis.
Table 4 Meta-Analytic Results: Transfer Criteria
Meta-analysis k N d � SD %Var
95% CI 80% CR
LL UL LL UL
Transfer 190 12,124 .78 .82 1.75 .09 .58 1.06 �1.42 3.05 Affective transfer 34 1,463 .23 .24 .34 14.88 .11 .37 �.19 .67 Cognitive transfer 21 844 .65 .68 .75 .75 .36 1.00 �.27 1.64 Skill-based transfer 155 10,233 .85 .89 1.87 .04 .61 1.18 �1.50 3.28 Job performance 35 1,686 .53 .56 .48 4.09 .39 .73 �.06 1.17
Training method Information 23 1,648 .43 .45 1.06 .88 .03 .87 �.91 1.81 Practice 28 1,613 .37 .39 .41 7.11 .23 .55 �.13 .91 Information and practice 43 2,548 .41 .43 .39 6.61 .31 .56 �.07 .94 Demonstration and practice 4 270 .68 .71 .00 11.58 .52 .90 .71 .71 Information, demonstration, and practice 45 2,600 2.02 2.20 3.02 1.97 1.35 3.05 �1.66 6.07
Feedback Feedback 68 4,250 1.32 1.40 2.64 .13 .79 2.00 �1.98 4.78 No feedback 122 7,648 .48 .50 .51 1.90 .37 .63 �.43 1.41 Single-source feedback 7 265 .50 .52 .79 2.51 �.06 1.10 �.50 1.54 360-feedback 16 1,015 .24 .25 .27 15.84 .11 .40 �.09 .60
Needs analysis Needs analysis 34 1,628 3.02 3.51 3.60 11.63 2.34 4.68 �1.09 8.12 No needs analysis 156 10,496 .40 .42 .45 5.09 .34 .49 �.16 .99
Spacing effect Spaced 133 9,113 .87 .92 1.89 .03 .61 1.23 �1.50 3.33 Massed 24 1,149 .43 .45 .65 3.25 .18 .71 �.39 1.28
Setting Virtual 11 562 .21 .22 .12 27.79 .06 .37 .06 .37 Face-to-face 139 8,378 1.04 1.10 2.11 .00 .76 1.43 �1.59 3.80
Location On-site 38 2,490 .35 .37 .30 10.98 .26 .47 �.02 .76 Off-site 32 3,476 .51 .54 .57 1.61 .34 .73 �.19 1.26
Attendance Voluntary 43 2,688 1.99 2.17 3.14 1.57 1.27 3.08 �1.85 6.20 Involuntary 46 3,653 .36 .38 .66 2.22 .18 .57 �.48 1.21
Leader level High-level 19 995 .36 .37 .32 10.92 .21 .54 �.04 .78 Middle-level 44 2,413 .52 .54 1.20 .72 .20 .89 �1.00 2.08 Low-level 58 3,140 1.84 1.99 3.05 1.23 1.23 2.74 �1.91 5.89
Evaluator’s affiliation Academic 150 8,767 .45 .47 .59 3.21 .38 .56 �.28 1.22 Practitioner 11 697 1.28 1.35 2.07 .16 .17 2.54 �1.29 4.00 Mix 29 2,660 1.88 2.04 3.26 .72 .90 3.18 �2.14 6.21
Content Intrapersonal 6 231 .21 .21 .00 100.00 .09 .33 .21 .21 Interpersonal 10 279 .26 .28 .28 23.14 .04 .51 �.09 .64 Leadership 35 1,289 .53 .55 .65 3.22 .33 .77 �.28 1.38 Business 19 2,876 1.56 1.67 2.94 .17 .39 2.94 �2.10 5.44
Trainer External 70 4,678 .49 .52 .58 2.53 .38 .65 �.23 1.26 Internal 18 1,172 .43 .45 .54 3.63 .20 .70 �.24 1.13 Self-administered 10 534 .21 .22 .14 25.95 .05 .38 .03 .40
Note. k � number of independent studies; N � sample size; d � repeated measures Cohen’s d; � � Cohen’s d corrected for unreliability in the criterion; SD � corrected standard deviation; %Var � percent of variance accounted for by sampling error; CI � confidence interval; LL � lower limit; UL � upper limit; CR � credibility interval.
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1697LEADERSHIP TRAINING: A META-ANALYSIS
Hypothesis 3d, involuntary training programs exhibited a signifi- cantly stronger effect for results than voluntary programs, t � 5.30, p � .05. There was no significant difference between involuntary and voluntary training programs for learning, t � �1.44, p � .05.
Regarding Hypothesis 4, for results, training programs that spanned across multiple sessions displayed a significantly larger effect size than training programs including one massed training session, t � 5.26, p � .05. This same pattern was also found for transfer, t � 2.28, p � .05 but there was no significant difference for learning, t � �0.97, p � .05, providing support for Hypothesis 4c and 4d, but no support for Hypothesis 4b (Hypothesis 4a could not be investigated).
Our analysis for Hypothesis 5 indicates that training programs provided to a sample of low-level leaders were associated with significantly stronger transfer effect sizes than those provided to high-level leaders, t � 6.59, p � .05 or middle-level leaders, t � 4.11, p � .05. There was no significant difference in transfer between high-level leaders and middle-level leaders, t � �0.89, p � .05. Moreover, for learning, there were no significant differ- ences between high-level and middle-level leaders, t � �0.65, p � .05, high-level and low-level leaders, t � �1.90, p � .05, and middle-level and low-level leaders, t � �1.47, p � .05. This same pattern was found for results, as there were also no significant differences between high-level and middle-level leaders, t � 0.00,
Table 5 Meta-Analytic Results: Results Criteria
Meta-analysis k N d � SD %Var
95% CI 80% CR
LL UL LL UL
Results 78 11,640 .66 .72 .56 .76 .60 .84 .01 1.43 Organizational outcomes 53 10,466 .69 .75 .56 .51 .61 .89 .03 1.46 Subordinate outcomes 30 2,507 .21 .22 .27 11.19 .11 .34 �.13 .58
Training method Information 9 1,136 .10 .11 .00 67.39 .04 .18 .11 .11 Practice 11 688 .35 .38 .00 24.40 .25 .51 .38 .38 Information and practice 22 1,208 .55 .60 .63 2.20 .33 .86 �.21 1.40 Information, demonstration, and practice 17 1017 .42 .45 .58 3.71 .19 .72 �.28 1.19
Feedback Feedback 34 2,027 .76 .84 .73 .61 .60 1.07 �.09 1.77 No feedback 44 9,653 .64 .70 .52 .64 .55 .84 .04 1.36 Single-source feedback 10 364 .51 .56 .80 2.60 .08 1.03 �.47 1.58 360-feedback 7 276 .72 .79 1.06 .58 .04 1.54 �.57 2.15
Needs analysis Needs analysis 20 689 .40 .43 .75 3.81 .11 .76 �.52 1.39 No needs analysis 58 10,991 .67 .73 .54 .60 .60 .87 .04 1.43
Spacing effect Spaced 50 8,173 .45 .48 .42 2.46 .37 .60 �.05 1.02 Massed 10 335 .17 .18 .00 59.42 .05 .32 .18 .18
Setting Face-to-face 62 8,308 .43 .47 .41 3.23 .37 .56 �.05 .99
Location On-site 15 1,075 1.12 1.25 .76 .18 .88 1.61 .28 2.21 Off-site 13 1,028 .37 .40 .29 7.87 .21 .59 .03 .77
Attendance Voluntary 24 1,523 .48 .52 .54 3.22 .30 .74 �.17 1.21 Involuntary 12 2,846 1.24 1.39 .42 .78 1.16 1.62 .85 1.93
Leader level High-level 6 383 .34 .36 .31 8.75 .06 .67 �.03 .76 Middle-level 24 1,217 .33 .36 .62 4.15 .11 .61 �.43 1.15 Low-level 18 1,303 .27 .29 .00 53.60 .22 .36 .29 .29
Evaluator’s affiliation Academic 57 6,364 .94 1.04 .61 .03 .89 1.19 .26 1.83 Practitioner 6 4,211 .36 .39 .21 2.67 .23 .55 .11 .66 Mix 14 1,098 .53 .58 .36 4.76 .39 .78 .13 1.04
Content Intrapersonal 4 244 .62 .67 .00 100.00 .61 .74 .67 .67 Interpersonal 4 410 .37 .40 .09 31.85 .25 .55 .29 .50 Leadership 17 5,115 .45 .49 .40 1.51 .31 .66 �.03 1.00 Business 8 953 .16 .18 .10 23.75 .05 .30 .05 .30
Trainer External 37 3,186 .64 .69 .65 1.06 .49 .90 �.14 1.53 Internal 6 173 .63 .69 1.37 .64 �.45 1.82 �1.07 2.44 Self-administered 4 363 .48 .52 .21 8.74 .25 .79 .25 .79
Note. k � number of independent studies; N � sample size; d � repeated measures Cohen’s d; � � Cohen’s d corrected for unreliability in the criterion; SD � corrected standard deviation; %Var � percent of variance accounted for by sampling error; CI � confidence interval; LL � lower limit; UL � upper limit; CR � credibility interval.
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1698 LACERENZA, REYES, MARLOW, JOSEPH, AND SALAS
p � .05, high-level and low-level leaders, t � 0.56, p � .05, and middle-level and low-level leaders, t � 0.56, p � .05. Thus, support for Hypothesis 5 was found for one of the three outcomes investigated.
Our analysis for Hypothesis 6 indicates that programs delivered via internal instructors displayed significantly stronger effects for learning than those that were self-administered, t � 2.12, p � .05. However, there were no significant differences between programs delivered by external and internal trainers, t � �1.16, p � .05 or self-administered and externally delivered programs, t � �1.65, p � .05 for learning. Moreover, programs delivered by external instructors were associated with significantly stronger effects for transfer than self-administered programs, t � 3.72, p � .05 but there was no significant difference between programs which were self-administered and provided by internal trainers, t � �1.75, p � .05. There was also no significant difference in transfer between programs provided by external trainers and those pro- vided by internal trainers, t � 0.48, p � .05. Finally, regarding result outcomes there were no significant differences between self-administered training programs and those administered by an internal, t � �0.31, p � .05 or external, t � �1.14, p � .05 trainer. There was also no difference for results between programs delivered by external and internal trainers, t � 0.00, p � .05.
When testing Hypothesis 7, we found that programs utilizing information-based methods exhibited significantly stronger effects for learning than those using practice-based methods, t � 2.24, p � .05. The opposite pattern occurred for results, and significant differences were not found for transfer, t � 0.26, p � .05.
Results for Hypothesis 8 show that training programs incorpo- rating information, demonstration, and practice-based methods displayed a significantly larger effect size for learning than pro- grams incorporating only information, t � 3.15, p � .05 or practice-based methods, t � 8.32, p � .05. Similarly, programs utilizing all three delivery methods displayed stronger effects for
learning when compared with programs incorporating both infor- mation and practice-based methods, t � 3.93, p � .05. However, no significant differences in learning were found when programs incorporating information, demonstration, and practice were com- pared with those utilizing information and demonstration, t � 0.26, p � .05. For transfer, programs including information, demonstra- tion, and practice-based methods displayed significantly stronger effects when compared with programs incorporating only practice, t � 7.06, p � .05 and information-based methods, t � 4.22, p � .05. This same pattern was also found for transfer when programs using all three delivery methods were compared to programs incorporating combinations of information and practice-based methods, t � 7.16, p � .05 and demonstration and practice-based methods, t � 5.92, p � .05. For the results criterion, programs incorporating all three delivery methods displayed stronger effect sizes when compared to programs using only information-based methods, t � 2.52, p � .05. However, there was no significant difference between programs utilizing information, demonstration, and practice compared with interventions using information and practice for this outcome, t � �0.77, p � .05 as well as those using practice only, t � 0.50, p � .05.
When testing Hypothesis 9, we found programs that included feedback had a significantly stronger effect size for transfer than those with no feedback, t � 5.69, p � .05. However, there was no significant difference between interventions using feedback and those not incorporating feedback for reactions, t � 0.79, p � .05, learning, t � 0.56, p � .05 and results, t � 0.86, p � .05. Thus, Hypothesis 9 was only supported for one of the four outcomes. No support was found for Hypothesis 10, as there were no significant differences between programs using 360-degree feedback com- pared to programs using single source feedback for learning, t � 0.43, p � .05, transfer, t � 0.92, p � .05, or results, t � 0.50, p � .05.
Regarding Hypothesis 11, training programs conducted on-site displayed significantly stronger effects for results than programs completed off-site, t � 4.71, p � .05. However, this effect was not significant for learning, t � 0.80, p � .05 or transfer, t � �1.55, p � .05. Our analysis for Hypothesis 12 further indicated that face-to-face programs exhibited a significantly stronger effect size for transfer than virtual programs, t � �5.94, p � .05. However, there were no significant differences between face-to-face and virtual programs for learning, t � �1.83, p � .05 and this hypothesis could not be assessed for results or reactions results due to a low number of primary studies. Therefore, Hypothesis 11 and 12 received support for only one criterion.
Table 7 Meta-Analytic Regression Results: Publication Year
Variable k � R2
Reaction 7 .31 .10 Learning 150 �.01 .00 Transfer 190 �.00 .01 Results 76 .34� .12
Note. k � number of independent studies; � � standardized estimate; R2 � variance explained. � p � .05.
Table 6 Meta-Analytic Regression Results: Duration
Variable k � R2 �R2
Reactions Model 1: Duration 6 .79 .62 Model 2: Duration 6 34.95
Duration2 5 �34.16 .68 .06 Learning
Model 1: Duration 113 �.07 .01 Model 2: Duration 113 �.12
Duration2 113 .04 .01 .00 Transfer
Model 1: Duration 145 �.04 .00 Model 2: Duration 145 �.20
Duration2 145 .17 .00 .00 Results
Model 1: Duration 54 .32� .10 Model 2: Duration 54 2.58�
Duration2 54 �2.31� .33 .23 Model 3 (no outliers): Duration 52 .57
Duration2 52 �.02 .30 .20
Note. k � number of independent studies; � � standardized estimate; R2 � variance explained; �R2 � change in R2 above and beyond Model 1; Duration2 is the squared duration term. � p � .05.
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1699LEADERSHIP TRAINING: A META-ANALYSIS
Exploratory analyses were conducted to provide further insight into additional factors influencing leadership training effectiveness.4
When investigating training content, findings indicate that programs including certain competency domains produced significantly greater effects than others; however, all trained competencies were effective (i.e., exhibited significant, nonzero effect sizes). For learning, those incorporating business competencies displayed greater effects than interpersonal competencies, t � 3.31, p � .05 intrapersonal compe- tencies, t � 2.11, p � .05. Those with intrapersonal competencies were also greater than interpersonal competencies, t � 2.36, p � .05. All other content comparisons, in regard to learning, were not signif- icant. Similar findings were found for transfer. Programs that trained business competencies produced the greatest effects compared with intrapersonal, interpersonal, and leadership (t � 3.52, 3.17, 2.34, respectively, p � .05), while those incorporating leadership produced significantly greater effects than intrapersonal competencies, t � 3.26, p � .05. For results, programs with leadership content produced significantly greater effects than business content, t � 3.11, p � .05. Those including intrapersonal content were significantly more effec- tive than programs including interpersonal and business content, t � 5.73 and 13.06, p � .05, and programs including interpersonal content were more effective than business content, t � 3.87, p � .05.
Interestingly, results suggested that studies with both academic and practitioner evaluators produced the greatest learning improvements when compared to studies evaluated only by academics, t � 3.08, p � .05 or practitioners, t � 4.31, p � .05 (see Table 3). Larger learning effect sizes were also associated with solely academic evaluators as compared to practitioners, t � 2.60, p � .05. The same was found for transfer, as a mixed set of evaluators was associated with a signifi- cantly larger effect size than solely academic evaluators, t � 4.57, p � .05. However, for transfer, the difference between a mixed set of evaluators and practitioner evaluators was not significant, t � �0.75, p � .05 and neither was the difference between academic and prac- titioner evaluators, t � �1.85, p � .05. Studies with only academic evaluators displayed the largest effects for result outcomes when compared to practitioner evaluators, t � 5.47, p � .05 and a mix of evaluators, t � 3.58, p � .05 (see Table 5). The difference between a mix of evaluators and practitioner evaluators for results outcomes was not significant, t � �1.48, p � .05.
We also ran exploratory analyses for duration of training and publication year as continuous moderators, which are presented in Tables 6 and 7. Training duration exhibited a positive, significant relationship with results (ß � 0.32, SE � 0.00, t � 2.43, p � .02) and nonsignificant relationships with other criteria. To examine whether the relationship between training program length and each outcome is better explained as a curvilinear effect, we tested the curvilinear relationship between duration and each criterion. Re- sults indicated a significant contribution of the quadratic term over and above the linear term (ß � �2.31, R2 � 0.33, � R2 � 0.23, p � .01) for result criteria; however, after two influential outliers were removed, the curvilinear effect disappeared (ß � �0.02, R2 � 0.30, � R2 � 0.20, p � .05) indicating that longer programs are more effective. The curvilinear relationship was also nonsig- nificant for reactions, learning, and transfer. Publication year dis- played a positive, significant relationship with results (ß � 0.34, SE � 0.01, t � 3.14, p � .002), but there was no significant effect of publication year on any other criteria.
Discussion
The guiding purpose of the current study was to answer the following questions: (a) How effective are leadership training programs? and (b) How should one design, deliver, and implement a leadership training program to maximize effectiveness? To ac- complish this objective, we conducted a meta-analytic review of 335 leadership training evaluation studies and investigated whether the effectiveness of training (as measured by positive changes in reactions, learning, transfer, and results; Kirkpatrick, 1959) was significantly affected by the inclusion of several train- ing design, delivery, and implementation features. We elaborate on our findings by answering both questions below.
Question 1: How Effective Are Leadership Training Programs?
Overall, our findings are more optimistic about the effectiveness of leadership training than popular press articles that suggest most leadership training programs are minimally effective (Nelson, 2016) and that leadership cannot be taught because “managers are created, leaders are born” (Morgan, 2015). Specifically, the current results indicate that leadership training is effective in improving reactions (� � .63), learning (� � .73), transfer (� � .82), and results (� � .72). Moreover, this study is the first to demonstrate that leadership training improves trainee reactions, in contrast to the idea often perpetuated in the popular press that individuals generally dislike training (Kelly, 2012).
Interestingly, not only do the current results suggest inaccura- cies in some popular press conclusions about the effectiveness of leadership training, but our results also suggest that previous meta-analyses may have underestimated the effectiveness of lead- ership training. For example, the current study’s effect sizes are substantially larger than what was previously reported in Burke and Day (1986; �learning � .38; �transfer � .49; �results � .67), and although the current results are difficult to compare with meta- analytic effect sizes for learning and transfer provided by Collins and Holton (2004; �learning � .35–1.36; �transfer � .38 –1.01), Powell and Yalcin (2010; �learning � .37 – 1.32; �transfer � .30 – .63), and Taylor et al. (2009b; �transfer � .13–.64), because each
4 An additional exploratory analysis was conducted for program label to investigate whether the effectiveness of a training program is related to its label. Leadership researchers and practitioners tend to use several terms interchangeably when referring to leadership training programs (i.e., train- ing, development, leader, leadership, and managerial). Our exploratory analyses showed that the label given to the training program did not impact effectiveness in regard to learning. For transfer, programs labeled “train- ing” resulted in greater outcomes compared with those labeled “develop- ment” (t � 4.72, p � .05), while the opposite was found for results (t � �9.82, p � .05). To investigate this further, we conducted t-test comparisons on programs including “leadership,” “supervisory,” or “man- agement” in their program label. Most labels were not significantly differ- ent from one another except for the following: programs labeled “leader- ship development” produced significantly weaker transfer than those named “leadership training” (t � �4.88, p � .05) and “management development” labels were associated with significantly weaker effect sizes than those entitled “leadership training,” “supervisory training,” and “man- agement training” (t � �5.80, �4.96, �2.33, respectively; p � .05). Regarding results as a criterion, all labels displayed similar effect sizes except programs labeled “leadership development,” as programs with this label were significantly more effective than “leadership training” (t � 5.68, p � .05) and “management training” (t � 10.03, p � .05).
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1700 LACERENZA, REYES, MARLOW, JOSEPH, AND SALAS
of these presented a wide range of effect sizes depending on the study design or the rater source, our findings do suggest sub- stantially larger effect sizes for the results criterion than previ- ous meta-analyses (Collins & Holton, 2004: � � .39; Powell & Yalcin, 2010: � � .35). Moreover, the current results are generated from more than triple the amount of primary studies included in previous meta-analyses. In answer to the question posed above, our results not only suggest that leadership train- ing is effective, but they also suggest that leadership training is more effective than popular press and previous meta-analyses indicate.
The present effort also indicates that leadership training is effective at improving affective, cognitive, and skill-based out- comes, corroborating prior meta-analytic findings (Avolio, Reich- ard, et al., 2009). Similarly, results echoed prior work by indicating that affective outcomes are typically lower than cognitive and skill-based outcomes (i.e., affective learning; Avolio, Reichard, et al., 2009; Kalinoski et al., 2013). In most leadership training evaluation studies, the affective outcomes involve emotional states like anxiety (e.g., Eary, 1997). In this context, criteria such as anxiety might be difficult to change because the training content provided to trainees does not necessarily train the leader on how to reduce his or her anxiety, but instead, is designed to improve leadership behaviors (which ideally reduces one’s anxiety). For example, Eary (1997) exposed trainees to material related to mil- itary leadership, such as communication and team building, but included anxiety levels as an affective outcome. Perhaps if lead- ership training programs involved more content devoted to the improvement of affective criteria, these programs would be equally effective across affective, cognitive, and learning criteria. Future research should examine whether affective content can improve affective outcomes to a greater extent.
Results also indicate that both organizational and subordinate outcomes increase as a result of leadership training. Interestingly, however, our findings show that leadership training improved organizational outcomes to a greater degree than subordinate out- comes, suggesting that the trickle-up effect (i.e., leadership im- provements aggregate up to the organizational-level) is stronger than the trickle-down effect (i.e., leadership improvements flow down to subordinates). We find this surprising given that leader- ship is a dyadic interaction between leader and follower; thus, the direct network tie between a leader and a follower should naturally allow improvements in leadership to be relayed to followers, whereas the process by which leadership improvements influence organizational outcomes is not as direct (e.g., a leader who has taken transformational leadership training can use the training in daily, direct interactions with her subordinates, but the process via which this training influences ROI is not as direct). Therefore, future research could benefit from a stronger understanding of how to maximize subordinate outcomes of leadership training.
Question 2: How Should One Design, Deliver, and Implement a Leadership Training Program to Maximize Effectiveness?
To address this question, we evaluated 15 training design, delivery, and implementation features through a series of moder-
ator tests. These findings are discussed below along with theoret- ical implications and areas of focus for future research.
Does a needs analysis enhance the effectiveness of leadership training? Arthur, Bennett, Edens, and Bell’s (2003) meta- analysis of the general training literature did not find a clear pattern of results for studies incorporating a needs analysis; how- ever, the current study found support for known training recom- mendations (Salas et al., 2012; Tannenbaum, 2002) by showing that programs developed from a needs analysis result in greater transfer and learning. Not only does this support longstanding generic training theory and recommendations for training, but it also supports popularly held beliefs in organizations that leader- ship training is not one size fits all (Gurdjian, Halbeisen, & Lane, 2014). Thus, we recommend leadership training developers con- duct a needs analysis before designing a program as the current meta-analytic data suggests this benefits learning and transfer (and is no more/less effective than programs that have not conducted a needs analysis for the results criterion).
Are voluntary leadership training programs more effective than mandatory programs? Our findings suggest voluntary attendance may be a double-edged sword, whereby it increases transfer, yet decreases organizational results (there was no effect on learning). We offer one possible interpretation for these results: Whereas voluntary attendance may increase transfer because it increases trainee motivation, it may also result in lower attendance, and thus, a reduced number of trainees whose leadership improve- ments can influence higher-level, organizational results. When looking at the average sample size within our meta-analytic data- base, categorized by outcome type and attendance policy, we found some support for this explanation. The average sample size for mandatory programs evaluating organizational outcomes was 263 trainees but only 41 trainees for voluntary programs. There- fore, perhaps mandatory programs foster organizational outcomes to a greater degree because the number of participants is larger.
Are temporally spaced training sessions more effective than massed training sessions? Interestingly, the current results sug- gest that distributed leadership training sessions do not result in an increase in learning outcomes, despite the noted importance of this method across learning theories (Hintzman, 1974; Paas et al., 2004; Sweller et al., 1998). Learning seems to occur regardless of the timing of training, suggesting that leadership content can be cognitively retained following a massed session. However, transfer and results criteria did produce findings that varied across tempo- ral spacing; programs with multiple sessions spaced temporally resulted in greater transfer and results than massed training ses- sions. Therefore, although spacing did not directly impact learning in this context, it appears that spacing is nevertheless beneficial for the downstream outcomes of transfer and results. A number of leadership training programs hold spaced training sessions (e.g., the Rockwood Leadership Institute, National Seminars Training’s Management & Leadership Skills for First-Time Supervisors & Managers, Academy Leadership’s Leadership Boot Camp), and due to their popularity, we sought to determine whether the time interval between sessions affects outcomes (i.e., weekly vs. daily). We conducted an exploratory, follow-up analysis and found no significant differences regarding learning and transfer between programs with sessions spaced daily or weekly; however, pro- grams with sessions spanned weekly (� � 1.14) produced larger
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1701LEADERSHIP TRAINING: A META-ANALYSIS
effects in the results criterion compared to those spaced daily (� � .44; t � 3.97, p � .05).
Is leadership training more effective for low-level leaders than high-level leaders? Previous meta-analytic work found that leadership training results in greater gains for low-level lead- ers compared with middle- and high-level leaders (Avolio, Reich- ard, et al., 2009); however, in this previous meta-analysis, outcome criteria were combined into a single outcome variable, obscuring an understanding of whether this apparent ceiling effect for higher- level leaders exists for all training criteria. In the current study, our results demonstrate that this ceiling effect is present only for transfer. High-, middle-, and low-level leaders experienced similar learning gains and similar improvements in the results criterion, suggesting that higher-level leaders are as equally motivated to learn as lower-level leaders and equally likely to improve organi- zational and subordinate outcomes. Thus, the current study sug- gests that middle- and high-level leaders can benefit from training (i.e., they can acquire new knowledge, skills, and abilities from training and improve organizational and subordinate outcomes), despite recent arguments that propose the expenditure of resources on developing senior leaders is wasteful because they are set in their ways (Lipman, 2016). In contrast, our findings showed stark differences in the transfer criterion across leader level such that transfer effects were approximately four times weaker for high- level leaders, perhaps indicating that the greater experience of high-level leaders has led to entrenched behaviors that are difficult to change. Thus, our results also suggest that higher-level leaders may require training that specifically focuses on enhancing trans- fer. In summary, the current results appear to indicate that previous findings were not due to a ceiling effect (i.e., wherein high-level leaders have no room to improve) or a motivational deficit (i.e., wherein high-level leaders are not motivated to learn), but rather a “transfer problem” (Baldwin & Ford, 1988, p. 63) that is perhaps caused by well-established behaviors that are difficult to change on-the-job, although this needs to be confirmed with future re- search.
Are self-administered leadership training programs less ef- fective than those administered by someone from the trainee’s organization (i.e., internal), or by someone outside of the or- ganization (i.e., external)? The current results indicate that self- administered leadership training programs are less effective than training administered by an internal or external trainer. Although self-administered training programs still exhibited nonzero im- provements, because their effectiveness compared to other training programs was substantially reduced, we caution practitioners from using self-administered leadership training programs and we urge academics to examine why these programs are less effective (e.g., Do self-administered trainees perceive the training to be less valuable? Are these trainees subsequently less motivated?).
Are practice-based training delivery methods more effective than information-based and demonstration-based delivery methods? Does the use of multiple training delivery methods increase the effectiveness of leadership training programs? Supporting previous meta-analytic research (Burke & Day, 1986; Taylor et al., 2009b) and basic learning theories that include practice as an essential component of learning (Kolb, 1984; Piaget, 1952), practice-based methods tend to be more effective than other delivery methods (i.e., practice was more effective than other methods for results, and equally effective for transfer). Also par-
allel to recommendations, we found that programs incorporating multiple methods were significantly more effective than programs using either a single delivery method or two delivery methods (Salas et al., 2012). In fact, by adding one or two methods to a single delivery method, transfer can be increased by more than one standard deviation. These results were consistent across outcome criteria; therefore, we recommend training developers utilize prac- tice if only one delivery method can be used, but employ multiple delivery methods whenever possible.
Does feedback enhance the effectiveness of leadership train- ing? Is 360-degree feedback more effective than feedback from a single source? The current results suggest that feedback sig- nificantly improves the onset of transfer following a leadership training program, further bolstering the argument for the use of feedback within training programs (e.g., Ford et al., 1998; Kluger & DeNisi, 1996; Smith-Jentsch, Campbell, Milanovich, & Reyn- olds, 2001). The same pattern was found for reactions, learning, and results such that the meta-analytic effect size for programs reporting the use of feedback were greater than those without— albeit this effect was nonsignificant. These findings indicate that there is utility in providing trainees with feedback during a lead- ership training intervention, with an emphasis on effectively de- signed feedback. Perhaps the nonsignificant difference in reac- tions, learning, and results stems from not implementing effective feedback designs. For instance, feedback provided within the context of leadership training may be focused more on character- istics of the self instead of task activities, as feedback intervention theory suggests (Kluger & DeNisi, 1996; Kluger & DeNisi, 1998), leading to a decrease in feedback effectiveness. Our findings also illustrate the use of 360-degree feedback in leadership training programs, as compared with not using 360-degree feedback, is related to higher results, but lower levels of learning and transfer. As these differences were not statistically significant and the effect sizes were generated from a relatively low number of primary studies (due to a lack of information available), it highlights the need for future research surrounding the use of 360-degree feed- back in the leadership development domain. The current results prove to be surprising given the widespread use of 360-degree feedback (i.e., approximately 90% of large organizations report the use of this feedback method; ETS, 2012), and may lend initial support to DeNisi and Kluger’s (2000) argument that multisource feedback might not be universally useful. For example, in the context of leadership training, increasing the number of feedback sources may not result in additional information received by the leader beyond feedback from a single-source (Borman, 1998; DeNisi & Kluger, 2000) and perhaps when negative feedback is received from multiple sources, this information could thwart improvement in one’s learning or behavioral transfer because it threatens one’s self-view. As such, the current findings indicate that the effect of 360-degree feedback on training outcomes might be more complex than originally concluded (i.e., potentially inhib- iting learning and transfer, while improving organizational and subordinate outcomes), highlighting a need for more research on this training method.
Are on-site training programs more effective than off-site training programs? Arthur et al. (2003) noted there was an extreme paucity of research on training location in their meta- analysis of the generic training literature. Over a decade later, we are able to provide meta-analytic evidence that supports the use of
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1702 LACERENZA, REYES, MARLOW, JOSEPH, AND SALAS
on-site training as it improves results to a greater degree than off-site training (on-site training is equally effective as off-site training for learning and transfer criteria). Employees learn and transfer material from off-site programs adequately; however, these programs may be less applicable to a trainee’s work envi- ronment, leading to a decrease in result outcomes. Specifically, off-site programs may feature less involvement from organiza- tional insiders, relying instead on personnel who lack familiarity with the training needs of the employees and the organization. Consequently, the program utilized may have more of a one-size fits all nature. This is problematic, as a one-size fits all approach has been criticized for not addressing the specific needs of the organization; specifically, within the context of assessment cen- ters, Krause and Gebert (2003) suggested that custom-made pro- grams may be “more sensitive and more precise in depicting a company’s environment and dynamics, as well as individual jobs” (Krause & Gebert, 2003, p. 396) than off-the-shelf interventions. We argue that this idea can also be applied to leadership training, as custom-made leadership training programs may better address the organization’s leadership needs. However, we further note that the decreased organizational results associated with off-site train- ing, as compared with on-site training programs, may also stem from additional sources, such as reduced fidelity or reduced con- venience for attendees. Given this, we recommend that leadership training programs use on-site training and we note that future research would benefit from an examination of why on-site pro- grams tend to be more effective than off-site programs.
Are leadership training programs that are virtually based less effective than face-to-face programs? Despite the recent surge of interest in virtually based leadership programs (e.g., Harvard Business Publishing, 2013), no meta-analytic compari- sons among these programs and face-to-face leadership training programs have been made, until now. In contrast to meta-analytic research suggesting that web-based programs are more effective than face-to-face programs in the generic training literature (Sitz- mann, Kraiger, Stewart, & Wisher, 2006), our results suggest the opposite to be true. Specifically, we found greater transfer in- creases after face-to-face leadership training compared to virtually based leadership training (and no difference between modalities for learning criteria). It may be the case that virtually based courses can be less effective for some criteria because this modality involves fewer opportunities for demonstration and practice (i.e., the effect of modality may be spurious) and future work should examine whether virtually based programs that use the same delivery method as face-to-face programs are equally effective.
Does some leadership training program content result in greater outcomes than other content? Previous meta-analyses suggest that leadership training outcomes are affected by the content trained (Burke & Day, 1986; Taylor et al., 2009b), and the current results support this claim. Regarding both learning and transfer, programs that trained business skills (e.g., problem- solving, data analysis, monitoring budgets) were the most effec- tive, paralleling Hogan and Warrenfeltz’s (2003) theory that busi- ness skills are the easiest to train out of the four competency domains. In contrast, we found that soft skills (i.e., leadership, interpersonal, intrapersonal competencies) improved organiza- tional and subordinate outcomes (i.e., results) more than hard skills (i.e., business), supporting leadership theories that highlight the importance of softer skills (e.g., leader–member exchange, Dan-
sereau, Graen, & Haga, 1975; transformational leadership, Burns, 1978). Thus, these results appear to suggest that although hard skills are the easiest to learn and transfer, soft skills matter the most for organizational and subordinate results. These results run counter to the anticipated demand for hard skills among employers (Schawbel, 2014), suggesting that softer skills are more important in predicting desired organizational and subordinate outcomes.
Do leadership training programs evaluated by a team of practitioners and academics result in greater outcomes com- pared with those evaluated by either academics or practitioners? Management literature has long acknowledged the gap between science and practice and has noted that bridging this gap would provide mutual enrichment in the field (Hodgkin- son & Rousseau, 2009). In support of this, our findings indicate that leadership training programs evaluated by a team of academ- ics and practitioners exhibited significantly greater learning and transfer outcomes compared to those evaluated by academics or industry experts only. This finding supports the argument that the scientist-practitioner model is necessary for producing superior work (Jones & Mehr, 2007). Although it is unclear whether mixed scientist and practitioner author teams worked together to design and deliver the training, results suggest that academics and prac- titioners who work together to evaluate (and possibly design/ deliver) leadership training programs are producing more effective programs, suggesting academics and practitioners should collabo- rate on leadership training ventures whenever possible.
Are longer leadership training programs more effective? Although Taylor et al.’s (2009b) meta-analysis of leadership train- ing did not find support for training length as a continuous mod- erator of training effectiveness, we found some evidence that length of training has an impact on effectiveness. Specifically, we found a linear relationship between training duration and the results criterion, indicating that longer training programs lead to improved organizational and subordinate outcomes (a curvilinear relationship was not supported after outliers were removed). These results are counterintuitive as they do not fully support CLT (which suggests that information overload causes strain on work- ing memory capacity reduced learning; Paas et al., 2004), which would predict a negative relationship between training duration and effectiveness. Future research would benefit from an exami- nation of whether the increased effectiveness associated with lon- ger programs is due to increased knowledge transfer, more time for multiple delivery methods, or perhaps increases in trainee percep- tions of training program value.
Has the effectiveness of leadership training increased over time? Current results suggest that over time, the effect of lead- ership training on results has increased; however, this was not the case for learning or transfer (i.e., improvements in learning and transfer have been relatively steady over time). This bolsters the common theme across the current data that leadership training is more valuable than previously thought (cf. Zimmerman, 2015; Myatt, 2012). In fact, the leadership training industry has improved its impact on results over the years.
Limitations
The current study provides several noteworthy contributions to the literature; however, there are several limitations. First, we note that we were only able to include evaluations of leadership training
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1703LEADERSHIP TRAINING: A META-ANALYSIS
programs reported in available unpublished and published litera- ture. It is probable that effective leadership training programs are published more often than ineffective leadership training pro- grams. Although our publication bias analyses indicated no bias, we note that ineffective leadership training programs might also be disproportionately excluded from unpublished work (i.e., organi- zations with ineffective programs do not give researchers access to data about the effectiveness of these programs). Thus, despite a lack of publication bias evident in the current article, our results may still be upwardly biased because our meta-analytic database can only reflect existing studies, which are more likely to have been conducted on effective rather than ineffective training pro- grams. Moreover, our results may not represent trends evident in practice that are not documented within available literature. For example, many leadership training programs have focused on emotional intelligence since Goleman’s (1998) widely read com- mentary on emotional intelligence in leadership, but we did not have enough data to specifically analyze emotional intelligence training for leaders separately from other forms of training. Sec- ond, despite the noted popularity of measuring training effective- ness with reactional data (Patel, 2010), the current search effort only identified seven experimental studies reporting trainee reac- tion data. We found that studies reporting reactions tend to utilize a post-only study design instead of a prepost or independent groups experimental design, excluding the vast majority of pri- mary studies that report reactions from meta-analytic examination. Although the current results indicate that leadership training im- proves trainee reactions, because of the lack of reaction data, any moderator analyses conducted within this criterion should be in- terpreted with caution (we provide several moderated effect sizes in Table 2 as initial evidence that should be confirmed with future research). Given that most organizations include reaction data in
their evaluation efforts (Patel, 2010), we urge scholars to interpret this lack of available primary data as a signpost for future research in this area.
Practical Implications
The current meta-analytic evidence stands in direct opposition to the recent argument posed by Myatt (2012), a popular press contributor, that leadership training “is indeed the #1 reason lead- ership development fails.” This study provides substantial evi- dence from 335 leadership training evaluation studies that these programs are effective and should be used across a variety of domains. In fact, results suggest that leadership training programs can lead to a 25% increase in learning, 28% increase in leadership behaviors performed on-the-job (i.e., transfer), 20% increase in overall job performance, 8% increase in subordinate outcomes, and a 25% increase in organizational outcomes (percent increase is equal to Cohen’s U3 � 50; Cohen, 1988). The results also suggest that the extent to which a program is effective is related to various design, delivery, and implementation elements. As such, this study is meant to serve as a guide for practitioners when developing a leadership training program, and in this vein, we provide a sum- mary of our recommendations to practitioners in Table 8.
The current results suggest that practitioners should identify intended outcomes before developing a leadership training pro- gram because training design characteristics affect each outcome differently. As such, questions for practitioners to ask at the early stages of development include: Who are my stakeholders and what outcome(s) are they trying to obtain? Am I interested in multiple outcomes, and if so, are some outcomes more important than others? If developers do not have a clear objective, the current results suggest a needs analysis may not only guide the develop-
Table 8 Evidenced-Based Best Practices for Designing a Leadership Training Program
1. Resist the temptation to think that leaders cannot be trained; evidence suggests leadership training programs are effective. 2. Conduct a needs analysis and identify the desired outcome(s) based on stakeholder goals before designing the program. 3. Use multiple delivery methods when possible (e.g., information, demonstration, and practice) and if limitations prevent this, choose practice instead
of other delivery methods. 4. Use caution when spending additional resources on 360-degree feedback (evidence indicates that it might not be more effective than single-source
feedback). 5. Provide multiple training sessions that are separated by time rather than a single, massed training session. 6. Use caution when implementing self-administered training and instead, choose an internal or external trainer (evidence shows no differences in the
effectiveness of internal and external trainers but indicates that self-administered training is less effective). 7. Consult with others outside of your field to ensure the program is both evidence-based and practically relevant (e.g., if you are a practitioner,
collaborate with an academic). 8. Ensure the program is designed appropriately according to the desired outcome using the guidelines provided below.
Learning Transfer Results
• Use multiple delivery methods • Use multiple delivery methods • Use multiple delivery methods • Conduct a needs analysis • Conduct a needs analysis • Hold on-site • Include hard skills (i.e., business skills) • Provide feedback • Require mandatory attendance
• Use a face-to-face setting • Have multiple sessions • Make attendance voluntary • Provide as much training as possible
(longer programs are more effective) • Have multiple sessions • Include soft skills
(i.e., intrapersonal, interpersonal, and leadership skills) • Include hard (i.e., business
skills) and soft skills (i.e., leadership skills)
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1704 LACERENZA, REYES, MARLOW, JOSEPH, AND SALAS
Table 9 Examples of Effective Training Programs
Moderator Citation Example Effect size
Needs analysis Porras et al. (1982) “Targeted supervisory skills were selected from a problem list created during the needs-assessment portion of the organizational diagnosis” (p. 437).
.74T
Baron et al. (1984) The content was “developed by examining the responsibilities, activities, and interactions of child care administrators and by adapting the teaching and social interaction technology that is used in Teaching-Family programs to managerial interactions” (p. 264).
.64L
Santrić Milicevic et al. (2011)
“The project began with manager competency needs assessment. Assessment results were used to design training and supervision so that each management team had a long term strategy, a business case plan and an implementation plan for total quality management of the work process at the end of the project” (p. 248).
1.34T
Morin and Latham (2000) “Prior to conducting this study, all supervisors had attended a one-day training programme on interpersonal communication skills. A needs analysis revealed that the participants had thorough knowledge of the requisite communication behaviours, but lacked confidence in applying them on the job. Hence, a one- day refresher training programme was conducted” (p. 569).
.95T
Osborne (2003) “In addition to program participant needs assessments completed prior to the commencement of each program module, I requested that executive committee members also complete a general needs assessment. Such information enabled me to focus program design on the needs of those sponsoring the program as well as on the needs of individual participants. The anonymous responses to this needs assessment were combined and analyzed. I highlighted frequent comments and used the information gained to guide program design and implementation. For example, several committee members expressed a concern for measurement of program impact. Once I determined that this was a need, I incorporated the pre and post program testing as well as interim and final benchmark statement comparisons into the study design” (p. 71).
.81T
House (2001) “The program was designed and developed after an extensive six month needs assessment was conducted to identify the skills and competencies new managers needed to be successful in the company’s environment. The needs assessment included interviewing and conducting focus groups with new and experienced managers in the company, executives of the company and human resources professionals. In addition, the needs assessment included benchmarking with other high technology companies to assess their management training practices” (p .14).
2.02L
Attendance policy Fitzgerald and Schutte (2010)
“They participated voluntarily and were free to withdraw from the study at any time” (p. 499).
.76T
Suwandee (2009) “The consent forms for program participation were provided to the middle executives of Kasem Bundit University in order to select volunteers to attend the program. Consequently, the participants were willing to enhance their knowledge and insights toward their organizational leadership potential” (p. 80).
.72T
Alexander (2014) The training was mandatory. 1.34RE Spacing effect Green (2002) “The training program is seven days in length, but the training sessions do not
run consecutively. The training program design provides that participants attend three days of training, return to the workplace for three weeks, and then return to attend two days of training. There is another three-week break before participants return for the final two days of training. During the periods between the training sessions, participants are asked to apply the skills learned and to return to the course with prepared assignments about their learning experiences” (pp. 53–54).
.86T
Parry and Sinha (2005) There was a 2-day intervention, followed by 3 months of implementing the leadership development program at work, and then another 2-day intervention.
.74T .98RE
Hassan, Fuwad, and Rauf (2010)
“Training was divided into four modules that were offered with a lag of 7 days” (p. 126).
.78T 1.28RE
Trainees’ level of leadership
Allen (2010) Participants were from nursing unit teams that rotate team leadership on a yearly basis.
1.25T
Latham and Saari (1979) Trained first-line supervisors. 1.18T Bostain (2000) Trained first-line supervisors. 4.17T May and Kahnweiler (2000) Trained first-line supervisors. 1.13T
Internal vs. External trainer
Birkenbach et al. (1985) “The second author was asked to train personnel from the training department of the company in the use of behaviour modelling . . . After the initial training course, the author worked with company training staff in developing the in- company programme. The major inputs were, however, made by the company’s own trainers” (p. 13).
1.44L
(table continues)
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1705LEADERSHIP TRAINING: A META-ANALYSIS
Table 9 (continued)
Moderator Citation Example Effect size
Donohoe et al. (1997) The Employee Assistance Programs (EAPs) coordinator trained the supervisors. 1.94L Nemeroff and Cosentino
(1979) “Two research associates from the Life Office Management Association served
as ‘trainers’ and administered the feedback and feedback plus goal setting treatments” (p. 569).
.71T .61RE
Practice-based method Bozer, Sarros, and Santora (2013)
“The coaching process included 10 to 12 sessions with weekly interventions. All coaching activities commenced with an assessment and identification of a developmental issue, followed by a feedback session, goal setting, action planning, and follow-up sessions, and concluded with an evaluation of outcomes, consistent with established coaching procedures” (p. 282).
.94RE
Nemeroff and Cosentino (1979)
Goal setting was used in the training. .61RE
Information-, demonstration-, and practice-based methods
Yirga (2011) The Basic Leadership Development Program involved reading material, class lectures, a coaching experience, and a team project.
1.12L
Hassan, Fuwad, and Rauf (2010)
“Importance of setting specific, difficult but attainable, goals was discussed in a lecture setting. Afterwards, a day long interactive session was conducted to identify appropriate goals and objectives for all participants according to their work requirements. Role playing and in basket exercises were conducted. The session concluded with a case study situation requiring transformational leadership exhibition by participants. Participants were then asked to come up with their goals and objectives in the next session” (p. 126). The other sessions included an assessment on what the leaders knew about transformational leadership, role playing, and self-reports on their behavioral changes.
.78T 1.28RE
Donohoe, Johnson, and Stevens (1997)
The training program included a discussion, a training film, and an activity for the supervisors to assess employee behavior and then re-assess the employee using a handbook’s evaluation criteria.
1.94L1.07T
House (2001) “Methods used to teach the program included a business simulation, lecture and discussion sessions, interactive role playing, case studies and skill building exercises. In addition, the program hosted four executive guest speakers throughout the week long training program” (p. 14).
2.02L
Feedback Engelbrecht and Fischer (1995)
“Feedback is given by the assessors to the senior of the participant, detailing strengths and weaknesses and also focusing on developmental action plans for the participant. A detailed and lengthy feedback is also given to the participant in an interactive (facilitative) manner” (p. 395).
.61T
Nemeroff and Cosentino (1979)
“One-hour interviews were held with each manager, and specific feedback on subordinate’s perceptions of 43 interview behaviors was reviewed and discussed. Feedback was given at one point in time by an external source other than the manager’s boss (trainer), in a face-to-face oral transmission session. In terms of content, the feedback given was in the form of means and standard deviations from the questionnaire responses of the manager’s subordinates. Norms were also provided in the form of scale and item mean and standard deviations for the total sample of managers in the company. The feedback, together with the norms, permitted managers to determine their own strengths and weaknesses on specific interview behaviors and to determine areas in which improvements might be needed. The meaning of the feedback data and sample norms was carefully explained to each manager by the trainers. Moreover, during the feedback sessions, great care was taken to insure a minimum of threat to the managers. They were asked to view the feedback as diagnostic rather than evaluative. The trainers pointed out that the feedback information would have developmental implications only to the extent that the managers used it to increase their repertoire of effective appraisal interview skills” (p. 569).
.71T
On-site Barling, Weber, and Kelloway (1996)
“The study took place in one region of one of the five largest banks in Canada” (p. 828).
.84RE
Knox, (2000) The training took place at a facility within the corporation and “established trainers currently employed or contracted by the company were used to facilitate the courses” (p. 34).
.70RE .61RE
Neck and Manz (1996) “The setting for the training study was the Agency Accounting Department of America West Airlines. America West is an international commercial airline employing approximately 12,100 people. America West Airlines is a major carrier based in Phoenix, Arizona with hubs in Phoenix, Las Vegas, and Columbus, Ohio” (p. 448).
Face-to-face Allen (2010) Coaching, instruction, reflection, and team meetings were face-to-face. 1.25T
Note. R � reactions; L � learning; T � transfer; RE � results.
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1706 LACERENZA, REYES, MARLOW, JOSEPH, AND SALAS
ment of the program, but may also result in a more effective program. Interestingly, our results also suggest voluntary training programs seem to be a double-edged sword, which is of concern for practitioners because most are hoping to increase both transfer and results. As previously mentioned, an explanation of this find- ing is that mandatory programs increase employee participation, thereby increasing the onset of higher-level, organizational results. Because more employees are in attendance, the effects are more likely to disseminate throughout the organization; greater atten- dance equals greater results. From this, we argue that training developers and researchers should identify ways in which volun- tary training programs can be more appealing to employees. Within the marketing literature, many researchers have identified effective promotional methods (e.g., Goldsmith & Amir, 2010; Grönroos, 1982; Khan & Dhar, 2010) such as bundling, in which two or more products are packaged together (Stremersch & Tellis, 2002), and research suggests this practice increases sales (e.g., Chakravarti, Krish, Paul, & Srivastava, 2002; Yadav & Monroe, 1993). Leadership training programs that are advertised as a “bun- dled” program (e.g., programs teaching leadership skills and pro- viding exposure to a new technology) may also have greater “sales” (i.e., voluntary attendance) than training interventions ad- vertised as targeting a single leadership competency.
In addition to the list of recommendations provided in Table 8, we also provide specific examples of highly effective training design, delivery, and implementation elements in Table 9, which were gathered by examining the primary studies within each hypothesized moderator that exhibited strong effect sizes (e.g., if a practitioner wanted to develop a training program with feedback, s/he may use Table 9 for examples of how feedback has been successfully incorporated into leadership training in previous work).
Conclusion
The current meta-analysis offers several contributions to the leadership and training literatures. First, our results suggest that leadership training is substantially more effective than previously thought, leading to improvements in perceptions of utility and satisfaction, learning, transfer to the job, organizational outcomes, and subordinate outcomes. Moreover, all but seven of the 120 effect sizes were positive and significantly different from zero, indicating that leadership training likely improves outcomes, re- gardless of its design, delivery, and implementation elements (i.e., leadership training is rarely a “failure;” cf. Myatt, 2012). Second, the current results suggest that leadership training is most effective when the training program is based on a needs analysis, incorpo- rates feedback, uses multiple delivery methods (especially prac- tice), uses spaced training sessions, is conducted at a location that is on-site, and uses face-to-face delivery that is not self- administered. Third, our results also have a variety of practical implications for the development of training programs, which we have summarized in Table 8 and provided examples of in Table 9 in order to guide scientists and practitioners in the development of evidence-based leadership training programs. Finally, we note that although the current meta-analysis suggests leadership training is effective, it does not promote a one-size fits all approach; many of the moderators of leadership training effectiveness investigated in the current study were important for some criteria but not all,
indicating that training program developers should first choose their desired criterion (or criteria) and then develop the training program based on this criterion.
References �References marked with an asterisk were included in the meta-analysis.
�Abrell, C., Rowold, J., Weibler, J., & Moenninghoff, M. (2011). Evalu- ation of a long-term transformational leadership development program. Zeitschrift Für Personalforschung, 25, 205–224.
Adair, J. (1983). Effective leadership. London, UK: Pan. �Adcock-Shantz, J. (2011). A study of the impact of a leadership develop-
ment program on a community college’s front-line and middle managers (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3490538)
�Agboola Segunro, O. (1997). Impact of training on leadership develop- ment: Lessons from a leadership training program. Evaluation Review, 21, 713–737. http://dx.doi.org/10.1177/0193841X9702100605
�Albanese, R. (1967). A case study of executive development. Training and Development Journal, 21, 28 –34.
Alessi, S. (2000). Simulation design for training and assessment. In H. F. O’Neil & D. H. Andrew (Eds.), Aircrew training and assessment (pp. 199 –224). Mahwah, NJ: Erlbaum.
�Alexander, B. K. (2014). A case for change: Assessment of an evidenced- based leadership development program. Dissertation Abstracts Interna- tional Section A, 74, 9.
�Al-Kassabi, M. A. (1985). A management training model for engineers in Saudi Arabia (Doctoral dissertation). Retrieved from ProQuest Disser- tations & Theses A&I; ProQuest Dissertations & Theses Global. (8606797)
�Allen, L. A. (2010). An evaluation of a shared leadership training program (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3420897)
Alliger, G. M., Tannenbaum, S. I., Bennett, W., Traver, H., & Shotland, A. (1997). A meta-analysis of the relations among training criteria. Per- sonnel Psychology, 50, 341–358. http://dx.doi.org/10.1111/j.1744-6570 .1997.tb00911.x
Alsamani, A. S. (1997). The management development program in Saudi Arabia and its effects on managerial job performance (Doctoral disser- tation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (9818670)
�Anghelone, J. D. (1981). The effect of a management training seminar upon the transfer of leadership skills to actual on the job performance (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (8214513)
�Aronoff, J., & Litevin, G. H. (1971). Achievement motivation training and executive advancement. The Journal of Applied Behavioral Science, 7, 215–229. http://dx.doi.org/10.1177/002188637100700207
Arthur, W., Jr., Bennett, W., Jr., Edens, P. S., & Bell, S. T. (2003). Effectiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology, 88, 234 –245. http:// dx.doi.org/10.1037/0021-9010.88.2.234
Arthur, W., Jr., Bennett, W., Jr., & Huffcutt, A. (2001). Conducting meta-analysis using SAS. Mahwah, NJ: Erlbaum, Inc.
Arvey, R. D., Rotundo, M., Johnson, W., Zhang, Z., & McGue, M. (2006). The determinants of leadership role occupancy: Genetic and personality factors. The Leadership Quarterly, 17, 1–20. http://dx.doi.org/10.1016/ j.leaqua.2005.10.009
Arvey, R. D., Zhang, Z., Avolio, B. J., & Krueger, R. F. (2007). Devel- opmental and genetic determinants of leadership role occupancy among women. Journal of Applied Psychology, 92, 693–706. http://dx.doi.org/ 10.1037/0021-9010.92.3.693
Atkins, P. W., & Wood, R. E. (2002). Self-versus oters’ ratings as predic- tors of assessment centers ratings: Validation evidence for 360-degree
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1707LEADERSHIP TRAINING: A META-ANALYSIS
feedback programs. Personnel Psychology, 55, 871–904. http://dx.doi .org/10.1111/j.1744-6570.2002.tb00133.x
�Augustin, D. A. (2003). An empirical investigation of leadership devel- opment training with respect to the leadership characteristics and be- haviors of emerging leaders within their organizational contexts (Doc- toral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3083789)
Avolio, B., Avey, J., & Quisenberry, D. (2010). Estimating return on leadership development investment. The Leadership Quarterly, 21, 633– 644. http://dx.doi.org/10.1016/j.leaqua.2010.06.006
Avolio, B. J., Reichard, R. J., Hannah, S. T., Walumbwa, F. O., & Chan, A. (2009). A meta-analytic review of leadership impact research: Ex- perimental and quasi-experimental studies. The Leadership Quarterly, 20, 764 –784. http://dx.doi.org/10.1016/j.leaqua.2009.06.006
Avolio, B. J., Rotundo, M., & Walumbwa, F. O. (2009). Early life expe- riences and environmental factors as determinants of leadership emer- gence: The role of parental influence and rule breaking behavior. The Leadership Quarterly, 20, 329 –342. http://dx.doi.org/10.1016/j.leaqua .2009.03.015
�Ayers, A. W. (1964). Effect of knowledge of results on supervisors’ post-training test scores. Personnel Psychology, 17, 189 –192. http://dx .doi.org/10.1111/j.1744-6570.1964.tb00060.x
Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for future research. Personnel Psychology, 41, 63–105. http:// dx.doi.org/10.1111/j.1744-6570.1988.tb00632.x
Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ: Prentice Hall.
Bandura, A., & Wood, R. (1989). Effect of perceived controllability and performance standards on self-regulation of complex decision making. Journal of Personality and Social Psychology, 56, 805– 814. http://dx .doi.org/10.1037/0022-3514.56.5.805
Barling, J., Christie, A., & Hoption, C. (2010). Leadership. In S. Zedeck (Ed.), Handbook of industrial and organizational psychology (pp. 183– 240). Washington, DC: APA Books.
�Barling, J., Weber, T., & Kelloway, E. K. (1996). Effects of transforma- tional leadership training on attitudinal and financial outcomes: A field experiment. Journal of Applied Psychology, 81, 827– 832. http://dx.doi .org/10.1037/0021-9010.81.6.827
�Baron, L., & Morin, L. (2010). The impact of executive coaching on self-efficacy related to management soft-skills. Leadership and Organi- zation Development Journal, 31, 18 –38. http://dx.doi.org/10.1108/ 01437731011010362
�Baron, R. L., Leavitt, S. E., Watson, D. M., Coughlin, D. D., Fixsen, D. L., & Phillips, E. L. (1984). Skill-based management training for child care program administrators: The teaching-family model revisited. Child Care Quarterly, 13, 262–277. http://dx.doi.org/10.1007/BF01118803
�Barrett, P. T. (2007). The effects of group coaching on executive health and team effectiveness: A quasi-experimental field study. Dissertation Abstracts International, Section A, 67, 2640.
�Barrett, R. S. (1965). Impact of the executive program on the participants. Journal of Industrial Psychology, 3, 1–13.
�Barthol, R. P., & Zeigler, M. (1956). Evaluation of a supervisory training program with How Supervise. Journal of Applied Psychology, 40, 403– 405. http://dx.doi.org/10.1037/h0044833
�Bass, B. M. (1962). Reactions to Twelve Angry Men as a measure of sensitivity training. Journal of Applied Psychology, 46, 120 –124. http:// dx.doi.org/10.1037/h0039191
�Beaton, R., Johnson, L. C., Infield, S., Ollis, T., & Bond, G. (2001). Outcomes of a leadership intervention for a metropolitan fire depart- ment. Psychological Reports, 88, 1049 –1066. http://dx.doi.org/10.2466/ pr0.2001.88.3c.1049
�Biggs, A., Brough, P., & Barbour, J. P. (2014). Enhancing work-related attitudes and work engagement: A quasi-experimental study of the
impact of an organizational intervention. International Journal of Stress Management, 21, 43– 68. http://dx.doi.org/10.1037/a0034508
�Biggs, D. A., Huneryager, S. G., & Delaney, J. J. (1966). Leadership behavior: Interpersonal needs and effective supervisory training. Per- sonnel Psychology, 19, 311–320. http://dx.doi.org/10.1111/j.1744-6570 .1966.tb00307.x
�Birkenbach, X. C., Karnfer, L., & ter Morshuizen, J. D. (1985). The development and the evaluation of a behaviour-modelling training pro- gramme for supervisors. South African Journal of Psychology, 15, 11–19. http://dx.doi.org/10.1177/008124638501500102
�Blackburn, W. (1985, August). A program evaluation of the basic super- visory development seminar. A report presented at San Diego State University, San Diego, CA.
�Blastorah, M. (2009). The effect of mentoring on leadership self-efficacy in nurses (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (NR67743)
�Bloom, P. J., & Sheerer, M. (1992). The effect of leadership training on child care program quality. Early Childhood Research Quarterly, 7, 579 –594. http://dx.doi.org/10.1016/0885-2006(92)90112-C
Blume, B. D., Ford, J. K., Baldwin, T. T., & Huang, J. L. (2010). Transfer of training: A meta-analytic review. Journal of Management, 36, 1065– 1105. http://dx.doi.org/10.1177/0149206309352880
Borman, W. C. (1998). 360-Degree ratings: An analysis of assumptions and a research agenda for evaluating their validity. Human Resource Management Review, 7, 299 –315. http://dx.doi.org/10.1016/S1053- 4822(97)90010-3
�Boss, R. W. (1980). Your CEO and natural team training. Training and Development Journal, 34, 76 – 80.
�Bostain, N. S. (2000). Evaluation of a management development behavior modeling training program in industry: Transfer of training (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (9968772)
�Bozer, G., Sarros, J. C., & Santora, J. C. (2013). The role of coachee characteristics in executive coaching for effective sustainability. Journal of Management Development, 32, 277–294. http://dx.doi.org/10.1108/ 02621711311318319
�Bramley, P. (1999). Evaluating effective management learning. Journal of European Industrial Training, 23, 145–153. http://dx.doi.org/10.1108/ 03090599910261826
�Bright, J. E. H., & Halse, R. (1998). An evaluation of an industry training programme. Training and Development in Australia, 4, 9 –11.
Brown, A. L., Bransford, J. D., Ferrara, R. A., & Campione, J. C. (1983). Learning, remembering, and understanding. In J. H. Flavell & E. M. Markman (Eds.), Handbook of child psychology (4th ed., Vol. 3, pp. 77–166). New York, NY: Wiley.
Brown, K. G. (2005). An examination of the structure and nomological network of trainee reactions: A closer look at “smile sheets.” Journal of Applied Psychology, 90, 991–1001. http://dx.doi.org/10.1037/0021-9010 .90.5.991
�Brown, T. C., & Warren, A. M. (2009). Distal goal and proximal goal transfer of training interventions in an executive education program. Human Resource Development Quarterly, 20, 265–284. http://dx.doi .org/10.1002/hrdq.20021
�Brown, T. C., & Warren, A. M. (2014). Evaluation of transfer of training in a sample of union and management participants: A comparison of two self-management techniques. Human Resource Development Interna- tional, 17, 277–296. http://dx.doi.org/10.1080/13678868.2014.907975
�Brown, W., & May, D. (2012). Organizational change and development: The efficacy of transformational leadership training. Journal of Man- agement Development, 31, 520 –536. http://dx.doi.org/10.1108/ 02621711211230830
�Bruwelheide, L., & Duncan, P. (1986). A method for evaluating corpo- ration training seminars. Journal of Organizational Behavior Manage- ment, 7, 65–94.
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1708 LACERENZA, REYES, MARLOW, JOSEPH, AND SALAS
Burke, M. J., & Day, R. R. (1986). A cumulative study of the effectiveness of managerial training. Journal of Applied Psychology, 71, 232–265. http://dx.doi.org/10.1037/0021-9010.71.2.232
�Burnaska, R. F. (1976). The effects of behavior modeling and training upon managers’ behaviors and employees’ perceptions. Personnel Psy- chology, 29, 329 –335. http://dx.doi.org/10.1111/j.1744-6570.1976 .tb00416.x
Burns, J. M. (1978). Leadership. New York, NY: Harper & Row. �Busch, J. R. (2003). School leadership formation: A multimethod evalu-
ation study of the Southern Tier Leadership Academy (Doctoral disser- tation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3102077)
�Byham, W., Adams, D., & Kiggins, A. (1976). Transfer of modeling training to the job. Personnel Psychology, 29, 345–349. http://dx.doi .org/10.1111/j.1744-6570.1976.tb00418.x
�Cadwalader, D. S. (1985). Evaluation of volunteer leadership develop- ment workshops: Effects of participation in the planning process (Doc- toral dissertation). Retrieved from ProQuest Dissertations & Theses. (8611514)
�Cainer, S. (1991). The application of an interpersonal skills program for supervisors in industry (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses Dissertations & Theses A&I; ProQuest Disser- tations & Theses Global. (NN69315)
�Canter, R. R., Jr. (1951). A human relations training program. Journal of Applied Psychology, 35, 38 – 45. http://dx.doi.org/10.1037/h0053739
�Carrick, L. A. (2010). Understanding the impact of a half day learning intervention on emotional intelligence competencies: An exploratory study (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3410485)
�Carron, T. J. (1964). Human relations training and attitude change: A vector analysis. Personnel Psychology, 17, 403– 424. http://dx.doi.org/ 10.1111/j.1744-6570.1964.tb00076.x
�Cato, B. (1990). Effectiveness of a management training institute. Journal of Park and Recreation Administration, 8, 38 – 46.
Chakravarti, D., Krish, R., Paul, P., & Srivastava, J. (2002). Partitioned presentation of multicomponent bundle prices: Evaluation, choice and underlying processing effects. Journal of Consumer Psychology, 12, 215–229. http://dx.doi.org/10.1207/S15327663JCP1203_04
Chen, J. C. (2014). Teaching nontraditional adult students: Adult learning theories in practice. Teaching in Higher Education, 19, 406 – 418. http:// dx.doi.org/10.1080/13562517.2013.860101
�Cherniss, C., Grimm, L. G., & Liautaud, J. P. (2010). Process-designed training: A new approach for helping leaders develop emotional and social competence. Journal of Management Development, 29, 413– 431. http://dx.doi.org/10.1108/02621711011039196
Chiaburu, D. S., & Marinova, S. V. (2005). What predicts skill transfer? An exploratory study of goal orientation, training self-efficacy and organizational supports. International Journal of Training and Devel- opment, 9, 110 –123. http://dx.doi.org/10.1111/j.1468-2419.2005 .00225.x
�Chochard, Y., & Davoine, E. (2011). Variables influencing the return on investment in management training programs: A utility analysis of 10 Swiss cases. International Journal of Training and Development, 15, 225–243. http://dx.doi.org/10.1111/j.1468-2419.2011.00379.x
�Clark, B. (1989). Professional development emphasizing the socialization progress for academicians planning administrative leadership roles (Unpublished doctoral dissertation). George Washington University, Washington, DC.
�Clark, H. B., Wood, R., Kuehnel, T., Flanagan, S., Mosk, M., & Northrup, J. T. (1985). Preliminary validation and training of supervisory interac- tional skills. Journal of Organizational Behavior Management, 7, 95– 115. http://dx.doi.org/10.1300/J075v07n01_07
�Clark, L. S. (1982). An analysis of changes in attitudes of supervisors and managers toward supervisory practices, company policies, and super-
visor opinions (Doctoral dissertation). Retrieved from ABI/INFORM Complete; ProQuest Dissertations & Theses A&I; ProQuest Disserta- tions & Theses Global. (8225498)
�Clement, R. W. (1982). Testing the hierarchy theory of training evaluation: An expanded role for trainee reactions. Public Personnel Management Journal, 11, 176 –184. http://dx.doi.org/10.1177/009102608201100210
Cohen, D. J. (1990). What motivates trainees? Training and Development Journal, 44, 91.
Cohen, J. (1988). Statistical power analysis for the behavioural sciences. Hillside, NJ: Erlbaum.
�Cole, N. (2008). How long should a training program be? A field study of “rules-of-thumb.” Journal of Workplace Learning, 20, 54 –70. http://dx .doi.org/10.1108/13665620810843647
Collins, D. B., & Holton, E. F. (2001). The effectiveness of managerial leadership development programs: A meta-analysis of studies from 1982 to 2001. Human Resource Development Quarterly, 15, 217–248. http:// dx.doi.org/10.1002/hrdq.1099
�Conner, C. E. (1992). Effects of a leadership training course on partici- pant self-perception of leadership style effectiveness (Doctoral disserta- tion). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (9305776)
�Cortvriend, P., Harris, C., & Alexander, E. (2008). Evaluating the links between leadership development coaching and performance. Interna- tional Coaching Psychology Review, 3, 164 –179.
�Cowles, R. J. (1993). Missouri Leadership Academy experience and staff perceptions of principal leadership effectiveness (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dis- sertations & Theses Global. (9412907)
�Culpin, V., Eichenberg, T., Hayward, I., & Abraham, P. (2014). Learning, intention to transfer and transfer in executive education. International Journal of Training and Development, 18, 132–147. http://dx.doi.org/ 10.1111/ijtd.12033
�Culpin, V., & Scott, H. (2012). The effectiveness of a live case study approach: Increasing knowledge and understanding of “hard” versus “soft” skills in executive education. Management Learning, 43, 565– 577. http://dx.doi.org/10.1177/1350507611431530
�Cunningham, G., & Kitson, A. (2000). An evaluation of the RCN clinical leadership development programme: Part 2. Nursing Standard, 15, 34 – 40.
Curado, C., Henriques, P. L., & Ribeiro, S. (2015). Voluntary or mandatory enrollment in training and the motivation to transfer training. Interna- tional Journal of Training and Development, 19, 98 –109. http://dx.doi .org/10.1111/ijtd.12050
�Dahinten, V. S., Macphee, M., Hejazi, S., Laschinger, H., Kazanjian, M., McCutcheon, A., . . . O’Brien-Pallas, L. (2014). Testing the effects of an empowerment-based leadership development programme. Journal of Nursing Management, 22, 16 –28. http://dx.doi.org/10.1111/jonm.12059
Dansereau, F., Jr., Graen, G., & Haga, W. J. (1975). A vertical dyad linkage approach to leadership within formal organizations: A longitu- dinal investigation of the role making process. Organizational Behavior and Human Performance, 13, 46 –78. http://dx.doi.org/10.1016/0030- 5073(75)90005-7
�Davis, B. L., & Mount, M. K. (1984). Effectiveness of performance appraisal training using computer assisted instruction and behavior mod- eling. Personnel Psychology, 37, 439 – 452. http://dx.doi.org/10.1111/j .1744-6570.1984.tb00521.x
Day, D. V. (2000). Leadership development: A review in context. The Leadership Quarterly, 11, 581– 613. http://dx.doi.org/10.1016/S1048- 9843(00)00061-8
DeChurch, L. A., Hiller, N. J., Murase, T., Doty, D., & Salas, E. (2010). Leadership across levels: Levels of leaders and their levels of impact. The Leadership Quarterly, 21, 1069 –1085. http://dx.doi.org/10.1016/j .leaqua.2010.10.009
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1709LEADERSHIP TRAINING: A META-ANALYSIS
DeNisi, A. S., & Kluger, A. N. (2000). Feedback effectiveness: Can 360-degree appraisals be improved? The Academy of Management Ex- ecutive, 14, 129 –139.
�Depiano, L. G., & McClure, L. F. (1987). The training of principals for school advisory council leadership. Journal of Community Psychology, 15, 253–267. http://dx.doi.org/10.1002/1520-6629(198704)15:2�253:: AID-JCOP2290150214�3.0.CO;2-X
�De Vries, M. K., Hellwig, T., Vrignaud, P., Ramo, L. G., Florent-Treacy, E., & Korotov, K. (2009). Sustainable effectiveness of a transforma- tional leadership development program: An exploratory study. INSEAD Working Papers Collection, 2009, 2–32.
�Diaz, A. L. (2008). Leadership training and emotional intelligence in school nurses (Doctoral dissertation). Retrieved from ProQuest Disser- tations & Theses A&I; ProQuest Dissertations & Theses Global. (3319126)
�DiPietro, R. B. (2003). The effectiveness of managerial training in a fast food restaurant chain (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3092537)
DiPietro, R. B. (2006). Return on investment in managerial training: Does the method matter? Journal of Foodservice Business Research, 7, 79 – 96. http://dx.doi.org/10.1300/J369v07n04_04
�Donohoe, T. L., Johnson, J. T., & Stevens, J. (1997). An analysis of an employee assistance supervisory training program. Employee Assistance Quarterly, 12, 25–34. http://dx.doi.org/10.1300/J022v12n03_02
Donovan, J. J., & Radosevich, D. J. (1999). A meta-analytic review of the distribution of practice effect: Now you see it, now you don’t. Journal of Applied Psychology, 84, 795– 805. http://dx.doi.org/10.1037/0021- 9010.84.5.795
Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56, 455– 463. http://dx.doi.org/10.1111/j.0006-341X.2000 .00455.x
�Duygulu, S., & Kublay, G. (2011). Transformational leadership training programme for charge nurses. Journal of Advanced Nursing, 67, 633– 642. http://dx.doi.org/10.1111/j.1365-2648.2010.05507.x
�Eary, C. R. (1997). An examination of the affects of selected strategic learning variables on trainees’ performance in an Army NCO Academy (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (9802861)
�Eden, D., Geller, D., Gewirtz, A., Gordon-Terner, R., Inbar, I., Liberman, M., . . . Shalit, M. (2000). Implanting Pygmalion leadership style through workshop training: Seven field experiments. The Leadership Quarterly, 11, 171–210. http://dx.doi.org/10.1016/S1048-9843(00)00042-4
�Edwards, M. E. (1992). Assessing the leadership profile of appointed second-line administrators in the American community college. Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses Global. (304058742).
�Elo, A., Ervasti, J., Kuosma, E., & Mattila-Holappa, P. (2014). Effect of a leadership intervention on subordinate well-being. Journal of Man- agement Development, 33, 182–195. http://dx.doi.org/10.1108/JMD-11- 2012-0146
�Engelbrecht, A., & Fischer, A. H. (1995). The managerial performance implications of a developmental assessment center process. Human Rela- tions, 48, 387– 404. http://dx.doi.org/10.1177/001872679504800405
ETS. (2012). 90% of companies use 360 but many struggle with common issues. Retrieved from http://www.personneltoday.com/pr/2012/02/90- of-companies-use-360-but-many-struggle-with-common-issues/
�Faerman, S., & Ban, C. (1993). Trainee satisfaction and training impact: Issues in training evaluation. Public Productivity and Management Re- view, 16, 299 –314. http://dx.doi.org/10.2307/3380872
�Faust, D. H. (1981). The effect of in-service administrative training on leadership style (Doctoral dissertation). Retrieved from ProQuest Dis-
sertations & Theses A&I; ProQuest Dissertations & Theses Global. (0535062)
�Fiedler, F. E., & Mahar, L. (1979). The effectiveness of contingency model training: A review of the validation of leader match. Personnel Psychology, 32, 45– 62. http://dx.doi.org/10.1111/j.1744-6570.1979 .tb00468.x
�Fiedler, F. E., & Mahar, L. (1979). A field experiment validating contin- gency model leadership training. Journal of Applied Psychology, 64, 247–254. http://dx.doi.org/10.1037/0021-9010.64.3.247
�Finn, F. (2007). Leadership development through executive coaching: The effects on leaders’ psychological states and transformational leadership behavior (Unpublished doctoral dissertation). Queensland University of Technology, Brisbane, Queensland, Australia.
�Fitzgerald, S., & Schutte, N. S. (2010). Increasing transformational lead- ership through enhancing self-efficacy. Journal of Management Devel- opment, 29, 495–505. http://dx.doi.org/10.1108/02621711011039240
Ford, J. K., Smith, E. M., Weissbein, D. A., Gully, S. M., & Salas, E. (1998). Relationships of goal orientation, metacognitive activity, and practice strategies with learning outcomes and transfer. Journal of Ap- plied Psychology, 83, 218 –233. http://dx.doi.org/10.1037/0021-9010.83 .2.218
�Frazier, L. (1979). An investigation of the effect of management training in a minority social action program (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (DP10618)
�Frese, M., Beimel, S., & Schoenborn, S. (2003). Action training for charismatic leadership: Two evaluations of studies of a commercial training module on inspirational communication of a vision. Personnel Psychology, 56, 671– 698. http://dx.doi.org/10.1111/j.1744-6570.2003 .tb00754.x
�Frost, D. E. (1986). A test of situational engineering for training leaders. Psychological Reports, 59, 771–782. http://dx.doi.org/10.2466/pr0.1986 .59.2.771
Garrison, D. R. (2011). E-Learning in the 21st century: A framework for research and practice (2nd ed.). London, UK: Routledge/Taylor and Francis.
�George, V. M. (1999). An organizational case study of shared leadership development in nursing (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (9947903).
Gephart, R. P. (2002). Introduction to the brave new workplace: Organi- zational behavior in the electronic age. Journal of Organizational Be- havior, 23, 327–344. http://dx.doi.org/10.1002/job.143
�Gerstein, L. H., Eichenhofer, D. J., Bayer, G. A., Valutis, W., & Jankowski, J. (1989). EAP referral training and supervisors’ beliefs about troubled workers. Employee Assistance Quarterly, 4, 15–30. http://dx.doi.org/10.1300/J022v04n04_02
Giangreco, A., Sebastiano, A., & Peccei, R. (2009). Trainees reaction to training: Analysis of the factors affecting overall satisfaction with train- ing. International Journal of Human Resource Management, 20, 96 – 111. http://dx.doi.org/10.1080/09585190802528417
Gibler, D., Carter, L., & Goldsmith, M. (2000). Best practices in leadership development handbook. San Francisco, CA: Jossey-Bass.
�Gist, M. E. (1989). The influence of training method on self-efficacy and idea generation among managers. Personnel Psychology, 42, 787– 805. http://dx.doi.org/10.1111/j.1744-6570.1989.tb00675.x
Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.
Goldsmith, K., & Amir, O. (2010). Can uncertainty improve promotions? Journal of Marketing Research, 47, 1070 –1077. http://dx.doi.org/10 .1509/jmkr.47.6.1070
Goldsmith, M., Lyons, L., & Freas, A. (Eds.). (2000). Coaching for leadership, San Francisco, CA: Jossey-Bass.
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1710 LACERENZA, REYES, MARLOW, JOSEPH, AND SALAS
Goldstein, I. L. (1980). Training in work organizations. Annual Review of Psychology, 31, 229 –272. http://dx.doi.org/10.1146/annurev.ps.31 .020180.001305
Goldstein, I. L. (1986). Training in organizations: Needs assessment, development, and evaluation (2nd ed.). Monterey, CA: Brooks/Cole.
Goldstein, I. L., & Ford, J. K. (2002). Training in organizations: Needs assessment, development, and evaluation (4th ed.). Belmont, CA: Wads- worth.
Goleman, D. (1998). Working with emotional intelligence. New York, NY: Bantam Books.
�Grady, M. W. (2014). Adoption of cultural competence levels within a leadership development program: A quantitative case study (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3610255)
�Green, E. (2002). The influence of individual and work environment characteristics on trainee motivation and training effectiveness mea- sures (Unpublished doctoral dissertation). North Carolina State Univer- sity, Raleigh, NC.
Greguras, G. J., & Robie, C. (1998). A new look at within-source interrater reliability of 360-degree feedback ratings. Journal of Applied Psychol- ogy, 83, 960 –968. http://dx.doi.org/10.1037/0021-9010.83.6.960
�Grizzard, T. (2008). The impact of instructional leadership on school climate: A model for principal and teacher improvement (Doctoral dissertation). Retrieved from Dissertation Abstracts International Sec- tion A: Humanities and Social Sciences; ProQuest Information & Learn- ing. (AAI3277896)
Grönroos, C. (1982). An applied service marketing theory. European Journal of Marketing, 16, 30 – 41. http://dx.doi.org/10.1108/EUM0000000004859
�Gruenfeld, L. W. (1966). Management development effect on changes in values. Training and Development Journal, 20, 18 –26.
Guinn, S. L. (1999). Executive development—Why successful executives continue to change. Career Development International, 4, 240 –243. http://dx.doi.org/10.1108/13620439910270634
Gurdjian, P., Halbeisen, T., & Lane, K. (2014). Why leadership- development programs fail. Retrieved from http://www.mckinsey.com/ global-themes/leadership/why-leadership-development-programs-fail
�Haccoun, R. R., & Hamtiaux, T. (1994). Optimizing knowledge tests for inferring learning acquisition levels in single group training evaluation designs: The internal referencing strategy. Personnel Psychology, 47, 593– 604.
�Hall, B. (1995). Leader traits and behavior as determinants of leadership effectiveness in a military academy (Doctoral dissertation). Retrieved from Dissertation Abstracts International, Section B: The Sciences and Engineering; ProQuest Information & Learning. (AAM9519563)
Hall, D. T. (1986). Dilemmas in linking succession planning to individual executive learning. Human Resource Management, 25, 235–265. http:// dx.doi.org/10.1002/hrm.3930250206
�Hamilton, T. A., & Cooper, C. (2001). The impact of outdoor manage- ment development (OMD) programmes. Leadership and Organization Development Journal, 22, 330 –340. http://dx.doi.org/10.1108/ EUM0000000006163
�Hand, H. H., & Slocum, J. W. (1972). A longitudinal study of the effects of a human relations training program on managerial effectiveness. Journal of Applied Psychology, 56, 412– 417. http://dx.doi.org/10.1037/ h0033592
�Haraway, D. L., & Haraway, W. M., III. (2005). Analysis of the effect of conflict-management and resolution training on employee stress at a healthcare organization. Hospital Topics, 83, 11–17. http://dx.doi.org/10 .3200/HTPS.83.4.11-18
�Harris, E. F., & Fleishman, E. A. (1955). Human relations training and the stability of leadership patterns. Journal of Applied Psychology, 39, 20 –25. http://dx.doi.org/10.1037/h0046585
�Harris, M. B., von Keudell, A., McMahon, G., & Bierer, B. (2014). Physician self-assessment of leadership skills. Physician Executive, 40, 30 –36.
Harvard Business Publishing. (2013). Harvard Business Publishing Corporate Learning delivers enhanced functionality in new version of leadership direct. Retrieved from http://www.trainingindustry.com/leadership/ press-releases/harvard-business-publishing-corporate-learning-delivers- enhanced-functionality-in-new-version-of-leadership-direct.aspx
�Hassan, R., Fuwad, S., & Rauf, A. (2010). Pre-training motivation and the effectiveness of transformational leadership training: An experiment. Academy of Strategic Management Journal, 9, 1– 8.
�Hawley, S. R., St Romain, T., Rempel, S. L., Orr, S. A., & Molgaard, C. A. (2012). Generating social capital through public health leadership training: A six-year assessment. Health Education Research, 27, 671– 679. http://dx.doi.org/10.1093/her/cyr037
Hazucha, J. F., Hezlett, S. A., & Schneider, R. J. (1993). The impact of 360-degree feedback on management skills development. Human Re- source Management, 32, 325–351. http://dx.doi.org/10.1002/hrm .3930320210
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press.
�Hill, S. A. (1992). Leadership development training for principals: Its initial impact on school culture (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (9307288)
Hintzman, D. L. (1974). Theoretical implications of the spacing effect. In R. L. Solso & R. L. Solso (Eds.), Theories in cognitive psychology: The Loyola Symposium (pp. 77–99). Oxford, UK: Erlbaum.
Ho, M. (2016). Investment in learning increases for fourth straight year. Retrieved from https://www.td.org/Publications/Magazines/TD/TD- Archive/2016/11/Investment-in-Learning-Increases-for-Fourth-Straight- Year
Hodgkinson, G. P., & Rousseau, D. M. (2009). Bridging the rigour– relevance gap in management research: It’s already happening! Journal of Management Studies, 46, 534 –546. http://dx.doi.org/10.1111/j.1467- 6486.2009.00832.x
Hogan, R., & Warrenfeltz, R. (2003). Educating the modern manager. Academy of Management Learning & Education, 2, 74 – 84. http://dx .doi.org/10.5465/AMLE.2003.9324043
�House, D. A. (2001). Evaluating a new manager management training program (Doctoral dissertation). Retrieved from ABI/INFORM Com- plete; ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (1403972)
�House, R. J., & Tosi, H. (1963). An experimental evaluation of a man- agement training program. Academy of Management Journal, 6, 303– 315. http://dx.doi.org/10.2307/255156
�Huckaby, G. C. (1998). The effectiveness of personality feedback in facilitating change among mid-level managers: A study of the Myers- Briggs type indicator (Unpublished doctoral dissertation). University of Mississippi, Oxford, MS.
Hughes, A. M., Gregory, M. E., Joseph, D. L., Sonesh, S. C., Marlow, S. L., Lacerenza, C. N., . . . Salas, E. (2016). Saving lives: A meta- analysis of team training in healthcare. Journal of Applied Psychology, 101, 1266 –1304. http://dx.doi.org/10.1037/apl0000120
Hunsaker, P. (1973). Incongruity adaptation capability and accomplish- ment in leadership training for turbulent environments. Presentation at Academy of Management Proceedings, United States Army.
Hunter, J. E., & Schmidt, F. L. (1990). Dichotomization of continuous variables: The implications for meta-analysis. Journal of Applied Psy- chology, 75, 334 –349. http://dx.doi.org/10.1037/0021-9010.75.3.334
Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Cor- recting error and bias in research findings. Thousand Oaks, CA: Sage http://dx.doi.org/10.4135/9781412985031
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1711LEADERSHIP TRAINING: A META-ANALYSIS
�Ibbetson, A., & Newell, S. (1998). Outdoor management development: The mediating effect of the client organisation. International Journal of Training and Development, 2, 239 –258. http://dx.doi.org/10.1111/1468- 2419.00052
Iles, P., & Preece, D. (2006). Developing leaders or developing leadership? The Academy of Chief Executives’ programmes in the North East of England. Leadership, 2, 317–340. http://dx.doi.org/10.1177/ 1742715006066024
�Ivancevich, J. M. (1979). Longitudinal study of the effects of rater training on psychometric error in ratings. Journal of Applied Psychology, 64, 502–508. http://dx.doi.org/10.1037/0021-9010.64.5.502
�Ivancevich, J. M. (1982). Subordinates’ reactions to performance ap- praisal interviews: A test of feedback and goal-setting techniques. Jour- nal of Applied Psychology, 67, 581–587. http://dx.doi.org/10.1037/0021- 9010.67.5.581
�Ivancevich, J. M., & Smith, S. V. (1981). Goal setting interview skills training: Simulated and on-the-job analyses. Journal of Applied Psychol- ogy, 66, 697–705. http://dx.doi.org/10.1037/0021-9010.66.6.697
�Jameson, C. (2010). The impact of training in transformational leadership on the productivity of a dental practice (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3409492)
Janiszewski, C., Noel, H., & Sawyer, A. G. (2003). A meta-analysis of the spacing effect in verbal learning: Implications for research on advertis- ing repetition and consumer memory. The Journal of Consumer Re- search, 30, 138 –149. http://dx.doi.org/10.1086/374692
�Jay, A. (2002). The effects of a leadership-development program on the performance of upper-level sales managers in a Fortune 1000 company (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3067087)
Jones, J. L., & Mehr, S. L. (2007). Foundations and assumptions of the scientist-practitioner model. American Behavioral Scientist, 50, 766 – 771. http://dx.doi.org/10.1177/0002764206296454
�Jorgensen, L. I., & Els, B. (2013). Efficacy evaluation of a leadership development assessment centre for managers. Journal of Psychology in Africa, 23, 113–118.
Kalinoski, Z. T., Steele-Johnson, D., Peyton, E. J., Leas, K. A., Steinke, J., & Bowling, N. A. (2013). A meta-analytic evaluation of diversity training outcomes. Journal of Organizational Behavior, 34, 1076 –1104. http://dx.doi.org/10.1002/job.1839
�Katzenmeyer, M. H. (1988). An evaluation of behavior modeling training designed to improve selected skills of educational managers (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (8822453)
�Kawakami, N., Kobayashi, Y., Takao, S., & Tsutsumi, A. (2005). Effects of web-based supervisor training on supervisor support and psycholog- ical distress among workers: A randomized controlled trial. Preventive Medicine, 41, 471– 478. http://dx.doi.org/10.1016/j.ypmed.2005.01.001
�Kawakami, N., Takao, S., Kobayashi, Y., & Tsutsumi, A. (2006). Effects of web-based supervisor training on job stressors and psychological distress among workers: A workplace-based randomized controlled trial. Journal of Occupational Health, 48, 28 –34. http://dx.doi.org/10.1539/ joh.48.28
�Kehr, H. M. (2003). Goal conflicts, attainment of new goals, and well- being among managers. Journal of Occupational Health Psychology, 8, 195–208. http://dx.doi.org/10.1037/1076-8998.8.3.195
Keith, N., & Frese, M. (2008). Effectiveness of error management training: A meta-analysis. Journal of Applied Psychology, 93, 59 – 69. http://dx .doi.org/10.1037/0021-9010.93.1.59
�Kelloway, E. K., Barling, J., & Helleur, J. (2000). Enhancing transforma- tional leadership: The roles of training and feedback. Leadership and Organization Development Journal, 21, 145–149. http://dx.doi.org/10 .1108/01437730010325022
Kelly, D. (2012). Why people hate training, and how to overcome it. Retrieved from https://www.mindflash.com/blog/2012/03/whypeople- hate-training-and-how-to-overcome-it/
�Kerin, R. (2010). The process of change in leadership development: Using 360-degree feedback to study the roles of reflection, planning, and support (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3412171)
�Khan, H. (2002). Effectiveness of a strategic management development program. Applied H. R. M. Research, 7, 49 –52.
Khan, U., & Dhar, R. (2010). Price-framing effects on the purchase of hedonic and utilitarian bundles. Journal of Marketing Research, 47, 1090 –1099. http://dx.doi.org/10.1509/jmkr.47.6.1090
�Kim, S. H. (2003). An examination of action learning as a method for developing transformational leadership behaviors and characteristics (Doctoral dissertation). Retrieved from ABI/INFORM Complete; Pro- Quest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3099672)
Kirkpatrick, D. (1959). Techniques for evaluating training programs. Jour- nal of the American Society for Training and Development, 13, 3–9.
Kirkpatrick, D. L. (1994). Evaluating training programs: The four levels. San Francisco, CA: Berrett-Koehler. (Original work published 1959)
Kirkpatrick, D. (1996). Great ideas revisited. Training & Development, 50, 54 –59.
Klein, K. J., & Ziegert, J. C. (2004). Leader development and change over time: A conceptual integration and exploration of research challenges. In D. V. Day, S. J. Zaccaro, & S. M. Halpin (Eds.), Leader development for transforming organizations: Growing leaders for tomorrow (pp. 359 – 382). Mahwah, NJ: Erlbaum, Inc.
Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254 –284. http://dx.doi.org/10.1037/0033-2909.119.2.254
Kluger, A. N., & DeNisi, A. (1998). Feedback interventions: Toward the understanding of a double-edged sword. Current Directions in Psycho- logical Science, 7, 67–72. http://dx.doi.org/10.1111/1467-8721 .ep10772989
�Knox, D. W. (2000). The effect of leadership training on manufacturing productivity of informal leaders (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3019193).
�Kohn, V., & Parker, T. (1972). The value of expectations in management development. Training and Development Journal, 26, 26 –30.
Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice Hall.
Kraiger, K., Ford, J. K., & Salas, E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology, 78, 311–328. http://dx.doi.org/10.1037/0021-9010.78.2.311
Krause, D. E., & Gebert, D. (2003). A comparison of assessment center practices in organizations in German-speaking regions and the United States. International Journal of Selection and Assessment, 11, 297–312. http://dx.doi.org/10.1111/j.0965-075X.2003.00253.x
�Kuchta, W. G. (1992). The impact of supervisor and peer influence on the effectiveness of performance management training. Retrieved from Pro- Quest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (304024992).
�Ladegard, G., & Gjerde, S. (2014). Leadership coaching, leader role- efficacy, and trust in subordinates. A mixed methods study assessing leadership coaching as a leadership development tool. The Leadership Quarterly, 25, 631– 646. http://dx.doi.org/10.1016/j.leaqua.2014.02.002
�Lafferty, B. D. (1998). Investigation of a leadership development pro- gram: An empirical investigation of a leadership development program (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (9826782)
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1712 LACERENZA, REYES, MARLOW, JOSEPH, AND SALAS
�Lam, S. S., & Schaubroeck, J. (2000). A field experiment testing frontline opinion leaders as change agents. Journal of Applied Psychology, 85, 987–995. http://dx.doi.org/10.1037/0021-9010.85.6.987
�Latham, G. P., & Saari, L. M. (1979). The application of social learning theory to training supervisors through behavior modeling. Journal of Applied Psychology, 64, 239 –246. http://dx.doi.org/10.1037/0021-9010 .64.3.239
�Lavoie-Tremblay, M., Anderson, M., Bonneville-Roussy, A., Drevniok, U., & Lavigne, G. L. (2012). Nurse executives’ perceptions of the executive training for research application (extra) program. Worldviews on Evidence-Based Nursing, 9, 186 –192. http://dx.doi.org/10.1111/j .1741-6787.2011.00218.x
�Lawrence, H. V., & Wiswell, A. K. (1993). Using the work group as a laboratory for learning: Increasing leadership and team effectiveness through feedback. Human Resource Development Quarterly, 4, 135– 148. http://dx.doi.org/10.1002/hrdq.3920040204
�Lawshe, C. H., Bolda, R. A., & Brune, R. L. (1959). Studies in manage- ment training evaluation: II. The effects of exposures to role playing. Journal of Applied Psychology, 43, 287–292. http://dx.doi.org/10.1037/ h0047802
�Lee, H., Spiers, J. A., Yurtseven, O., Cummings, G. G., Sharlow, J., Bhatti, A., & Germann, P. (2010). Impact of leadership development on emotional health in healthcare managers. Journal of Nursing Manage- ment, 18, 1027–1039. http://dx.doi.org/10.1111/j.1365-2834.2010 .01178.x
Lee, T. D., & Genovese, E. D. (1988). Distribution of practice in motor skill acquisition: Learning and performance effects reconsidered. Re- search Quarterly for Exercise and Sport, 59, 277–287. http://dx.doi.org/ 10.1080/02701367.1988.10609373
�Lefkowitz, J. (1972). Evaluation of a supervisory training program for police sergeants. Personnel Psychology, 25, 95–106. http://dx.doi.org/ 10.1111/j.1744-6570.1972.tb01093.x
�Leigh, J. M., Shapiro, E. R., & Penney, S. H. (2010). Developing diverse, collaborative leaders: An empirical program evaluation. Journal of Leadership & Organizational Studies, 17, 370 –379. http://dx.doi.org/10 .1177/1548051809355510
�Leimbach, G. J. (1993). The effects of vocational leadership development for individuals who participated in the Ohio vocational education lead- ership institute (Doctoral dissertation). Retrieved from ProQuest Disser- tations & Theses A&I; ProQuest Dissertations & Theses Global. (9325543)
�Leister, A., Borden, D., & Fiedler, F. E. (1977). Validation of contingency model leadership training: Leader match. Academy of Management Journal, 20, 464 – 470. http://dx.doi.org/10.2307/255420
�Leonard, H. S., & Goff, M. (2003). Leadership development as an inter- vention for organizational transformation: A case study. Consulting Psychology Journal: Practice and Research, 55, 58 – 67. http://dx.doi .org/10.1037/1061-4087.55.1.58
Leslie, J. B. (2009). The leadership gap: What you need, and don’t have, when it comes to leadership talent. Greensboro, NC: Center for Creative Leadership.
�Levy, D. D. (2013). The effect of a leadership development program on students’ self-perceptions of leadership ability (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dis- sertations & Theses Global. (3580075)
Lipman, V. (2016). Why do we spend so much developing senior leaders and so little training new managers? Retrieved from https://hbr.org/ 2016/06/why-do-we-spend-so-much-developing-senior-leaders-and-so- little-training-new-managers
�Long, C. A. (2007). An evaluation of performance as it relates to lead- ership training in the United States Coast Guard (Doctoral dissertation) Retrieved from ProQuest Dissertations & Theses. (3279247)
Lord, R. G., & Brown, D. J. (2004). Leadership processes and follower self-identity. Mahwah, NJ: Erlbaum.
�Luckett, M. T. (2005). The effects of leadership grid training on trans- formational leadership in a fortune 500 company (Doctoral dissertation) Retrieved from ProQuest Dissertations & Theses. (3161206)
�MacPhee, M., Dahinten, V. S., Hejazi, S., Laschinger, H., Kazanjian, A., McCutcheon, A., . . . O’Brien-Pallas, L. (2014). Testing the effects of an empowerment-based leadership development programme. Journal of Nursing Management, 22, 4 –15. http://dx.doi.org/10.1111/jonm.12053
Magerko, B., Wray, R. E., Holt, L. S., & Stensrud, B. (2005). Improving interactive training through individualized content and increased en- gagement. The Interservice/Industry Training, Simulation & Education Conference (I/ITSEC), 2005, 1–11.
�Mahoney, T. A., Jerdee, T. H., & Korman, A. (1960). An experimental evaluation of management development. Personnel Psychology, 13, 81–98. http://dx.doi.org/10.1111/j.1744-6570.1960.tb01520.x
�Marson, P. P. (1987). A study of behavior modeling in management training– A case study (Doctoral dissertation) Retrieved from ProQuest Dissertations & Theses. (8808077)
�Martineau, J. W. (1995). A contextual examination of the effectiveness of a supervisory skills training program (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses. (9600217)
�Mason, C., Griffin, M., & Parker, S. (2014). Transformational leadership development: Connecting psychological and behavioral change. Lead- ership and Organization Development Journal, 35, 174 –194. http://dx .doi.org/10.1108/LODJ-05-2012-0063
Maurer, T. J. (2002). Employee learning and development orientation: Toward an integrative model of involvement in continuous learning. Human Resource Development Review, 1, 9 – 44. http://dx.doi.org/10 .1177/1534484302011002
�May, G., & Dubois, P. (1963). Measurement of gain in leadership training. Educational and Psychological Measurement, 23, 23–31. http://dx.doi .org/10.1177/001316446302300103
�May, G. L., & Kahnweiler, W. M. (2000). The effect of a mastery practice design on learning and transfer in behavior modeling training. Personnel Psychology, 53, 353–373. http://dx.doi.org/10.1111/j.1744-6570.2000 .tb00205.x
McCauley, C. D., & Van Velsor, E. (2004). Our view of leadership development. In K. M. Hannum, J. W. Martineau, & C. Reinelt (Eds.), Handbook of leadership development (2nd ed., pp. 1–22). San Francisco, CA: Jossey-Bass.
McCormick, M. J. (2000). The influence of goal-orientation and sex-role identity on the development of leadership self-efficacy during a training intervention (Doctoral dissertation). Retrieved from ProQuest Disserta- tions & Theses. (9957488)
�McCoy, R. L. (2009). The impact of a simulation based leader develop- ment training program on transformational and transactional leader- ship behavior (Doctoral dissertation). Retrieved from ProQuest Disser- tations & Theses. (3399839)
�McCullough, H. F. (1981). Role playing versus case studies in interper- sonal skills training: Effects on the attitudes of middle level managers, their superiors, and subordinates (Doctoral dissertations). Retrieved from ProQuest Dissertations & Theses. (8201246)
�McGehee, W., & Gardner, J. E. (1955). Supervisory training and attitude change. Personnel Psychology, 8, 449 – 460. http://dx.doi.org/10.1111/j .1744-6570.1955.tb01222.x
�McGibben, L. W. (1994). Evaluating coaching skills training through subordinate’s view of organizational climate and managerial skills (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (1358204)
�McMahon, M., & Simons, R. (2004). Supervision training for professional counselors: An exploratory study. Counselor Education and Supervi- sion, 43, 301–309. http://dx.doi.org/10.1002/j.1556-6978.2004 .tb01854.x
�Merritt, P. E. (2003). Changing the future: The effects of a conflict management seminar on nurse managers in long-term care (Doctoral
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1713LEADERSHIP TRAINING: A META-ANALYSIS
dissertation). Retrieved from ProQuest Dissertations & Theses. (3108713)
�Meyer, C. (1981). Organizational development in the public sector: A re-educative intervention (Doctoral dissertation). Retrieved from Pro- Quest Dissertations and Theses, 0 –1.
Meyer, G. F., Wong, L. T., Timson, E., Perfect, P., & White, M. D. (2012). Objective fidelity evaluation in multisensory virtual environments: Au- ditory cue fidelity in flight simulation. PLoS One, 7, e44381. http://dx .doi.org/10.1371/journal.pone.0044381
Mezirow, J., & Taylor, E. (Eds.). (2009). Transformative learning in action: A handbook of practice. San Francisco, CA: Jossey-Bass.
�Miller, H. A., Watkins, R. J., & Webb, D. (2009). The use of psycholog- ical testing to evaluate law enforcement leadership competencies and development. Police Practice and Research, 10, 49 – 60. http://dx.doi .org/10.1080/15614260802128575
�Miner, J. B. (1960). The effect of a course in psychology on the attitudes of research and development supervisors. Journal of Applied Psychol- ogy, 44, 224 –232. http://dx.doi.org/10.1037/h0047654
�Moffie, D. J., Calhoon, R., & O’Brien, J. K. (1964). Evaluation of a management development program. Personnel Psychology, 17, 431– 440. http://dx.doi.org/10.1111/j.1744-6570.1964.tb00078.x
Morgan, J. (2015). Why giving managers leadership training is a waste of time. Retrieved from http://www.forbes.com/sites/jacobmorgan/2015/ 10/01/why-giving-managers-leadership-training-is-a-waste-of-time/ #441d1e0141a5
�Morin, L., & Latham, G. (2000). The effect of mental practice and goal setting as a transfer of training intervention on supervisors’ self-efficacy and communication skills: An exploratory study. Applied Psychology, 49, 566 –578. http://dx.doi.org/10.1111/1464-0597.00032
Morris, S. B., & DeShon, R. P. (1997). Correcting effect sizes computed from factor analysis of variance for use in meta-analysis. Psychological Methods, 2, 192–199. http://dx.doi.org/10.1037/1082-989X.2.2.192
Morris, S. B., & DeShon, R. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7, 105–125. http://dx.doi.org/10.1037/1082- 989X.7.1.105
�Morton, D. L. (1989). Toward a methodology for the evaluation of management training as a means to improve productivity (Doctoral dissertation). Retrieved from ABI/INFORM Complete; ProQuest Dis- sertations & Theses A&I; ProQuest Dissertations & Theses Global. (3090090)
�Moses, J. L., & Ritchie, R. J. (1976). Supervisory relationships training: A behavioral evaluation of a behavior modeling program. Personnel Psychology, 29, 337–343. http://dx.doi.org/10.1111/j.1744-6570.1976 .tb00417.x
�Mullen, J. E., & Kelloway, E. K. (2009). Safety leadership: A longitudinal study of the effects of transformational leadership on safety outcomes. Journal of Occupational and Organizational Psychology, 82, 253–272. http://dx.doi.org/10.1348/096317908X325313
Myatt, M. (2012). The # 1 reason leadership development fails. Retrieved from http://www.forbes.com/sites/mikemyatt/2012/12/19/the-1-reason- leadership-development-fails/#546bde6234ce
�Myerchin, T. S. (1980). The comparative effectiveness of two short-term supervisory management development courses: Individualized and mo- tivation theory based vs nonindividualized and traditional in a U.S. defense department organization (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (8106834)
�Myrtle, R. C. (1991). The evaluation of a multiyear management devel- opment program in a third world setting. Human Resource Development Quarterly, 2, 129 –141. http://dx.doi.org/10.1002/hrdq.3920020206
Nadler, D. A. (1977). Feedback and organization development: Using data-based methods. Reading, MA: Addison Wesley.
�Nakamura, Y. T. (2010). Global organizational leaders’ social capital formation: A case study (Doctoral dissertation). Retrieved from Pro- Quest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3424918)
�Neck, C. P., & Manz, C. C. (1996). Thought self-leadership: The impact of mental strategies training on employee cognition, behavior, and affect. Journal of Organizational Behavior, 17, 445– 467. http://dx.doi .org/10.1002/(SICI)1099-1379(199609)17:5�445::AID-JOB770�3.0 .CO;2-N
Nelson, C. (2016). Ditch your corporate leadership training program: Launch a Shakespeare book club instead. Retrieved from http://www .forbes.com/sites/christophernelson/2016/04/06/ditch-your-corporate- leadership-training-program-launch-a-shakespeare-book-club-instead/ #1946b8f539d6
�Nemeroff, W. F., & Cosentino, J. (1979). Utilizing feedback and goal setting to increase performance appraisal interviewer skills of managers. Academy of Management Journal, 22, 566 –576. http://dx.doi.org/10 .2307/255745
�Nielsen, K., Randall, R., & Christensen, K. B. (2010). Does training managers enhance the effects of implementing team-working? A longi- tudinal, mixed methods field study. Human Relations, 63, 1719 –1741.
�Nieminen, L. R., Smerek, R., Kotrba, L., & Denison, D. (2013). What does an executive coaching intervention add beyond facilitated multi- source feedback? Effects on leader self-ratings and perceived effective- ness. Human Resource Development Quarterly, 24, 145–176. http://dx .doi.org/10.1002/hrdq.21152
�Niska, J. M. (1991). Examination of a cooperative learning supervision training and development model (Unpublished doctoral dissertation). Iowa State University, Ames, IA.
Nunnally, J. C. (1978). Psychometric theory. New York, NY: McGraw- Hill.
�O’Brien, K. A. (1988). Improving managerial self-assessment: A field study (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses. (8817142)
�O’Dwyer, M., & Ryan, E. (2002). Management development-a model for retail business. Journal of European Industrial Training, 26, 420 – 429. http://dx.doi.org/10.1108/03090590210451515
O’Leonard, K. (2014). The corporate learning factbook: Benchmarks, trends and analysis of the U.S. training market. Oakland, CA: Bersin & Associates.
�Olivero, G., Bane, K. D., & Kopelman, R. E. (1997). Executive coaching as a transfer of training tool: Effects on productivity in a public agency. Public Personnel Management, 26, 461– 469. http://dx.doi.org/10.1177/ 009102609702600403
�Olson, L. G. (2005). A study of leadership development in the regional institute for health and environmental leadership (Doctoral dissertation). Retrieved from ABI/INFORM Complete; ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3177290)
�Orpen, C. (1985). The effects of behaviour modelling training on mana- gerial attitudes and performance: A field experiment. International Jour- nal of Manpower, 6, 21–24. http://dx.doi.org/10.1108/eb045029
�Osborne, D. J. (2012). In search of a universal transformational leadership development training program for diverse leaders: The efficacy of the Dale Carnegie Course. Dissertation Abstracts International Section A, 73, 1096.
�Osborne, J. E. (2003). The impact of training on leadership development (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3149741)
�Ostroff, C. (1991). Training effectiveness measures and scoring schemes: A comparison. Personnel Psychology, 44, 353–374. http://dx.doi.org/10 .1111/j.1744-6570.1991.tb00963.x
Paas, F., Renkl, A., & Sweller, J. (2004). Cognitive load theory: Instruc- tional implications of the interaction between information structures and
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1714 LACERENZA, REYES, MARLOW, JOSEPH, AND SALAS
cognitive architecture. Instructional Science, 32, 1– 8. http://dx.doi.org/ 10.1023/B:TRUC.0000021806.17516.d0
�Papa, M. J., & Graham, E. E. (1991). The impact of diagnosing skill deficiencies and assessment-based communication training on manage- rial performance. Communication Education, 40, 368 –384. http://dx.doi .org/10.1080/03634529109378861
�Parry, K. W., & Sinha, P. N. (2005). Researching the trainability of transformational organizational leadership. Human Resource Devel- opment International, 8, 165–183. http://dx.doi.org/10.1080/ 13678860500100186
�Parsons, M. B., & Reid, D. H. (1995). Training residential supervisors to provide feedback for maintaining staff teaching skills with people who have severe disabilities. Journal of Applied Behavior Analysis, 28, 317–322. http://dx.doi.org/10.1901/jaba.1995.28-317
Patel, L. (2010). ASTD state of the industry report 2010. Alexandria, VA: American Society for Training & Development.
�Perez, J., & Mirabella, J. (2013). The relationship between leadership practices and restaurant employee turnover (Doctoral dissertation) Re- trieved from ProQuest Dissertations & Theses A&I; ProQuest Disserta- tions & Theses Global. (3533748)
�Petersen, P. B. (1972). Leadership training. Training and Development Journal, 26, 38 – 42.
�Pfeifer, L. (2004). A comparison of the effectiveness of two training methodologies in the development of management soft skills (Doctoral dissertation). Retrieved from Dissertation Abstracts International Sec- tion A: Humanities and Social Sciences; ProQuest Information & Learn- ing. (AAI3131574)
�Phelps, R. (2005). Using a formal mentoring program to develop nurse leaders: An action research study (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3174531)
Piaget, J. (1952). The origin of intelligence in children. New York, NY: Norton.
�Porras, J. I., Hargis, K., Patterson, K. J., Maxfield, D. G., Roberts, N., & Bies, R. J. (1982). Modeling-based organizational development: A lon- gitudinal assessment. The Journal of Applied Behavioral Science, 18, 433– 446. http://dx.doi.org/10.1177/002188638201800405
�Posner, C. S. (1982). An inferential analysis measuring the cost effective- ness of training outcomes of a supervisory development program using four selected economic indices: An experimental study in a public agency setting (Doctoral dissertation). Retrieved from ProQuest Disser- tations & Theses A&I; ProQuest Dissertations & Theses Global. (8219438)
Powell, K. S., & Yalcin, S. (2010). Managerial training effectiveness: A meta-analysis 1952–2002. Personnel Review, 39, 227–241. http://dx.doi .org/10.1108/00483481011017435
Powers, W. T. (1973). Feedback: Beyond behaviorism. Science, 179, 351–356. http://dx.doi.org/10.1126/science.179.4071.351
�Probst, M. B. (2011). An analysis of leadership frame preference of academic administration: Using the Bolman and deal four frame model (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3450562)
�Rappe, C., & Zwick, T. (2007). Developing leadership competence of production unit managers. Journal of Management Development, 26, 312–330. http://dx.doi.org/10.1108/02621710710740084
Ray, J. W., & Shadish, W. R. (1996). How interchangeable are different estimators of effect size? Journal of Consulting and Clinical Psychol- ogy, 64, 1316 –1325. http://dx.doi.org/10.1037/0022-006X.64.6.1316
�Reaves, W. M. (1993). The impact of leadership training on leadership effectiveness (Doctoral dissertation). Retrieved from ProQuest Disserta- tions & Theses A&I; ProQuest Dissertations & Theses Global. (9320820)
Rehmann, A., Mitman, R., & Reynolds, M. (1995). A handbook of flight simulation fidelity requirements for human factors research. Tech. Rep.
No. DOT/FAA/ CT-TN95/46. Wright-Patterson AFB, OH: Crew Systems Ergonomics Information Analysis Center.
�Reid, D. H., Rotholz, D. A., Parsons, M. B., Morris, L., Braswell, B. A., Green, C. W., & Schell, R. M. (2003). Training human service super- visors in aspects of PBS evaluation of a statewide, performance-based program. Journal of Positive Behavior Interventions, 5, 35– 46. http:// dx.doi.org/10.1177/10983007030050010601
�Renaud, D. B. (2008). The effect of leadership development programs on performance: Targeted programming for nurse leaders (Doctoral dis- sertation). Retrieved from ProQuest Dissertations & Theses A&I; Pro- Quest Dissertations & Theses Global; ProQuest Social Sciences Pre- mium Collection. (3388307)
�Reynolds, J. C. (2012). The effect of a strengths-oriented approach to leadership development on the psychological capital and authentic leadership capacities of leaders in faith-based higher education institu- tions (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3548564)
�Rice, A. A. (2012). Evaluation of the impact of a large corporate leader- ship development course. Dissertation Abstracts International Section A, 72, 2594.
�Richardson, L. M. (2009). The effect of a school-year-long in-service leadership development grow-your-own program on new and veteran assistant principals’ perceived leadership effectiveness. Dissertation Ab- stracts International Section A, 70, 1872.
�Richmer, H. R. (2011). An analysis of the effects of enneagram-based leader development on self-awareness: A case study at a Midwest utility company (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses. (3629159)
Riggio, R. E. (2008). Leadership development: The current state and future expectations. Consulting Psychology Journal: Practice and Research, 60, 383–392. http://dx.doi.org/10.1037/1065-9293.60.4.383
�Rowland-Jones, R. (2012). The EFQM concepts of excellence approach to management development within the UAE healthcare industry utilizing action modalities. Human Resource Development International, 15, 501–514. http://dx.doi.org/10.1080/13678868.2012.721988
�Roy, S., & Dolke, A. (1971). Evaluation of a supervisory training pro- gram. Training and Development Journal, 25, 35–39.
�Russell, J. S., Wexley, K. N., & Hunter, J. E. (1984). Questioning the effectiveness of behavior modeling training in an industrial setting. Personnel Psychology, 37, 465– 481. http://dx.doi.org/10.1111/j.1744- 6570.1984.tb00523.x
Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68 –78. http://dx.doi.org/10.1037/0003- 066X.55.1.68
Salas, E., Benishek, L., Coultas, C., Dietz, A., Grossman, R., Lazzara, E., & Oglesby, J. (2015). Team training essentials: A research-based guide. New York, NY: Routledge.
Salas, E., & Cannon-Bowers, J. A. (2000). Design training systematically. In E. A. Locke (Ed.), The Blackwell handbook of principles of organi- zational behavior (pp. 43–59). Malden, MA: Blackwell.
Salas, E., Tannenbaum, S. I., Kraiger, K., & Smith-Jentsch, K. A. (2012). The science of training and development in organizations: What matters in practice. Psychological Science in the Public Interest, 13, 74 –101. http://dx.doi.org/10.1177/1529100612436661
�Santric Milicevic, M. M., Bjegovic-Mikanovic, V. M., Terzic-Supić, Z. J., & Vasic, V. (2011). Competencies gap of management teams in primary health care. European Journal of Public Health, 21, 247–253. http://dx .doi.org/10.1093/eurpub/ckq010
�Savan, M. (1983). The effects of a leadership training program on supervisory learning and performance (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (8412907)
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1715LEADERSHIP TRAINING: A META-ANALYSIS
Schawbel, D. (2014). Employers are demanding hard skills over soft skills, and how millennials can help. Retrieved from https://www.entrepreneur .com/article/238608
�Schulz, L., Puscas, L., Tucci, D., Woodard, C., Witsell, D., Esclamado, R., & Lee, W. (2013). Surgical training and Education in promoting professionalism: A comparative assessment of virtue-based leadership development in otolaryngology-head and neck surgery residents. Medi- cal Education Online, 18, 1– 6. http://dx.doi.org/10.3402/meo.v18i0 .22440
Schwartz, J., Bersin, J., & Pelster, B. (2014). Human Capital Trends 2014 Survey. Retrieved from http://dupress.com/articles/human-capital- trends-2014-survey-top-10-findings/
�Schwarz, F., Stilwell, W., & Scanlan, B. (1968). Effects of management development on manager behavior and subordinate perception. Training and Development Journal, 22, 38 –50.
�Seiler, S., Fischer, A., & Voegtli, S. A. (2011). Developing moral decision-making competence: A quasi-experimental intervention study in the Swiss armed forces. Ethics & Behavior, 21, 452– 470. http://dx .doi.org/10.1080/10508422.2011.622177
�Seplowin, V. M. (1972). A study of perceptions before and after a managerial development course. Comparative Group Studies, 3, 135– 158. http://dx.doi.org/10.1177/104649647200300301
�Shaffer, P. L. (1976). Evaluation of a management development method- ology. Academy of Management Proceedings, 1976, 53–57. http://dx.doi .org/10.5465/AMBPP.1976.4975533
�Shelton, M. P. (2014). Teacher leadership: Development and research based on teacher leader model standards (Doctoral dissertation). Re- trieved from ProQuest Dissertations & Theses A&I; ProQuest Disserta- tions & Theses Global. (3628634)
�Shettel, H., Clapp, D. J., & Klaus, D. J. (1963). The application of a by-pass technique to programmed instruction for managerial training (Unpublished doctoral dissertation). American Institutes for Research in the Behavioral Sciences, Pittsburgh, PA.
Sitzmann, T., Brown, K. G., Casper, W. J., Ely, K., & Zimmerman, R. D. (2008). A review and meta-analysis of the nomological network of trainee reactions. Journal of Applied Psychology, 93, 280 –295. http:// dx.doi.org/10.1037/0021-9010.93.2.280
Sitzmann, T., Kraiger, K., Stewart, D., & Wisher, R. (2006). The compar- ative effectiveness of web-based and classroom instruction: A meta- analysis. Personnel Psychology, 59, 623– 664. http://dx.doi.org/10.1111/ j.1744-6570.2006.00049.x
�Sivanathan, N., Turner, N., & Barling, J. (2005, August). Effects of transformational leadership training on employee safety performance: A quasi-experiment study. Paper presented at Academy of Management Annual Meeting, Evanston, IL.
�Skarlicki, D. P., & Latham, G. P. (1997). Leadership training in organi- zational justice to increase citizenship behavior within a labor union: A replication. Personnel Psychology, 50, 617– 633. http://dx.doi.org/10 .1111/j.1744-6570.1997.tb00707.x
�Skorupski, J. (2006). The function of context, influence and intuition in solving problems of leadership (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3206240)
�Smeltzer, L. R. (1981). The relationship between writing style and lead- ership style. Journal of Business Communication, 18, 23–32. http://dx .doi.org/10.1177/002194368101800204
�Smith, P. E. (1976). Management modeling training to improve moral and customer satisfaction. Personnel Psychology, 29, 351–359. http://dx.doi .org/10.1111/j.1744-6570.1976.tb00419.x
�Smith, R., White, P., & Montello, P. (1992). Investigation of interpersonal management training for educational administrators. The Journal of Educational Research, 85, 242–245. http://dx.doi.org/10.1080/ 00220671.1992.9941122
Smith-Jentsch, K. A., Campbell, G. E., Milanovich, D. M., & Reynolds, A. M. (2001). Measuring teamwork mental models to support training needs assessment, development, and evaluation: Two empirical studies†. Journal of Organizational Behavior, 22, 179 –194. http://dx.doi.org/10 .1002/job.88
�Sniderman, R. L. (1992). The use of SYMLOG in the evaluation of the effectiveness of a management development program (Unpublished doc- toral dissertation). California School for Professional Psychology, Los Angeles, CA.
�Steensma, H., & Groeneveld, K. (2010). Evaluating a training using the “four levels model.” Journal of Workplace Learning, 22, 319 –331. http://dx.doi.org/10.1108/13665621011053226
Sterne, J. A. C., Becker, B. J., & Egger, M. (2005). The funnel plot. In H. R. Rothstein, A. J. Sutton, & M. Borenstein (Eds.), Publication bias in meta-analysis: Prevention, assessment and adjustments (pp. 75–98). Hoboken, NJ: Wiley.
�Stewart, G. L., Carson, K. P., & Cardy, R. L. (1996). The joint effects of conscientiousness and self-leadership training on employee self-directed behavior in a service setting. Personnel Psychology, 49, 143–164. http:// dx.doi.org/10.1111/j.1744-6570.1996.tb01795.x
�Stoller, J. K., Rose, M., Lee, R., Dolgan, C., & Hoogwerf, B. J. (2004). Teambuilding and leadership training in an internal medicine residency training program. Journal of General Internal Medicine, 19, 692– 697. http://dx.doi.org/10.1111/j.1525-1497.2004.30247.x
�Stoltz, P. G. (1989). Developing communication skills through outdoor experiential leadership training: A quantitative and qualitative analysis (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (9005262)
Stremersch, S., & Tellis, G. J. (2002). Strategic bundling of products and prices: A new synthesis for marketing. Journal of Marketing, 66, 55–72. http://dx.doi.org/10.1509/jmkg.66.1.55.18455
�Stromei, L. K. (1998). An evaluation of the effectiveness of a formal mentoring program for managers, and the determinants of protégé satisfaction (Doctoral dissertation). Retrieved from ProQuest Disserta- tions & Theses. (9839225)
�Stryker, J. (2001). 360-degree feedback and its impact on leadership behavior (Doctoral dissertation). Retrieved from ProQuest Dissertation and Theses. (3029592)
�Sullivan, J. V. (1993). The effects of leadership training and organiza- tional culture on the leadership attitudes of U.S. marine corps drill instructors (Doctoral dissertation). Retrieved from ProQuest Disserta- tions & Theses A&I; ProQuest Dissertations & Theses Global. (9329317)
�Supic, Z. T., Bjegovic, V., Marinkovic, J., Milicevic, M. S., & Vasic, V. (2010). Hospital management training and improvement in managerial skills: Serbian experience. Health Policy, 96, 80 – 89. http://dx.doi.org/ 10.1016/j.healthpol.2010.01.002
�Suwandee, A. (2009). Organizational leadership development among the middle executives of Kasem Bundit University, Thailand (Doctoral dis- sertation). Retrieved from ProQuest Dissertations & Theses A&I; Pro- Quest Dissertations & Theses Global. (3349800)
Sweller, J., Van Merrienboer, J. J., & Paas, F. G. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, 251–296. http://dx.doi.org/10.1023/A:1022193728205
Tannenbaum, S. I. (2002). A strategic view of organizational training and learning. In K. Kraiger (Ed.), Creating, implementing, and maintaining effective training and development: State-of-the-art lessons for practice (pp. 10 –52). San Francisco, CA: Jossey-Bass.
�Tannenbaum, S. I., & Woods, S. B. (1992). Determining a strategy for evaluating training: Operating within organizational constraints. Human Resource Planning, 15, 63– 81.
�Taylor, P. J., Russ-Eft, D. F., & Taylor, H. (2009a). Gilding the outcome by tarnishing the past: Inflationary biases in retrospective pretests. The
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1716 LACERENZA, REYES, MARLOW, JOSEPH, AND SALAS
American Journal of Evaluation, 30, 31– 43. http://dx.doi.org/10.1177/ 1098214008328517
Taylor, P. J., Russ-Eft, D. F., & Taylor, H. (2009b). Transfer of manage- ment training from alternative perspectives. Journal of Applied Psychol- ogy, 94, 104 –121. http://dx.doi.org/10.1037/a0013006
�Teckchandani, A., & Schultz, F. C. (2014). The vision thing: An experi- ential exercise introducing the key activities performed by leaders. The Journal of Leadership Studies, 8, 63–70. http://dx.doi.org/10.1002/jls .21315
�Tharenou, P., & Lyndon, J. T. (1990). The effect of a supervisory development program on leadership style. Journal of Business and Psychology, 4, 365–373. http://dx.doi.org/10.1007/BF01125246
Tharenou, P., Saks, A. M., & Moore, C. (2007). A review and critique of research on training and organizational-level outcomes. Human Re- source Management Review, 17, 251–273. http://dx.doi.org/10.1016/j .hrmr.2007.07.004
�Thomas, T. A. (1998). The impact of training and culture on leadership values and perceptions at the United States Army Engineer School (Doctoral dissertation). Retrieved from ProQuest Dissertations & The- ses. (9821644)
�Thoms, P., & Greenberger, D. B. (1998). A test of vision training and potential antecedents to leaders’ visioning ability. Human Resource Development Quarterly, 9, 3–19. http://dx.doi.org/10.1002/hrdq .3920090102
�Tipton, V. (2003). The effectiveness of the current training practices of middle-level managers in industry as reflected in the practices of the Verizon Corporation (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3120090)
�Torp, S. (2008). How a health and safety management training program may improve the working environment in small- and medium-sized companies. Journal of Occupational and Environmental Medicine, 50, 263–271. http://dx.doi.org/10.1097/JOM.0b013e318163866f
�Tracey, J. B., Tannenbaum, S. I., & Kavanagh, M. J. (1995). Applying trained skills on the job: The importance of the work environment. Journal of Applied Psychology, 80, 239 –252. http://dx.doi.org/10.1037/ 0021-9010.80.2.239
�Trefz, M. K., & Howell, R. D. (1991). Factors associated with perceived efforts of trainees to transfer learning from a management training activity (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses. (9130573)
Tziner, A., Fisher, M., Senior, T., & Weisberg, J. (2007). Effects of trainee characteristics on training effectiveness. International Journal of Selec- tion and Assessment, 15, 167–174. http://dx.doi.org/10.1111/j.1468- 2389.2007.00378.x
�Unsworth, K. L., & Mason, C. M. (2012). Help yourself: The mechanisms through which a self-leadership intervention influences strain. Journal of Occupational Health Psychology, 17, 235–245. http://dx.doi.org/10 .1037/a0026857
U.S. Merit Systems Protection Board. (2015). Training and development for the senior executive service: A necessary investment. Washington, DC: U.S. Government Printing Office.
van Merrienboer, J. J., & Sweller, J. (2005). Cognitive load theory and complex learning: Recent developments and future directions. Educa- tional Psychology Review, 17, 147–177. http://dx.doi.org/10.1007/ s10648-005-3951-0
�Vendrell, E. G. (1998). The implementation and evaluation of a critical incident management simulation training program adapted specifically for police recruit-level training (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (9838808)
�Vicker, D. R. (1998). Relationships between interpersonal communication competence and learning in management development training with an experiential learning approach (Doctoral dissertation). ProQuest Dis-
sertations & Theses A&I; ProQuest Dissertations & Theses Global. (9829420)
�Waggoner, E. G. (2001). An experimental study of the effectiveness of traditional and adjunct management training for entry level managers at a national aerospace research laboratory (Doctoral dissertation). Re- trieved from ABI/INFORM Complete; ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3002461)
Wakefield, N., Abbatiello, A., Agarwal, D., Pastakia, K., & van Berkel, A. (2016). Leadership awakened: Generations, teams, science. Retrieved from http://dupress.com/articles/identifying-future-business-leaders- leadership/
�Wakeley, J. H., & Shaw, M. E. (1965). Management training–An inte- grated approach (Doctoral dissertation). Retrieved from ProQuest Dis- sertations & Theses A&I; ProQuest Dissertations & Theses Global. (9831047)
�Wang, C., Wei, S., Xiang, H., Wu, J., Xu, Y., Liu, L., & Nie, S. (2008). Development and evaluation of a leadership training program for public health emergency response: Results from a Chinese study. BMC Public Health, 8, 377. http://dx.doi.org/10.1186/1471-2458-8-377
�Ward, R. C. (2008). Assessing learning transfer and performance im- provement in an action learning leadership development program (Doc- toral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3327069)
�Warr, P., & Bunce, D. (1995). Trainee characteristics and the outcomes of open learning. Personnel Psychology, 48, 347–375. http://dx.doi.org/10 .1111/j.1744-6570.1995.tb01761.x
Weaver, S. J., Rosen, M. A., Salas, E., Baum, K. D., & King, H. B. (2010). Integrating the science of team training: Guidelines for continuing education. The Journal of Continuing Education in the Health Profes- sions, 30, 208 –220. http://dx.doi.org/10.1002/chp.20085
Weiss, H. M. (1990). Learning theory and industrial and organizational psychology. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (2nd ed., Vol. 1, pp. 171–221). Palo Alto, CA: Consulting Psychologists Press.
�Wellinghoff, S. G. (1990). Effects of cognitive-behavioral training for service/industrial middle managers and supervisors in thought organi- zation and problem-solving techniques (Doctoral dissertation). Re- trieved from ProQuest Dissertations & Theses A&I; ProQuest Disserta- tions & Theses Global. (9109767)
�Werle, T. (1985). The development, implementation and evaluation of a training program for middle-managers (Unpublished doctoral disserta- tion). Union Graduate School, Schenectady, NY.
�Wernsing, T. S. (2010). Leader self-awareness development: An interven- tion and test of a theoretical model (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (3390666)
Wexley, K. N., & Latham, G. P. (1991). Developing and training human resources in organizations (2nd ed.). New York, NY: Harper Collins.
Wexley, K. N., & Latham, G. P. (2002). Developing and training human resources in organizations (3rd ed.). Upper Saddle River, NJ: Prentice Hall.
�Wexley, K. N., & Nemeroff, W. F. (1975). Effectiveness of positive reinforcement and goal setting as methods of management development. Journal of Applied Psychology, 60, 446 – 450. http://dx.doi.org/10.1037/ h0076912
�Williams, A. J. (1992). The effects of police officer supervisory training on self-esteem and leadership style adaptability considering the impact of life experiences (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (9312165)
�Williams, D. E. (2014). Follower perception of leaders’ pre and post- training transformational leadership behaviors (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses A&I; ProQuest Dis- sertations & Theses Global. (3625062)
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1717LEADERSHIP TRAINING: A META-ANALYSIS
�Williams, P. V. (1999). Leadership training and evaluation in a chemical plant maintenance department: A case study (Doctoral dissertation). Retrieved from ABI/INFORM Complete; ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (1396769)
Witherspoon, R., & White, R. P. (1996). Executive coaching: A continuum of roles. Consulting Psychology Journal: Practice and Research, 48, 124 –133. http://dx.doi.org/10.1037/1061-4087.48.2.124
Wofford, J. C., Goodwin, V. L., & Whittington, J. L. (1998). A field study of a cognitive approach to understanding transformational and transac- tional leadership. The Leadership Quarterly, 9, 55– 84. http://dx.doi.org/ 10.1016/S1048-9843(98)90042-X
�Wolf, M. S. (1996). Changes in leadership styles as a function of a four-day leadership training institute for nurse managers: A perspective on continuing education program evaluation. Journal of Continuing Education in Nursing, 27, 245–252.
�Wolfe, J., & Moe, B. L. (1973). An experimental evaluation of a hospital supervisory training program. Hospital Administration, 18, 65–77.
�Woods, E. F. (1987). The effects of the center for educational adminis- trator development management training program on site administra- tors’ leadership behavior (Doctoral dissertation). Retrieved from Pro- Quest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (0561058)
Wright, P. M., McCormick, B., Sherman, W. S., & McMahan, G. C. (1999). The role of human resource practices in petro-chemical refinery performance. International Journal of Human Resource Management, 10, 551–571. http://dx.doi.org/10.1080/095851999340260
�Xu, Q. J., & Jiang, J. (2010). The moderating role of cultural similarity in leadership training effectiveness. Journal of European Industrial Train- ing, 34, 259 –269. http://dx.doi.org/10.1108/03090591011031746
Yadav, M. S., & Monroe, K. B. (1993). How buyers perceive savings in a bundle price: An examination of a bundle’s transaction value. Journal of Marketing Research, 30, 350 –358. http://dx.doi.org/10.2307/3172886
�Yirga, B. T. (2011). Effects of a basic leadership development program on participants at Dallas county community college district (Doctoral dis- sertation). Retrieved from ProQuest Dissertations & Theses A&I; Pro- Quest Dissertations & Theses Global; ProQuest Social Sciences Pre- mium Collection. (3464009)
�Yoshimura, K. (2010). Developing technical leaders in a global organi- zation: Examining the influence of culture on multi-source feedback and performance (Doctoral dissertation). Retrieved from ProQuest Disserta- tions & Theses. (3442584)
�Yost, P. R. (1996). A reconsideration of the utility of assessing trainee reactions when evaluating the effectiveness of training programs (Doc- toral dissertation). Retrieved from ProQuest Dissertations & Theses. (9719857)
�Young, D. E., & Dixon, N. M. (1995). Extending leadership development beyond the classroom: Looking at process and outcomes. Greensboro, NC: Center for Creative Leadership.
Zhang, D., Zhao, J. L., Zhou, L., & Nunamaker, J. F., Jr. (2004). Can e-learning replace classroom learning? Communications of the ACM, 47, 75–79. http://dx.doi.org/10.1145/986213.986216
Zimmerman. (2015). Why leadership industry has failed. Retrieved from https://www.gsb.stanford.edu/insights/jeffrey-pfeffer-why-leadership- industry-has-failed
�Zohar, D. (2002). Modifying supervisory practices to improve subunit safety: A leadership-based intervention model. Journal of Applied Psy- chology, 87, 156 –163. http://dx.doi.org/10.1037/0021-9010.87.1.156
Received November 20, 2015 Revision received May 9, 2017
Accepted May 9, 2017 �
E-Mail Notification of Your Latest Issue Online!
Would you like to know when the next issue of your favorite APA journal will be available online? This service is now available to you. Sign up at https://my.apa.org/portal/alerts/ and you will be notified by e-mail when issues of interest to you become available!
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
ti on
or on
e of
it s
al li
ed pu
bl is
he rs
. T
hi s
ar ti
cl e
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
1718 LACERENZA, REYES, MARLOW, JOSEPH, AND SALAS
- Leadership Training Design, Delivery, and Implementation: A Meta-Analysis
- Leadership Training Defined
- The Design, Delivery, and Implementation of Leadership Training
- Reactions
- Learning
- Transfer
- Results
- Training Design, Delivery, and Implementation: Moderator Analyses
- Training Design Characteristics
- Needs analysis
- Training attendance policy
- Spacing effect
- Trainees’ level of leadership
- Training instructor
- Training Delivery and Implementation Characteristics
- Delivery method
- Feedback
- Source of feedback
- Training location
- Training setting
- Exploratory Moderators
- Training content
- Training evaluator affiliation
- Training duration
- Publication date
- Method
- Literature Search and Inclusion Criteria
- Coding Procedures
- Evaluation criteria
- Delivery method
- Training content
- Trainees’ level of leadership
- Additional codes
- Analyses
- Results
- Discussion
- Question 1: How Effective Are Leadership Training Programs?
- Question 2: How Should One Design, Deliver, and Implement a Leadership Training Program to Maxim ...
- Does a needs analysis enhance the effectiveness of leadership training?
- Are voluntary leadership training programs more effective than mandatory programs?
- Are temporally spaced training sessions more effective than massed training sessions?
- Is leadership training more effective for low-level leaders than high-level leaders?
- Are self-administered leadership training programs less effective than those administered by som ...
- Are practice-based training delivery methods more effective than information-based and demonstra ...
- Does feedback enhance the effectiveness of leadership training? Is 360-degree feedback more effe ...
- Are on-site training programs more effective than off-site training programs?
- Are leadership training programs that are virtually based less effective than face-to-face progr ...
- Does some leadership training program content result in greater outcomes than other content?
- Do leadership training programs evaluated by a team of practitioners and academics result in gre ...
- Are longer leadership training programs more effective?
- Has the effectiveness of leadership training increased over time?
- Limitations
- Practical Implications
- Conclusion
- References
