Training program for employees and first-line supervisors.

The Leadership Quarterly 21 (2010) 633–644

Contents lists available at ScienceDirect

The Leadership Quarterly

journal homepage: www.elsevier.com/locate/leaqua

Estimating return on leadership development investment

Bruce J. Avolio a,⁎, James B. Avey b, David Quisenberry c

a Department of Management & Organization, Michael G. Foster School of Business, University of Washington, United States b Northwest Center for Organizational Research, College of Business, Central Washington University, United States c Global Leadership Institute, College of Business Administration, University of Nebraska-Lincoln, United States

a r t i c l e i n f o

⁎ Corresponding author. E-mail address: [email protected] (B.J. Av

1048-9843/$ – see front matter © 2010 Elsevier Inc. doi:10.1016/j.leaqua.2010.06.006

a b s t r a c t

Keywords:

When making capital investment decisions organizational leaders are trained to consider the financial return on investment. Yet, the same expectation typically does not exist for investments in leadership training. We suggest that decisions regarding leadership training and development ought to use a similar approach as the process leads to organizations incurring cost for an anticipated benefit, like any other investment. In the current paper, we describe how to estimate the return on leadership development investment (RODI) and the implications for measuring organizational effectiveness from such analyses. Using different guiding assumptions, scenarios, length of the intervention, and level of management participating in the leader development program, the expected return on investment from leadership development interventions ranged from a low negative RODI to over 200%.

© 2010 Elsevier Inc. All rights reserved.

Leadership impact Return on leadership development Leadership development investment

1. Introduction

“The issue today is whether America can nourish enough good leaders to forge a bright path into the 21st century.” — David Gergen, U.S. News and World Report.

As managers move further into the 21st century, there is increasing evidence that a war for leadership talent looms on the near horizon for many organizations around the globe. This war for talent may subside for short periods of time as the world economies address significant downturns, but will no doubt return as the economies begin to recover and grow. According to a 2006 survey of Human Resource (HR) leaders, the number one problem for HR directors is identifying and developing the leadership talent needed for growth and expansion of their respective organizations (Fegley, 2006). In a similar study conducted in 2007, 44% of the organizational leaders surveyed, reported that increasing the effectiveness of training as being their first or second priority (“Industry Report,” 2007). Given the combination of the perceived value of effective training coupled with a need for organizations to maintain competitive cost structures and fiscal responsibility, we propose it is important for organizational leaders to assess the extent to which they are investing in the most optimal training processes in terms of their return on leadership development investment (RODI). In other words, if organizations are developing leadership within their company through training, a technique for assessing the fiscal impact of training would be extremely useful to executive decision-makers.

We offer a strategy for examining the return on investment from leadership development initiatives. The approach we describe is called return on development investment (RODI) as it seeks to predict the financial return from investment in human development. We have two goals for focusing on RODI. Our first goal is to establish the importance of viewing investments in leadership development like other organizational investments by determining the anticipated return, as is done in the area of human capital assessment (Ulrich, Zenger, & Smallwood, 1999; Becker, Huselid, & Ulrich, 2001; Cascio & Boudreau, 2008). Our second goal is to demonstrate how one determines the RODI for leadership development interventions.

olio).

All rights reserved.

634 B.J. Avolio et al. / The Leadership Quarterly 21 (2010) 633–644

Before describing how to calculate RODI, a brief discussion of costing human behavior is necessary. Critics (e.g., Latham & Whyte, 1994) as well as proponents (e.g., Cascio, 1999; Cascio & Boudreau, 2008) of RODI methodologies point to the dynamic nature of human behavior and a seemingly unending list of potential contingencies that create complexity when applying valuation techniques. When estimating a return on any investment, there is a set of known information and a set of assumptions. We know there is a set of known unknowns, which encompass areas we suspect affect our estimates, but we do not know to what degree. Finally, there are the unknown unknowns, these are areas we have not anticipated nor included assumptions or estimates in our calculations.

We take the position that estimating the RODI for leadership development is no different than any other analysis; the more accurate one's assumptions and data are, the more accurate the estimate. Unfortunately, organizational leaders have frequently been told that it is too complex or unreliable to estimate the return on investment for leadership development leading key decision-makers away from requesting and conducting such analyses (Avolio, Sosik, Jung, & Berson, 2003). We believe this argument is not only specious, but it has significantly delayed progress in advancing the science and practice associated with leadership development. Nevertheless, we also recognize that even when assumptions are made explicit there is some potential for error in RODI estimates due to both the known unknowns and the unknown unknowns. However, the problem of known unknowns and unknown unknowns applies to all aspects of business, not just leadership development. We suggest similar to other management disciplines, the more we attempt to figure out how to best estimate the return on development the better we will be in doing so in the future. With increasing emphasis placed on human capital development in organizations, the timing could not be better for advocating that, whenever feasible, RODI be estimated for leadership development interventions.

The purpose of this article is to highlight an example of RODI for leadership development interventions using methods developed by Cascio and Boudreau (2008). We also offer empirical evidence regarding the effectiveness of leadership interventions reported in the leadership literature as the basis for estimating RODI.

At the outset, we want to be clear that we do not believe that there is a single “best” method for calculating RODI as noted in prior literature (e.g., see Flamholtz, Bullen, & Hua, 2002: Schultz, 1961). However, we do intend to suggest and show that the discipline involved in calculating RODI will be beneficial to advancing both the science and practice of leadership development.

1.1. Status of leadership development

Leadership development is the least explored topic within the field of leadership research and theory (Avolio, 2007; Day, Harrison, & Halpin, 2008). Systematic investigations of leadership development interventions are rare in the literature as are theories of leadership development (Avolio & Luthans, 2006). In fact, Avolio and Luthans (2006) report that a review of the leadership intervention literature from the last hundred years only produced 201 articles on studies examining the impact of leadership interventions, and less than half were focused on leadership development.

While a theory of leadership development has yet to be fully developed (e.g., see Avolio, 2007; Day, 2000) there have been some empirical research studies which have shown leadership development interventions can have a positive effect on the attitudes, behaviors, and performance of leaders and their followers. For example, in the first meta-analysis of published leadership development interventions the authors reported a moderately positive effect in 12 out of 17 studies (Burke & Day, 1986). Further, Burke, and Day (1986) analyzed 70 studies, published between 1952 and 1982 that focused on the effectiveness of managerial training, concluding that while managerial training was moderately effective, there was a clear need to conduct more empirical research before any firm conclusions could be drawn. It is important to note, that many of the studies included in this meta-analysis specifically focused on leadership development.

Collins and Holton (2004) extended this research when they conducted a meta-analysis of 83 intervention studies published between 1982 and 2001. Collins and Holton provided evidence to replicate earlier findings reported by Burke and Day (1986), showing that managerial training produced positive outcomes with effect sizes ranging from moderate to strong effects d=.35 to d=1.37. In their conclusions, Collins and Holton recommended that future research clarify the level of effectiveness of managerial training on organizational performance outcomes. We respond to this call by highlighting a technique that can be used to examine the effects of managerial training (leadership development specifically) in terms of dollar value return to the organization.

1.2. Investments in human resources

Evidence supporting investments in human capital, including leadership development, has been reported in prior research (Huselid, 1995; Huselid, Jackson, & Schuler, 1997). These and other more recent findings have prompted organizational leaders to think more about the tangible value of human capital. However, while many organizations invest in developing leadership, few have focused on determining whether the investment was worthwhile. Leadership development interventions appear to be something that is ‘nice to have’ when sufficient resources exist, but few organizational leaders seem to know whether it has any true value with respect to RODI (Avolio, 2005). This situation places leadership development in a precarious position.

While organizational leaders have acknowledged the importance of examining RODI, many have also reported a lack of confidence in the methods used to calculate these estimates, which may have inhibited them from making it a priority in determining whether leadership development was a worthwhile investment. Along these lines, it has been estimated that only 10 to 20% of organizations investing in leadership development ever actually evaluate the effectiveness of a leadership development program on anything approximating performance outcomes (Avolio et al., 2003).

635B.J. Avolio et al. / The Leadership Quarterly 21 (2010) 633–644

In sum, we advocate that leadership development should be analyzed like other investment decisions. Furthermore, the main purpose of this study is to demonstrate how RODI can be calculated based on local assumptions characterizing participants and their organization. In the next section, we discuss the process of calculating RODI using results from a comprehensive meta- analysis of leadership intervention studies. We follow our overview of the methods used, with a report of our findings, and conclude with a discussion of results and implications for practice.

2. Method

Return on investment (ROI) is a ubiquitous financial calculation used throughout business. In general, the cost of doing something is compared to the financial effect of doing it. The utility of calculating ROI is that organizations use these indices to make decisions regarding what investments are and are not worthy of any further pursuit solely based on financial criteria.

A vigorous debate regarding the efficacy of applying an ROI approach to investments in human development has gone on in both the research and practice literatures for decades (e.g., Flamholtz et al., 2002: Schultz, 1961). During this time period, a number of models have been developed for calculating human assets. For example, Jack Phillips' model is one that is probably familiar to leadership development researchers. Phillips' Leadership Scorecard methodology incorporates a Net Benefits over Total Costs calculation to determine ROI (Phillips, 2003). Phillip's method takes into account the entire costs of a leadership development intervention with a fixed beginning and end date. Phillips then offers a straight forward method for calculating returns for certain conditions.

For the current study, we have chosen Cascio's ROI methodology, which allows for evaluating leadership development intervention effectiveness over multiple points in time, rather than at a fixed beginning and end date. This methodology offered us the opportunity to assess the sustainability of the effects of a leadership development intervention over an extended period of time.

In “Investing in people: financial impact of human resource initiatives”, (Cascio & Boudreau, 2008) a method is presented for calculating the financial impact of HR activities, such as leadership interventions. This method includes a series of utility analyses for estimating the effect of employee training interventions, which is based on the Brogden–Cronbach–Gleser Model for costing human resources investments (Brogden, 1946, 1949; Cronbach & Gleser, 1965). Cascio and Boudreau (2008, Chapter 11) also refers to the origins of the specific formula he recommends as being based on the work of Schmidt, Hunter, and Pearlman (1982). Indeed, a critical component to his recommended strategy, termed SDy, was drawn from work by Schmidt and his colleagues as well as from the Cascio–Ramos Estimate of Performance in Dollars (CREPID) technique.

Cascio's formula is similar to other return on investment (ROI) equations in that the expected financial cost of investment (in leadership development) is subtracted from the expected financial increase from that specific investment. This number (overall increase or decrease) is then divided by the overall initial investment cost. The product is a rate of return or RODI. The data that is typically required to calculate the RODI include the number of people going through training, the costs of training, the expected effect of training and duration of that effect, as well as the estimated dollar value impact for those who have gone and not gone through the intervention.

Cascio's formula consists of the following:

RODI=NTdSDy−C

where:

N=number of participants in development intervention. T=expected time duration of change in leadership behaviors (converted to fraction in years such that a year and 6 months would be 1.5). d=effect size of intervention, also considered as the average difference in outcomes between trained participants and untrained counterparts. SDy=standard deviation of dollar valued job performance among untrained employees. When dollarized performance metrics are not available, the performance metric may be a function of 40% of annual salary. In this case, 40% of one's annual salary is a conservative estimate of that individual's dollar value to the firm in terms of performance. C=total cost of training the expected number of participants.

It is well established that metrics assessing performance impact and behavioral change, which are each core ingredients for the formula provided above, can be dynamic and influenced by many internal and external factors (Ulrich et al, 1999; Becker et al, 2001; Cascio & Boudreau, 2008). Thus, we have tried to clearly articulate the set of assumptions that were used to guide our analyses and reporting of findings, which are described in detail in the sections below. Also, although the terms “manager” and “leader” are sometimes used distinctly, in this article we use the terms interchangeably. A leadership intervention for this analysis is considered a developmental experience using some form of training, introspection, receiving feedback and exercises to increase the effectiveness of how one leads an individual or group.

The results of a recently published meta-analysis of leadership interventions served as the foundation for the present RODI analysis (Avolio, Hannah, Reichard, Chan, & Walumbwa, 2009). Findings were used to estimate 1) the average “effect size” (mathematical value of the relationship between an intervention and subsequent improvement in leadership effectiveness or performance) and 2) confidence intervals (range of effect sizes), which both represent a major part of the RODI formula. The average effect size was used as the single best point estimate for the RODI analysis. The confidence interval (95%) around that point

636 B.J. Avolio et al. / The Leadership Quarterly 21 (2010) 633–644

estimate provides a range of effect sizes that are likely to be obtained depending on the nature of the intervention. In other words, with analyses using 133 available studies one can be 95% confident that the high & low effect size estimates include the “true” average effect size of the leadership interventions used in these studies.

To account for the diminishing effects and impact of a leadership intervention over the course of time, we used 50% of the confidence interval, such that the high prediction is not as high as the meta-analysis would prescribe and the low estimate not as low, in order to avoid over or under predicting the RODI. In other words, we constrained the high and low point of the interval to provide a more conservative estimate of returns. This decision was in part influenced by arguments provided by Latham and Whyte (1994), who suggest that utility analyses often lead to outrageous ranges and claims of ROI. Thus, we set out to use a conservative estimate as a starting point for estimating the RODI of leadership development interventions. We pursued this more conservative approach, as we wanted to offer the effect sizes reported in this study for use by organizations considering investing in leadership development as representing a general indicator of the impact of leadership development interventions. Using this more conservative approach should provide a more realistic anticipated return on investment depending on what the provider has estimated the effect size to be.

A basic assumption guiding the estimation of RODI is being able to estimate the effect of one's intervention. This is analogous to determining the statistical power for a study, in that one has to have an estimate of effect size to do so. With regard to estimating statistical power in new areas of research, statisticians frequently recommend using similar research streams to estimate effect sizes. The same logic can be used in the current situation for estimating RODI. For example, if a provider does not know the estimated effect size for a new program, then it is possible to use the average effect sizes reported by Avolio and colleagues (2009) as a starting point. Specifically, Avolio and colleagues (2009) report a range of different effect sizes depending on the focus of the leader development intervention, the length of the intervention and the model underlying the intervention, which could be used to ball park the effect size for a new program, or one where the effects have not yet been estimated.

In addition to effect sizes, the effectiveness of leadership development is likely to be multi-level. Leadership involves more than one person and effects are typically diffused and cascaded to others (Berson & Avolio, 2004). For example, a CEO who improves upon his or her leadership abilities is likely to positively impact his or her direct team of VPs, who in turn may enhance the effectiveness of their direct and indirect followers as various types of performance associated with effective leadership cascades throughout an organization. Although we realize that the effects of leadership can diffuse throughout the entire organization, the present analysis takes a more conservative approach in that we will only calculate the RODI for leaders and their direct followers. The level of effectiveness was assumed to be the upper level leaders and corresponding effects were included for their immediate followers (next level leaders).

Leadership interventions can occur in a matter of hours, weeks or at various points accumulated across one's life-span. To account for this variability in terms of the timing of the intervention, we explicitly stated the amount of time that constituted the intervention. Avolio et al. (2009) reported that the interventions included in their meta-analysis lasted from one to seven days. Based on their findings we calculated RODI for a 1.5 day intervention and a 3 day intervention because organizations are more likely to consider shorter term leadership interventions.

Although leadership development interventions typically involve specific developmental targets or learning objectives, our RODI analysis is based on a generic intervention. More specifically, effect sizes were based on averages across theoretical models (e.g., transformational, traditional leadership,) based on the findings reported by Avolio et al. (2009). Therefore, these results may fluctuate depending on the effect size for the model used.

The 3 day intervention model includes an additional set of techniques and learning objectives beyond what can be accomplished in 1.5 days. Therefore, we made the assumption that the developmental effect would be greater in the 3 day intervention model as compared to the 1.5 day model.

In terms of application, it is possible that the effect of a 3 day versus 1.5 day intervention could be linear, curvilinear negative, curvilinear positive, triadic, quadratic or an exponential calculation. For example, the pattern of impact across individuals may depend on the individual's level of developmental readiness. Specifically, some individuals who are at a high level of readiness may show a rather positive and dramatic impact from the start of an intervention, while others who are less ready may take longer to evidence the impact from the intervention. Also, one might find that a shorter intervention may maintain current levels of

Table 1

Relevant assumptions and decision points underlying accompanying return on development analysis

1. Calculations based on 30 upper-level leaders 2. Calculations based on 100 mid-level leaders 3. Salary for upper-level leaders set at $100,000 4. Salary for mid-level leaders set at $70,000 5. Salary for mid-level followers set at $50,000 6. Calculations set for 1.5 day and 3 day intervention 7. Effect size doubled for twice the length of time in intervention 8. For cost structure, the hosting organization will incur cost for leadership trainer, support staff, technology, transportation, meal and hotel expenses 9. Calculations based on effect sizes drawn from overall meta-analytic results and will differ by type of intervention at demonstrated by Avolio and colleagues (2009)

10. For cost structure, employees are “worth” twice their salary in terms of potential revenue for the organization 11. Leaders and followers will be influenced by the intervention for 8 weeks

637B.J. Avolio et al. / The Leadership Quarterly 21 (2010) 633–644

leadership performance, whereas a longer intervention may show a greater effect downstream in the individual's development, producing different patterns across different individuals. For the sake of simplicity and for demonstration purposes, we assumed a linear effect resulting in twice the effect size for a 3 day versus 1.5 day intervention.

In addition to length of time, interventions can occur in a variety of ways; on-site at a company, off-site, mediated by technology (i.e. networking through interactive internet programming), or some combination of all of those. Changes in location can influence program costs (e.g., travel expenses and facility expenses) and thus influence RODI. Consequently, we calculated RODI for an on-site, off-site and technology mediated leadership intervention program using different costs assumptions for each method.

Given the calculation of RODI includes the initial cost of the training intervention, the following section describes and categorizes costs of development used in our study. Specifically, the costs were identified by interviewing a small group of leaders (approximately 10) at multiple levels (e.g. supervisors and executives) each from different organizations, including three Fortune 500 firms.

In the next section we summarize our findings. The findings are presented using the above assumptions to guide our estimates of RODI. A list of the assumptions at each decision point is included in Table 1.

3. Results

Table 2 shows the categories and projected costs of a leadership development intervention. The costs were separated into three groups reflecting three different mediums of intervention: on-site, off-site and on-line captured as overall cost per medium.

The first listed cost is ‘Time in Participant Salary’ which is the number of hours that participants are engaged in the intervention multiplied by their hourly salary. This was used to account for lost direct labor as the leader will be engaged in development and not directly working toward the organization's objectives. Based on the data collected from our interviews, we assumed a salary of $100 k for upper level leaders, $70 k for mid level leaders and $50 k for followers of mid level leaders.

The second listed cost was ‘Lost Production Time’ and is included to capture opportunity costs for individuals who directly impact revenue for the organization (e.g. sales). For example, if a sales employee attends the training, we include costs as both hourly/salary and as lost sales for that time invested in training. The conservative assumption for this analysis was twice the salary rate. For example, if an employee earned $100 per hour, then the 1.5 day training program would cost $1200 in terms of Time in Participant Salary. In addition, it is assumed that the ‘Lost Production Time’ for the employee would be approximately $2400 in that same time frame. Therefore the labor and opportunity cost for this participant would be $3600. Although this may be an artificially high estimate (e.g. a sales person who generated $2400 in revenue is likely only creating 10%–30% of that in actual profit) it is a conservative estimate in terms of cost structure for this analysis, as well as RODI. For a comparison of RODI between our conservative estimates of lost production time and less conservative figures please see Table 3.

Technology costs are also included in our calculation. Technology costs include the cost for all technology needed for conducting the interventions such as hardware, programmers for coding software and initial platform set up. The different costs for Middle and Upper Leaders is based on conversations with large (greater than 1000 employees noted above) organizations from across the United States that had identified an average leadership development intervention target of 30 for upper level leaders and 100 for mid-level leaders.

As mentioned previously, we used a linear model for this analysis. This means that with each positive factor (e.g. development for high versus low performers) RODI is likely to substantially increase. It should be noted that a non-linear model application

Table 2 Cost structure based on 1.5 day developmental leadership intervention.

Cost of training On-site a Off-site On-line local

Time in participant salary a (ex. $100/h) $1200 $1600 $1200 Lost production time a (ex. $100/h) $2400 $3200 $2400 Instructor $5000 $5000 $1500 Instructor support staff $1000 $1000 $5000 Technology a $500 $500 $10,000 Materials-up. $250 $250 $250 Materials-mid. $750 $750 $750 Trainer traveling expenses $2000 $2000 $0 Travel costs for participants-up a $0 $3000 $0 Travel costs for participants-mid a $0 $10,000 $0 Meals-up $3600 $3600 $0 Meals-mid $10,000 $10,000 $0 Hotel conference room for training-up $400 $500 $0 Hotel conference room for training-mid $800 $1000 $0 Hotel stay for participants-up $0 $4500 $0 Hotel stay for participants-mid $0 $15,000 $0 Total $27,900 $61,900 $21,100

a On-site is the location of the leadership development program which helps determine costs. These analyses were intended to show the cost structure and effect size of outside providers (e.g. academic/practitioners using a validated leadership model as the base of their intervention). On site in this case does no insinuate internal (e.g. HR) personnel delivering the program. We use the same figures here that guided the work of Avolio et al. (2009).

t

Table 3 Comparison of RODI between estimating methods of lost production time.

RODI On-site (our estimate) On-site (less conservative estimate)

Low Average High Low Average High

1.5 day developmental intervention RODI %-up. ldr −146% 61% 200% −178 % 169% 200% RODI %-mid. ldr −150% 72% 200% −187% 200% 200%

Offsite Offsite

Low Average High Low Average High

RODI %-up. ldr −142% 44% 200% −165% 125% 200% RODI %-mid. ldr −142 % 46% 200% −166% 130% 200%

On-line On-line

Low Average High Low Average High

RODI %-up. ldr −144% 52% 200% −170% 144% 200% RODI %-mid. ldr −151% 76% 200% −190% 200% 200%

3 day developmental intervention RODI %-up. ldr −147% 64% 200% −180% 178% 200% RODI %-mid. ldr −154% 87% 200% −200% 200% 200%

Offsite Offsite

Low Average High Low Average High

RODI %-up. ldr −143% 50% 200% −169% 139% 200% RODI %-mid. ldr −144% 52% 200% −170% 144% 200%

On-line On-line

Low Average High Low Average High

RODI %-up. ldr −147% 64% 200% −180% 176% 200% RODI %-mid. ldr −153% 82% 200% −196% 200% 200%

Table 4 Estimated effect size.

Total trained Time in years Effect size Salary 40% of salary

Up. level leader 30 0.167 0.52 $100,000 $40,000 Up. level leader-low 30 0.167 −0.15 $100,000 $40,000 Up. level leader-high 30 0.167 1.19 $100,000 $40,000 Mid level leader 100 0.167 0.52 $70,000 $28,000 Mid level leader-low 100 0.167 −0.15 $70,000 $28,000 Mid level leader-high 100 0.167 1.19 $70,000 $28,000 Up. level follower 100 0.167 0.25 $70,000 $28,000 Up. level follower-low 100 0.167 0.03 $70,000 $28,000 Up. level follower-high 100 0.167 0.46 $70,000 $28,000 Mid level follower 1000 0.167 0.25 $50,000 $20,000 Mid level follower-low 1000 0.167 0.03 $50,000 $20,000 Mid level follower-high 1000 0.167 0.46 $50,000 $20,000

638 B.J. Avolio et al. / The Leadership Quarterly 21 (2010) 633–644

would reveal quite different results, which could yield significantly higher or lower estimates depending on the effect size and nature of the impact. While the linear model predicts very high (e.g. greater than 200%) RODI in some cases, we have capped the RODI estimate for these analyses at 200%. Our rationale for capping RODI represents an attempt to balance a linear estimating model with a dynamic performance criterion model. Given ideal conditions, RODI of greater than 200% is possible; however, the numerous factors that may inhibit subsequent performance and transfer of development make it unlikely that higher returns would be sustainable.

Since different effect sizes were obtained from the meta-analysis for management levels and the type of leadership intervention, we calculated overall return for each management level (i.e. upper-level leader and followers, mid-level leader and followers) and for each type of leadership intervention (i.e. on-site, off-site and virtual). As evident in Table 3, on average, the range of RODI is 44%–72% for a 1.5 day intervention and 50%–87% for a three day intervention depending on location and level of participant in the intervention.

The effect sizes are estimates of the intervention effect on leadership performance. High, average and low effect sizes were incorporated from the leadership meta-analysis that was referred to above (Avolio et al., 2009). The effect sizes reported in these analyses do not take into account “boosters” or activities/exercises after the primary leadership intervention has concluded that

639B.J. Avolio et al. / The Leadership Quarterly 21 (2010) 633–644

might support and/or sustain the initial effects of these developmental interventions. We assumed the intervention had an influence on the leader and his or her followers over a time span of 2 months. This time was then transformed to time in years (i.e. 2 months=.167 years) to acquire an overall annual return. Again, this is a conservative estimate in that a highly salient event could affect someone for years as opposed to months. It is also plausible that leadership performance might decrease after the close of the intervention as the participant struggles to apply new skills and knowledge learned in the intervention. This again, highlights the value of the RODI approach, which can be used to assess the longitudinal effects and return of the leadership intervention (Table 4).

With respect to ‘Return on Salary’, when viewed as a transaction, an organization pays an upper-level leader $100 k per year for his or her performance. If the individual's performance increases by 50% and the organization continues to pay $100 k per year, the organization assumes an indirect increase of $50,000 from that individual. However, Cascio (1999) notes that there are far more factors that comprise one's salary than performance value to the company. He therefore recommends a more conservative figure of 40% of the salary rather than 100%. Therefore, we used 40% of the assumed annual salary in this analysis.

Annual Estimated Return on Development Investment is listed in Tables 5 and 6 for the off-site program. These tables illustrate an estimate of financial return on development for a leadership intervention based on the summation of research from meta- analyses and assumptions mentioned above. Results of our analyses were separated into a 1.5 day intervention (Table 5) and a 3 day intervention (Table 6) with the primary differences between the two being cost structure and degree of intervention effectiveness in the form of an effect size estimate. In addition, each section is also separated into the three intervention mediums of on-site, off-site and on-line. A confidence interval for the effect size was used to create a range of RODI estimates. Although the effect size for the 3 day intervention was larger, we used the same standard deviation at the 1.5 day intervention to create the range of estimates (e.g. high, medium and low effect) leading to more conservative results with the high effect group not being as high and the low not as low.

Overall, as evident in Tables 5 and 6, there is small risk of a negative RODI for leadership interventions. However, when using the mean effect size (vs. one standard deviation above or below) looking across intervention site, results suggest a positive financial return from $31,935–$39,535 for 30 upper level leaders in a 1.5 day intervention and $76,748–$104,748 for 100 mid-level leaders in a 1.5 day intervention.

When considering a 3 day intervention, results indicate a positive financial return from $69,370–$81,570 for the upper level leaders and $165,996–$226,396 for the mid-level leaders. When considering effects on followers, the financial return across sites was $116,900 total for upper level followers and $835,000 total for mid-level leader's followers. The expected return increased to $233,800–$1,670,000 respectively for the 3 day intervention.

Table 5 RODI 1.5 day intervention a.

On-site

Low return Average return High return

Up. level leader ($94,733) $39,535 $173,803 Mid level leader ($211,344) $101,948 $415,240 Up. level follower $14,028 $116,900 $215,096 Mid level follower $100,200 $835,000 $1,536,400 RODI %-up. ldr −146% 61% 200% b RODI %-mid. ldr −150% 72% 200% b

Off-site

Low return Average return High return

Up. level leader ($102,333) $31,935 $166,203 Mid level leader ($236,544) $76,748 $390,040 Up. level follower $14,028 $116,900 $215,096 Mid level follower $100,200 $835,000 $1,536,400 RODI %-up. ldr −142% 44% 200% b RODI %-mid. ldr −142% 46% 200% b

On-line

Low return Average return High return

Up. level leader ($98,733) $35,535 $169,803 Mid level leader ($208,544) $104,748 $418,040 Up. level follower $14,028 $116,900 $215,096 Mid level follower $100,200 $835,000 $1,536,400 RODI %-up. ldr −144% 52% 200% b RODI %-Mid. Ldr −151% 76% 200% b

Note: Values in parenthesis represent losses. a Figures here were also used in Avolio et al. (2009). b The RODI analyses were capped at 200% for return.

Table 6 RODI 3.0 day intervention.

On-site

Low return Average return High return

Up. level leader ($186,966) $81,570 $350,106 Mid level leader ($400,188) $226,396 $852,980 Up. level follower $28,056 $233,800 $430,192 Mid level follower $200,400 $1,670,000 $3,072,800 RODI %-up. ldr −147% 64% 200% a RODI %-mid. ldr −154% 87% 200% a

Off-site

Low return Average return High return

Up. level leader ($199,166) $69,370 $337,907 Mid level leader ($460,588) $165,996 $792,580 Up. level follower $28,056 $233,800 $430,192 Md level follower $200,400 $1,670,000 $3,072,800 RODI %-up. ldr −143% 50% 200% a RODI %-mid. ldr −144% 52% 200% a

On-line

Low return Average return High return

Up. level leader ($187,466) $81,070 $349,606 Mid level leader ($407,088) $219,496 $846,080 Up. level follower $28,056 $233,800 $430,192 Mid level follower $200,400 $1,670,000 $3,072,800 RODI %-up. ldr −147% 64% 200% a RODI %-mid. ldr −153% 82% 200% a

Note: values in parenthesis represent losses. a The RODI analyses were capped at 200% for return.

640 B.J. Avolio et al. / The Leadership Quarterly 21 (2010) 633–644

The preceding analysis highlights several important differences in level of intervention, degree of effectiveness, medium of intervention and subsequent financial return on development. Specifically, as with most investments, organizations can gain or lose money by investing in leadership development depending on the target of the intervention, proven effects, cost structure, etc. Due to the range of effect that upper leaders can have in their organizations, the potential for a significant positive RODI generally increases with higher levels of leadership. Due to the high cost of upper leadership interventions in part, due to their higher salaries versus mid-level interventions, there is a greater potential for loss with higher levels of leadership. In addition, on-site interventions generally possess greater potential for the highest levels of RODI again due to reduced cost structure.

3.1. Extension to the RODI analyses

In this section, we propose both contextual and individual factors that can influence the RODI estimates obtained from leadership interventions. Our purpose here is to highlight different facets of the context and individual going through the intervention that future research should consider in terms of estimating RODI. These and other factors will no doubt impact the type of assumptions one must use in calculating RODI.

3.1.1. Context factors At a firm or unit level perspective, a climate that enhances organizational support for transferring what is learned into practice

(e.g., see Rogg, Schmidt, Shull, & Schmitt, 2001) may provide the psychological safety neccesary to risk trying out new techniques learned in the developmental intervention. Clearly, future research should take into consideration how the transfer climate would impact the intervention and estimates of RODI.

The extent to which the training is generalized across all organizational levels can also have an impact on RODI. For example, if only the middle level managers are trained in an organization, they may not have the support for transfering leadership training back to their organization from their supervising leaders. The lack of support from upper level leaders, or other types of support, would certainly have an impact on RODI.

The geographical location of participants can also potentially impact the positive transference of training to the organization. For example, to the extent, particpants can continue to remain in contact with each other to support the transfer of training, the more likely the training effects could be reinforced over time. We suspect that peers holding peers accountable for transfering the training could be a positive context factor in terms of boosting RODI.

641B.J. Avolio et al. / The Leadership Quarterly 21 (2010) 633–644

3.1.2. Individual factors At an individual level, there are a number of factors that could positively or negatively contribute to RODI like talent, motivation

to perform, developmental readiness, opportunities to lead and motivation to lead, which we also did not factor into the analyses above (Chan & Drasgow, 2002; Maurer, 2002; Avolio, 2003). Any given leadership intervention will influence performance to varying degrees partially depending on differences across individuals in their current levels of ability, motivation and performance, as well as in terms of the goals of the intervention and how directly they relate to revenue generating activity.

It has been demonstrated that general mental ability is often a significant predictor of performance in the workplace (Hunter & Schmidt, 1996; Schmidt & Hunter 1998; 2000). Thus, all things being equal, more talented people selected for developmental interventions will likely transfer the trained knowledge into their job at a faster rate leading to higher RODI.

Another way to boost RODI is to consider the self-efficacy (i.e., self-confidence) and intrinsic motivation level of the trainees selected for the intervention. These theories suggest that more confident and intrinsically motivated employees tend to be higher performers as they expend more effort in their jobs. Along these lines, a meta-analysis examining the effectiveness of training interventions, reported that the level of motivation to participate in training was shown to be a significant predictor of the transfer of knowledge and performance above and beyond general intelligence (Colquitt, LePine, & Noe, 2000). Thus, given that higher performers are often more motivated, and higher motivation leads to more positive training effectiveness, we expect top, motivated performers to produce better results as a consequence of leadership developmental interventions.

Developmental readiness is considered critical to the developmental process in terms of the expected positive effects of interventions on individuals and how well the trainingis applied back in the organization (Maurer, 2002; Avolio, 2003). For example, if an individual is generally more oriented toward learning, they will likely be motivated to acquire new knowledge, which would be expected to contribute more positively to development. Similarly, Maurer (2002) argues for the importance of participants having a feedback orientation in terms of how well any leadership intervention will impact on the participant's leadership development. Specifically, individuals who are more oriented toward feedback will not only solicit feedback which contributes to their development, but apply that feedback to changes in the way they think and behave once back in the organization.

The last component we mention in terms of selection for leadership development is an individual trait known as motivation to lead (Chan & Drasgow, 2002). An important difference between motivation to lead and the aforementioned training motivation is the focal point of the individual's motivation. While those high in training motivation are generally motivated to learn the content of any developmental program, those high in motivation to lead may be more predisposed to apply what they have learned about their leadership to improving their performance as leaders given the intrinsic desire to be a leader. Thus, it is likely that motivation to lead compliments training motivation in terms of acquiring and applying content of leadership development programs.

Table 7 RODI top performers: 1.5 day intervention.

On-site

Low return Average return High return

Up. level leader $7471 $141,739 $276,007 Mid level leader $27,132 $340,424 $653,716 Up. level follower $51,436 $229,124 $406,812 Mid level follower $367,400 $1,636,600 $2,905,800 RODI %-up. ldr 12% 200% a 200% a

RODI %-mid. ldr 19% 200% a 200% a

Off-site

Low return Average return High return

Up. level leader ($129) $134,139 $268,407 Mid level leader $1932 $315,224 $628,516 Up. level follower $51,436 $229,124 $406,812 Mid level follower $367,400 $1,636,600 $2,905,800 RODI %-up. ldr −1% 186% 200%d RODI %-mid. ldr 1% 189% 200%d

On-line

Low return Average return High return

Up. level leader $3471 $137,739 $272,007 Mid level leader $29,932 $343,224 $656,516 Up. level follower $51436 $229,124 $406,812 Mid level follower $367,400 $1,636,600 $2,905,800 RODI %-up. ldr 5% 200% a 200% a

RODI %-mid. ldr 22% 200% a 200% a

a The RODI analyses were capped at 200% for return.

Table 8 RODI top performers: 3 day intervention.

On-site

Low return Average return High return

Up. level leader $17,442 $285,978 $554,514 Mid level leader $76,765 $703,349 $1,329,933 Up. level follower $102,872 $458,248 $813,624 Mid level follower $734,800 $3,273,200 $5,811,600 RODI %-up. ldr 14% 200% a 200% a

RODI %-mid. ldr 30% 200% a 200% a

Off-site

Low return Average return High return

Up. level leader $5242 $273,778 $542,314 Mid level leader $16,365 $642,949 $1,269,533 Up. level follower $102,872 $458,248 $813,624 Mid level follower $734,800 $3,273,200 $5,811,600 RODI %-up. ldr 4% 197% 200% a

RODI %-mid. ldr 5% 200% a 200% a

On-line

Low return Average return High return

Up. level leader $16,942 $285,478 $554,014 Mid level leader $69,865 $696,449 $1,323,033 Up. level follower $102,872 $458,248 $813,624 Mid level follower $734,800 $3,273,200 $5,811,600 RODI %-up. ldr 13% 200% a 200% a

RODI %-mid. ldr 26% 200% a 200% a

a The RODI analyses were capped at 200% for return.

642 B.J. Avolio et al. / The Leadership Quarterly 21 (2010) 633–644

It is important to note, that all of the above constructs have been measured reliably in prior research. Thus, scales are available to measure each of these constructs so that organizations can make the best determination on whom to invest in leadership development interventions and at what point in time in terms of their readiness to engage in learning.

Overall evidence suggests that certain individuals are likely to generate greater returns to organizations as a result of the investment in their development. Therefore, we completed analyses in tandem with increased effect sizes (again based on the data reported in the meta-analysis) to determine the differential effectiveness for top performers in a leadership intervention over average performers in that same intervention. Table 7 (1.5 day intervention) and Table 8 (3 day intervention) are the result of these analyses).

These tables and the preceding analysis highlight the importance of selection for development. Specifically, using the same RODI formula, when top performers are selected for leadership development interventions the subsequent performance and ultimately financial effectiveness for the organization is substantially increased. For example, average RODI for top performers in a 3 day intervention ranged from 197%–200% (capped) as compared to 50%–87% for average performers. This analysis demonstrates that even when effect sizes are expected to be low, positive RODI is likely to emerge due to high performer's motivation and ability to apply learned content to enhancing their leadership potential and performance.

4. Discussion

Our results demonstrate a wide range of estimated effects and RODI for different types of leadership interventions. The ranges of RODI effects includes a negative ($460,588) to highly positive $5,811,600 effect in terms of dollars returned to an organization for the respective interventions based on the assumptions used in this study. These results signal that on average, one could expect a positive and substantial return on the effects of leadership interventions in terms of leadership effectiveness/performance.

Based on previous meta-analytical and utility procedures suggested by Cascio and Boudreau (2008), any organization can estimate the effect of a proposed leadership intervention before deciding on whether or not to invest in that leadership intervention. Our evidence suggests that at least a moderate effect size is needed to get a positive return on development.

Not only can one estimate the dollar effect of leadership interventions, we argue here that these analyses should be considered prior to investing substantial revenue in leadership interventions. Given the information available from both the meta-analysis and costing HR models, leadership development investments should not be thought of as a subjective activity where return on investment analyses are not needed or impossible to calculate. We take the position that any organization considering investing financial capital in leadership development should attempt to conduct an RODI analysis. Of course, this requires that the providers for the leadership development intervention are able to answer the following question: What has been the effect size of your intervention based on validation evidence collected thus far? Placing pressure on providers to offer such evidence will in our view enhance both the practice and science of leadership development.

643B.J. Avolio et al. / The Leadership Quarterly 21 (2010) 633–644

Beyond the return on investment, it may be considered a cost of doing business to invest in employees even if returns are potentially negative. While in most of our scenarios the estimates of RODI were positive, there may be cases where an organization invests in all employees through a leadership development program because it is “the right thing to do” or for outcomes not measurable in terms of financial returns. For example, the RODI may come in the long-term with regard to building a more positive employment climate. In addition, organizational leaders might make this investment even when RODI results suggest a poor return as it may enhance employee retention, or the firm's image, making it easier to recruit top talent. Therefore, RODI is likely best used as a financial tool for individuals and organizations to know what return to expect, not necessarily as a single metric to determine whether or not to execute the program.

Knowing the RODI will also help a culture of placing greater expectations and emphasis on supporting the transfer of learning back to the organization. If leaders know how much money has been invested and projections for returns, they will likely be more motivated to support a successful transfer of training and positive impact on performance. We would then expect these leaders to follow up on training by debriefing what was learned, asking to meet with participants periodically to offer feedback, and/or helping to build the change expected from the program into their performance evaluations. Indeed, the expectation of a positive return alone may be sufficient motivation to facilitate a positive return!

The analyses described above are also valuable for upper leadership to share with the organization for two additional reasons. First, in organizations where leaders prefer more traditional approaches to anticipating returns on financial investment, we suspect that such leaders may be more supportive of investments in leadership development, if the decision choices are presented as a typical business case.

The second reason for upper leadership to share these analyses within their organization is that participants in the intervention clearly have a role in acquiring desired RODI. Leaders will be more likely to engage in the intervention and use learned techniques after the intervention if they recognize the cost of the intervention on their behalf and their role in leading effectively to obtain desired RODI. Finally, it helps send a clear message of what organizational leaders expect in terms of change.

The comparative analysis of average and top performers highlights the importance of selection for training participants. Also, included in the selection of top candidates is the idea that they are developmentally ready to transfer what they have learned into their leadership role within their organization (Avolio & Hannah, 2008). Overall, including high and more motivated performers in a leadership development intervention would be expected to generate greater RODI for the organization. In addition, one's organizational role must also be considered. For example, those individuals with a greater span of control, or influence potential (e.g. top leaders, leader's of key initiatives, project leaders) may be more likely to generate greater RODI for the organization as development is likely to affect more facets of organizational performance.

4.1. Potential limitations of the study

Although our analyses were based on published meta-analytic results and Cascio and Boudreau's (2008) utility analysis, caution must be used in interpreting the findings reported here. Specifically, we suggest that the following may be considered as limitations to the interpretation of our analyses and results. Longitudinal data were sparse in the meta-analysis used to estimate effect sizes. Thus, assumptions on the sustainability of effects are based on limited data. Moreover, the meta-analytic conclusions that served as a foundation for our assumptions were from studies largely conducted in the United States. While applications of these results to non-U.S. based organizations may be acceptable, how our findings generalize outside the U.S. remains to be tested (Yukl, 2006).

Previous work has not looked at subsequent effectiveness throughout levels of the organization. In other words, if a senior leadership team participated in a leadership development intervention, the subsequent effect on their followers and their followers' followers is not well understood or documented. We werealsonot able to breakout interventionsthatlooked at developingleadership at earlier vs. later points in the life span where RODI may differ. Therefore, the point at which the leadership interventions are conducted as well as the nature of the work being done may moderate effects of leadership interventions and ultimately affect RODI.

Finally, our assumptions and calculations were derived primarily from large (vs. medium or small) organizations. Using these assumptions could affect the reported results primarily in two ways. First, the cost structure was driven by data collected from larger Fortune 500 organizations. Therefore, certain assumptions (such as on-site facilities for 40 or more participants) may not be applicable to smaller organizations. Second and likely more influential to the results is the size of leadership teams. This analysis was based on 30 upper level leaders (e.g. executives or senior leaders) and 100 mid-level leaders. Smaller organizations may not have this large of a leadership team at the senior or mid level. Thus, results must be interpreted for smaller organizations with this understanding in mind.

In addition, it might be useful in future estimates of RODI to use salary data that may be more representative of a broader range of organizations. There are numerous sources documenting national survey data that could be used for this purpose.

4.2. Conclusions and implications

One of the key goals of this research was to change the way organizational leaders think about their investment in leadership development. Oftentimes, organizations are more willing to invest in leadership development, when sufficient ‘extra’ funds are available. Too often, a downturn in the economy signals a delay or discontinuance of training and development. Although such decisions may be warranted, it is hoped that the results presented in this study offer leaders a quantitative basis for making such decisions. Indeed, there may be times when the economy is slow that represents the very best time to invest in leadership development, as the loss in potential sales in a down economy may be marginalized. In our view, whether it is done internally or externally, leadership development interventions should be proven valid and at some point in the interventions development, the

644 B.J. Avolio et al. / The Leadership Quarterly 21 (2010) 633–644

providers should be able to offer demonstrated return in terms of the investment being made by organizations. Until this is done, business leaders may continue to see leadership development as a cost versus an investment with the potential for very robust returns. Also, future research should attempt to examine all of the components that go into calculating RODI in terms of the types of outcomes or impact that is used in the analyses as well as the nature of assessments used to select individuals for leadership development. Ultimately, our goal here is encourage decision-makers to consider the financial impact and return of leadership development interventions as being important criteria they consider prior to making an investment in developing leaders.

References

Avolio, B. J. (2003). Examining the full range model of leadership: Looking back to transform forward. In D. Day, & S. Zacarro (Eds.), Leadership development for transforming organizations: Growing leaders for tomorrow (pp. 71−98). Mahwah, NJ: Erlbaum.

Avolio, B. J. (2005). Leadership development in balance: Made/born. NJ: Erlbaum & Associates. Avolio, B. J. (2007). Promoting more integrative strategies for leadership theory-building. American Psychologist, 62, 25−33. Avolio, B. J., & Hannah, S. T. (2008). Developmental readiness: Accelerating leadership development. Consulting Psychology Journal, 60, 331−347. Avolio, B. J., Hannah, S., Reichard, R., Chan, A., & Walumbwa, F. (2009). 100 years of leadership intervention research. Leadership Quarterly, 20, 764−784. Avolio, B. J., & Luthans, F. (2006). The high impact leader: Moments matter in accelerating authentic leadership. New York NY: McGraw-Hill. Avolio, B. J., Sosik, J. J., Jung, D. I., & Berson, Y. (2003). Leadership models, methods, and applications. In W. C. Borman, D. R. Ilgen, & R. J. Klimoski (Eds.), Handbook of

Psychology (Vol. 12) (pp. 277−307). Hoboken, NJ: Wiley. Becker, B. E., Huselid, M. A., & Ulrich, D. (2001). The HR scorecard linking people, strategy and performance. Boston: Harvard Business School Press. Berson, Y., & Avolio, B. J. (2004). Linking transformational and strategic leadership: Examining the leadership system of a high-technology organization in a

turbulent environment. The Leadership Quarterly, 15, 625−646. Brogden, H. E. (1946). On the interpretation of the correlation coefficient as a measure of predictive efficiency. Journal of Educational Psychology, 37, 65−76. Brogden, H. E. (1949). When testing pays off. Personnel Psychology, 2, 171−183. Burke, M. J., & Day, R. R. (1986). A cumulative study of the effectiveness of managerial training. Journal of Applied Psychology, 71, 232−245. Cascio, W. (1999). Costing human resource: The financial impact of behavioral organizations. New York: Wiley. Cascio, W. F., & Boudreau, J. W. (2008). Investing in people: Financial impact of human resource initiatives. New Jersey: FT Press. Chan, K. Y., & Drasgow, F. (2002). Toward a theory of individual differences and leadership: Understanding motivation to lead. Journal of Applied Psychology, 86,

491−498. Collins, D. B., & Holton, E. F. (2004). The effectiveness of managerial leadership development programs: A meta-analysis of studies from 1982 to 2001. Human

Resource Development Quarterly, 15, 217−242. Colquitt, J., LePine, J., & Noe, R. (2000). Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. Journal of Applied

Psychology, 85, 678−707. Cronbach, L. J., & Gleser, G. C. (1965). Psychological tests and personnel decisions, 2nd ed. Urbana, IL: University of Illinois Press. Day, D. V. (2000). Leadership development: A review in context. Leadership Quarterly, 11, 581−613. Day, D. V., Harrison, M. M., & Halpin, S. M. (2008). An integrative approach to leader development: Connecting adult development, identity, and expertise. NY:

Routledge. Fegley, S. (2006). SHRM 2006 strategic HR management survey report.Alexandria, VA: Society for Human Resource Management September. Flamholtz, E. G., Bullen, M. L., & Hua, W. (2002). Human resource accounting: A historical perspective and future implications. Management Decision, 40, 947−954. Hunter, J. E., & Schmidt, F. L. (1996). Intelligence and job performance: Economic and social implications. Psychology, Public Policy and Law, 2, 447−472. Huselid, M. A. (1995). The impact of human resource management practices on turnover, productivity, and corporate financial performance. Academy of

Management Journal, 38, 635−672. Huselid, M. A., Jackson, S. E., & Schuler, R. S. (1997). Technical and strategic human resource management effectiveness as determinants of firm performance.

Academy of Management Journal, 40, 171−188. Industry Report (November/December, 2007). Training, 8–24. Latham, G. P., & Whyte, G. (1994). The futility of utility analysis. Personnel Psychology, 47, 31−46. Maurer, T. (2002). Employee learning and development orientation: Toward an integrative model of involvement in continuous learning. Human Resources

Development Review, 1, 9−44. Phillips, J. A. (2003). Return on investment in training and performance improvement programs. Burlington, MA: Butterworth-Heinemann. Rogg, K. L., Schmidt, D. B., Shull, C., & Schmitt, N. (2001). Human resource practices, organizational climate, and customer satisfaction. Journal of Management, 27,

431−449. Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of

research findings. Psychological Bulletin, 1242, 262−274. Schmidt, F. L., & Hunter, J. E. (2000). Select on intelligence. In E. A. Lock (Ed.), Handbook of principles of organizational behavior (pp. 3−14). Oxford, UK: Blackwell. Schmidt, F. L., Hunter, J. E., & Pearlman, K. (1982). Assessing the economic impact of personnel programs on workforce productivity. Personnel Psychology, 35,

333−347. Schultz, T. W. (1961). Investment in human capital. American Economic Review, 1, 1−17. Ulrich, D., Zenger, J., & Smallwood, N. (1999). Results-based leadership. Boston: Harvard Business School Press. Yukl, G. A. (2006). Leadership in organizations. Upper Saddle River, NJ: Prentice Hall.