Business

Does Team Training Improve Team Performance? A Meta-Analysis

Eduardo Salas, Deborah DiazGranados, Cameron Klein, C. Shawn Burke, and Kevin C. StagI, University of Central Florida, Orlando, Florida, and Gerald F. Goodwin and Stanley M. Halpin, Army Research Institute, Arlington, Virginia

Objective: This research effort leveraged the science of training to guide a taxonomic integration and a series of meta-analyses to gauge the effectiveness and boundary con- ditions of team training interventions for enhancing team outcomes. Background: Disparate effect sizes across primary studies have made it difficult to determine the true strength of the relationships between team training techniques and team outcomes. Method: Several meta-analytic integrations were conducted to examine the rela- tionships between team training interventions and team functioning. Specifically, we assessed the relative effectiveness of these interventions on team cognitive, affective, process, and perfonnance outcomes. Training content, team membership stability, and team size were investigated as potential moderators of the relationship between team training and outcomes. In total, the database consisted of 93 effect sizes representing 2,650 teams. Results: The results suggested that moderate, positive relationships exist between team training interventions and each of the outcome types. The findings of moderator analyses indicated that training content, team membership stability, and team size moderate the effectiveness of these interventions. Conclusion: Our findings sug- gest that team training interventions are a viable approach organizations can take in order to enhance team outcomes. They are useful for improving cognitive outcomes, affective outcomes, teamwork processes, and performance outcomes. Moreover, results suggest that training content, team membership stability, and team size moderate the effectiveness of team training interventions. Application: Applications of the results from this research are numerous. Those who design and administer training can ben- efit from these findings in order to improve the effectiveness of their team training interventions.

Attention to team training has grown exponentially during the past 10-20years. It is driven by at least the perception that the "high performance team " is a critical element in the design of organizations that will be effective in the global economy.

- Campbell & Kuncel (2001, p. 299)

INTRODUCTION

The nature of work has changed. Organizations now face increased competition and collaboration within and across organizational, geographic, and temporal boundaries; a need to engage a demo- graphically heterogeneous workforce; a need to deal with advancements in information technol- ogy; a need to promote safety; and a need to fos-

ter enduring customer relations (Noe, 2002; Pfeffer & Sutton, 2000; Tannenbaum, 2002). One re- sponse to these changes has been the use of work teams as a preferred performance management technique. "Teams are ubiquitous. Whether we are talking about software development, Olympic hockey, disease outbreak response, or urban war- fare, teams represent the critical unit that 'gets things done' in today's world" (Marks, 2006, p. i).

Both governmental agencies and private indus- try are increasingly relying upon work teams as a preferred perfonnance arrangement to fulfill their visions, execute their complex missions, and ac- complish their goals (Salas, Stagl, & Burke, 2004). In fact, a recent random sample of U.S. organiza- tions indicated that nearly half (48%) used some

Address correspondence to Eduardo Salas, Institute for Simulation & Training, University of Central Florida, 3100 Technology Pkwy., Orlando, FL 32826; [email protected]. ÍÍÍ/M4/V FACTORS, Vol. 50, No. 6, December 2008, pp. 903-933. DOI 10.1518/ 001872008X375009. Copyright © 2008, Human Factors and Ergonomics Society.

904 December 2008 - Human Factors

type of team-based structure, with ongoing project teams being the most frequently reported (Devine, Clayton, Philips, Dunford, & Meiner, 1999).

The increasing frequency of team-based forms of organizing presents unique opportunities and creates challenges. Opportunities via the use of teams abound. For example, collectives can lever- age shared mental models, compensatory pro- cesses, and affective states such as cohesion to deal more effectively with the complex, stressful, and sometimes chaotic contexts that typify modem operations (Orasanu & Salas, 1993). The process gains yielded by teams can ultimately culminate in enhanced efficiencies, quality and safety im- provements, creativity, and/or adaptation (Banker, Field, Schroeder, & Sinha, 1996; Burke, Stagl, Salas, Pierce, & Kendall, 2006; Cohen & Led- ford, 1994; Foushee, 1984).

However, challenges inherent to the use of teams exist. Most notably, how should organiza- tions compose, manage, and develop team mem- bers? More specifically, how does one design and deliver team training? What kind of teatn train- ing should be implemented? And finally, does it work? This research was motivated by the need to answer these questions.

Team training targets latent teamwork knowl- edge, skills, and/or attitudinal competencies (KSAs) as well as manifest team processes and performance for improvement (Salas & Cannon-Bowers, 1997, 2000). Teams can be trained to make better deci- sions (Orasanu & Fischer, 1997), to perform better under stress (Driskell, Salas, & Johnston, 1995), and to make fewer errors (Wiener, Kanki, & Helm- reich, 1993).

Enhancing teamwork through team training is a chief concern in many industries, including com- mercial and military aviation (Boehm-Davis, Holt, & Seamster, 2001; Helmreich & Foushee, 1993; Oser, Salas, Merket, & Bowers, 2001), health care (Baker, Salas, Barach, Battles, & King, 2007; Da- vies, 2001 ; Salas, DiazGranados, Weaver, & King, 2008), and energy (Flin, 1995; Flin & O'Connor, 2001), to name just a few. In these industries (and others), a great deal of research has been generated on team training. Unfortunately, no comprehensive source exists in the team training literature that demonstrates whether or not team training even matters and, if it does, by how much. Further, no source outlines the conditions under which team training works. This research initiative seeks to fill these voids.

The following sections of this paper are orga- nized as follows. We first describe the rationale for the current meta-analytic initiative. Next, we ad- dress the conceptual foundation on which this research is based. Included here is a short discus- sion centered upon the nature of teams and team performance, as well as a longer discourse on the current state of team training efforts. Eight specific study hypotheses are then put forward, followed by a detailed accounting of the study methods. An ensuing description of meta-analytic results is sub- sequently presented, followed by implications for applied work with teams - a discussion that fully incorporates study limitations and directions for future research on team training.

WHY A TEAM TRAINING INTEGRATION?

This research effort includes a series of meta- analyses, each of which was conducted to gauge the effectiveness of team training for enhancing team outcomes. Specifically, we sought to (a) clar- ify and contribute to the science of training by pro- viding a clarification of the nature of team training via a taxonomic integration of team training inter- ventions; (b) establish the boundary conditions of team training for enhancing team outcomes via moderator analyses with additional variables of interest; and (c) impart valuable information for organizational stakeholders charged with design- ing, delivering, and evaluating team training inter- ventions.

First, several limitations of the existing base of research on team training and development have served to impede knowledge accumulation and in- tegration (Campbell & Kuncel, 2001). As Judge, Cable, Colbert, and Rynes (2007) recently noted, "Regardless of the quality of an idea, the ability to draw inferences about a phenomenon is con- strained by the quality of the methods used to gather data about it" (p. 493). In the team training area, practical methodological limitations wit- nessed in primary studies that rely on relatively small samples of teams generally result in an unac- ceptably large sampling error - error that can ad- versely influence the consistency and quality of independent research studies' conclusions. At the same time, disparate effect sizes across primary studies have made it difficult to determine the true strength of the relationships between team train- ing techniques and team outcomes.

The use of meta-analysis can help overcome

TEAM TRAINING EFFECTIVENESS 905

these issues. Specifically, meta-analytically inte- grating the results of independent studies allows one to amalgamate, with increased precision and certainty, the significance, strength, and predictable variation of the effects of team training on team functioning. Although recent attempts iiave been made to empirically (and meta-analytically) review the infiuence of team development interventions on team functioning (e.g.. Salas, Nichols, & Dris- kell, 2007; Salas, Rozell, Mullen, & Driskell, 1999), this research advances (and expands upon) prior efforts by including more primary studies, employ- ing enhanced methodology, and examining key variables that will help clarify the existing knowl- edge base on team training.

We should note that most team training evalu- ations have been conducted in the military and in aviation. These sectors are the primary investors of funding and effort to understand team perfor- mance and how to develop and deliver team train- ing. The current research includes team training evaluations from several sectors (i.e., military, avi- ation, industry, lab/classroom, and medical set- tings). As will be discussed at a later point, the primary studies included in this meta-analysis are heavily represented by military research.

This meta-analysis also provides clarity and understanding concerning the amorphous empir- ical literature on team training. In practice, team training appears in many shapes, sizes, and forms and is called by many different names and labels (e.g., assertiveness training, crew resource man- agement, cross-training, group process training, problem-solving training, task-focused simulation training). What has long been needed in this do- main is a taxonomic integration and empirical in- vestigation of the utility of these interventions based upon the content or focus of the interven- tions, rather than upon the myriad of names given to these strategies by independent researchers.

Fortunately, an inclusive taxonomy has already emerged. Upon summarizing work conducted by Salas and colleagues (e.g., Cannon-Bowers, Tan- nenbaum, Salas, & Volpe, 1995; Mclntyre & Salas, 1995; Salas, Dickinson, Converse, & Tannen- baum, 1992), Goldstein and Ford (2002) argued that [team] training methods are designed to en- hance taskwork, teamwork, and process improve- ment skills. Meta-analytic reviews have been conducted of team interventions that focused on process improvement skills (i.e., team building; Klein et al., in press; Salas et al., 1999), so there is

now a need to empirically analyze the team train- ing literature along the teamwork-taskwork con- tinuum. The current research investigates the unique and relative value of these two distinct focal points of team training.

A second impetus for this research is to inves- tigate moderators of team training effectiveness. These investigations will help provide answers to a number of important questions, including the following:

• Does the effectiveness of team training vary based upon the content of the training?

• Does team training work better with intact versus ad hoc teams?

• Do larger teams benefit more from team training than smaller teams?

Answers to these questions will go a long way to- ward advancing the literature and providing deeper insights into the effectiveness of team training.

Finally, although there is evidence to suggest that training (and team training) has a significant, positive impact on key measures of business performance (e.g., George, Hannibal, & Hirsch, 2004; Mathieu & Leonard, 1987) and safety (Salas, Burke, Bowers, & Wilson, 2001; Salas, Wilson, Burke, & Wightman, 2006), there is a need to form a more accurate understanding of the impact of team training, specifically. Accurate esti- mates of the efficacy of team training are impor- tant for both researchers and practitioners because tens of millions of dollars are spent annually on team training in military, aviation, health care, and other industry settings.

In short, the present research provides a sub- stantial contribution to the current understanding of the efficacy of team training. In comparison with other quantitative reviews (e.g.. Salas, Nich- ols, et al., 2007) and qualitative reviews (e.g.. Salas et al., 2001,2008), the current effort uses more rig- orous methods to assess a greater number of pub- lished and unpublished primary studies; distills knowledge from a wider range of interventions, outcomes, and moderator variables; and offers actionable guidance to organizational decision makers. Before describing in detail the research questions and hypotheses posed in the present ini- tiative, a brief discussion of teams, team perfor- mance, and team training is provided.

TEAMS, TEAM PERFORMANCE, AND TEAM TRAINING

"Teams embedded in organizations exist to

906 December 2008 - Human Factors

perform tasks" (Ilgen, 1999, p. 131). These tasks are often complex, difficult, and dynamic. They re- quire expertise, experience, and the perspectives of multiple individuals synchronizing their work to perform collectively in the pursuit of common goals (Mohammed & Ringseis, 2001).

For the purposes of this meta-analysis, teams are defined as a distinguishable set of two or more individuals who interact dynamically, adaptively, and interdependently; who share common goals or purposes; and who have specific roles or functions to perform (Salas et al., 1992). Teams can be dis- tinguished from work groups by their task, feed- back, and goal interdependencies (e.g., Saavedra, Earley, & Van Dyne, 1993) as well as by their formal structure and coordination demands (Tan- nenbaum, Beard, & Salas, 1992). That is, groups are usually more loosely constituted. They are col- lectives that may be perceived as social entities, and they may even have common goals, but they also possess task connections that are less well de- fined (e.g., juries, committees, councils).

For the current research integration, team per- formance is defined as an emergent phenomenon resulting from the goal-directed process whereby members draw from their individual and shared resources to display taskwork processes, team- work processes, and integrated team-level pro- cesses to generate products and provide services (Kozlowski & Klein, 2000; Salas, Stagl, Burke, & Goodwin, 2007).

Frameworks of team performance and effec- tiveness generally follow an input, throughput, and output format (e.g., Hackman, 1983,1987; McGrath, 1984; Nieva, Fleishman, & Rieck, 1978; Salas, Stagl, et al., 2007; Steiner, 1972; Tannenbaum et al., 1992) in which inputs might include individual and team characteristics, capabilities, and states; through- puts include team communication, coordination, collaboration, and decision-making processes; and outputs consist of the goods or services produced by a team. Stakeholders make judgments about the effectiveness of the quality, quantity, and timeliness of the products or services produced, as well as about the changes in the team and its members that result as performance unfolds (Hackman, 2002).

Training is a systematic, planned intervention designed to facilitate the acquisition of job-related KSAs (Goldstein & Ford, 2002). Concerning team training, one widely cited definition positions it as "a set of tools and methods that, in combination with required [team-based] competencies and

training objectives, form an instructional strategy" (Salas & Cannon-Bowers, 1997, p. 254). Task- focused team training enables team members to become aware of, learn about, and practice requi- site team competencies (i.e., KSAs) and perfor- mance processes while receiving feedback on their performance. Moreover, similar to that of individ- ual training, the science of team training involves identifying the optimal combination of tools (e.g., team task analysis), delivery methods (e.g., practice based, information based, demonstration based), and content (e.g., knowledge, skills, attitudes; Salas & Cannon-Bowers, 1997).

Many narrative and empirical evaluations of team training have been conducted (e.g., Denson, 1981; Dyer, 1984; Marks, Zaccaro, & Mathieu, 2000; Mathieu, Heffner, Goodwin, Salas, & Cannon- Bowers, 2000; Salas et a l , 2001,2006). Although the majority of these assessments have yielded ample support for team training, differences have been observed in the efficacy with which assorted team training strategies improve team outcomes. In an effort to summarize the relative contributions of various team training strategies. Salas, Nichols, et al. (2007) recently conducted a meta-analysis on the subject. Specifically, they investigated the relative efficacy of three team training strategies: cross-training, coordination and adaptation train- ing, and team self-correction training.

Their database consisted of 28 effect sizes (from seven studies) assessing the efficacy of team train- ing for team performance. For the overall test, there was a low, significant tendency for team training to improve team performance (r = .29). However, when they examined the relative efficacy of dif- ferent team training strategies, cross-training did not appear to be effective (r = -.09), whereas the efficacy of self-correction training (r - .45) and team coordination and adaptation training (r = .61) was confirmed.

Although the findings from the Salas, Nichols, et al. (2007) team training meta-analysis suggest that coordination and adaptation training is ef- fective for producing team performance im- provements, some caution should be exercised in interpreting these results and directly comparing them with those of other meta-analyses. The meta- analytic technique that Salas, Nichols, et al. (2007) utilized was based on procedures outlined by Mullen (1989), Mullen and Rosenthal (1985), and Rosenthal ( 1991 ). That approach to meta-analytic integration is considerably different (e.g., use of

TEAM TRAINING EFFECTIVENESS 907

transformations, not calculating confidence inter- vals) from the procedure we utilized in the present study, which followed the procedure provided in Hunter and Schmidt (2004).

In sum, the findings from qualitative and lim- ited quantitative research on team training inter- ventions have largely supported the effectiveness of these techniques. However, none is as compre- hensive, robust, or systematic as this one. This is (to our knowledge) the first thorough research en- deavor undertaken to gauge the impact of team training on team outcomes. Given the widespread use of team training in many important settings, it is also a timely and critical integration.

HYPOTHESES: THEIR RATIONALE AND CONCEPTUAL FOUNDATION

As noted, several research questions concern- ing the importance of team training for enhancing targeted team outcomes guided this effort. In an effort to address these questions, we present test- able hypotheses and their theoretical rationale in the following sections. A model that graphically illustrates these research questions is offered in Figure 1.

Does Team Training Work?

The first question posed in this meta-analysis is whether team training works. This general ques- tion treats all forms of team training as equivalent and, likewise, treats all targeted outcomes of team training interventions as interchangeable. Al- though simplistic from a scientific standpoint, this overarching issue is the single most important question posed in the current study. It asks, in a general way, whether team training is useful as a lever to produce change. If the answer to this question is no, then an enormous amount of sci- entific effort and limited organizational resources have surely been wasted. Moreover, such null re- sults would more broadly call into question the widely accepted premise that planned interven- tions are useful in some measurable way.

Fortunately, research on instructional design theory and, more recently cognitive, metacognitive, and macrocognitive theory, collectively suggests that team training is indeed useful for prompting meaningful change and in helping teams achieve performance levels deemed effective by organiza- tional stakeholders. Supporting this growing base of theory are the mounting findings from primary

Training Content

Taskwork Teamwork Mix

Team Training

Moderators

Team Membership

Stabiiity

• Intact • Ad Hoc

Team Size

' Small ' Medium • Large

H3-H6 H7 H8

HI -H2

Outcomes

Cognitive

Affective

Process

Performance

Figure 1. Model depicting study hypotheses (H1-H8).

908 December 2008 - Human Factors

team training studies (e.g., Ellis, Bell, Ployhart, Hollenbeck, & Ilgen, 2005; E. E. Entin, Serfaty, & Decken, 1994; Klein, Stagl, Salas, Parker, & Van Eynde, 2007). Given the abundance of theoretical support and empirical evidence speaking directly to the usefulness of team training, the following hypothesis is advanced:

Hypothesis 1: Team training is positively re- lated to overall team outcomes.

Separating Outcomes

In any meta-analysis, a balance must be struck between casting too wide a net (i.e., including widely disparate independent and dependent vari- ables) and being too restrictive (i.e., reducing the meta-analytic database to a small number of stud- ies). Clearly, results likely to be obtained from the examination of the first research question can be criticized for thoughtlessly combining "apples and oranges" and concluding that "fruit tastes good" (Kraiger, 1985). Thus, there was a need for more finely tuned analyses that examined the relation- ships between more specific and restrictive defi- nitions of independent and dependent variables. This set of analyses is directed at the next question posed in this study, namely, "Is team training, when conceptualized as a single overarching interven- tion, an effective technique for producing planned changes in specific types of team-level outcomes?"

This issue is important, as team researchers have offered compelling propositions about (Marks, Mathieu, & Zaccaro, 2001), and meta-analytic evi- dence for (LePine, Piccolo, Jackson, Mathieu, & Saul, 2008), behavioral and cognitive processes constituting teamwork. Team performance emerges as teams and their members draw upon individual and shared characteristics, capabilities, and cogni- tive and affective states to enact the core processes that constitute teamwork (Burke, Stagl, Salas, et al., 2006). In turn, the performance outcomes result- ing from this unfolding, recursive process are eval- uated by organizational stakeholders in terms of their effectiveness (Hackman, 2002). If the utility of team training is to be fully realized, it must be a lever to affect all of the core components of team- work and produce performance outcomes that are deemed effective in the workplace.

Gagné (1962) astutely argued that the most fun- damental design issue in training is the specifica- tion of what is to be learned. This is the specification of training or instructional objectives. These objec-

tives specify the knowledge, skills, beliefs, atti- tudes, or choice behaviors that a trainee or trainees should be capable of accomplishing upon success- ful completion of the training program (Campbell & Kuncel, 2001; Goldstein & Ford, 2002). Team training is no different from individual training in this regard. Once these objectives are chosen, the training environment is tailored to achieve them in the most efficient way possible. Thus, in- structional objectives provide input for the design of training programs as well as the criteria used to judge the programs' adequacy (Goldstein & Ford, 2002).

Because the primary studies included in the present meta-analytic initiative addressed a wide variety of instructional objectives, a meaningful taxonomy for organizing training outcomes was needed to guide this effort. In line with the team theory noted previously and with prior team train- ing meta-analyses (e.g.. Salas, Nichols, et al., 2007) - and consistent with Kraiger, Ford, and Salas's (1993) taxonomy - training outcomes can be classified as cognitive, affective, and perfor- mance outcomes as well as behavioral processes. Each of the training outcomes investigated in the primary studies included in the present meta- analysis were classified into one of these four taxons.

In the present study, team member cognitive outcomes predominantly consisted of declarative knowledge gains from pre- to posttest measure- ment. Team member affective outcomes included socialization, trust, confidence in team members' ability, and attitudes concerning the perceived ef- fectiveness of team communication and coordina- tion processes. Team processes examined in this review included behavioral measures of commu- nication, coordination, strategy development, self- correction, assertiveness, decision making, and situation assessment. Finally, the assessment of team performance integrated quantity, quality, ac- curacy, efficiency, and effectiveness outcomes. To help answer the question posed at the beginning of this section, the following hypotheses are ad- vanced:

Hypotheses 2a to 2d: Team training is positively related to team-level (a) cognitive outcomes, (b) affective outcomes, (c) behavioral processes, and (d) perfonnance outcomes.

Training Content

It is interesting to know whether team training

TEAM TRAINING EFFECTIVENESS 909

is useful for improving specific team outcomes, but it is perhaps more importatit to ask, "What type of team training is most effective for improving team functioning?" To this end, the current re- search addresses the relative influence of specific training interventions on valued team outcomes. Although particular team training interventions are often referred to by many different names, it can be argued that every team training interven- tion targets some combination of taskwork or teamwork skills (e.g., Cannon-Bowers et al., 1995; Goldstein & Ford, 2002; Mclntyre & Salas, 1995; Salas et al., 1992). Thus, three content categories were coded in the present study: taskwork, team- work, and interventions that utilized a mix of taskwork and teamwork content. Illuminating the relationships between these three types of team training and each of the four types of outcomes noted previously is important to the objective of the present study.

Team training interventions targeting taskwork KSAs seek to develop the technical competen- cies of team members (Salas et al., 1992). Cross- training is a common example of a team training intervention that includes a heavy taskwork com- ponent. In cross-training, members learn about and sometimes practice the responsibilities of mul- tiple positions and/or roles, including those of their teammates. Having acquired interpositional knowledge and honed teamwork KSAs, indi- viduals who have been cross-trained are better prepared to recognize when their teammates are overburdened and to step in and perform their co- workers' duties when called upon.

In contrast, training interventions targeting teamwork KSAs are focused on improving how individuals work together effectively as a team. For example, teamwork skills targeted by team training may include mutual performance moni- toring, feedback, leadership, management, coor- dination, communication, and decision making (e.g., Cannon-Bowers et a l , 1995; Stagl, Salas, & Fiore, 2007). Team coordination and adaptation training is one example of a team training inter- vention that is focused on providing teamwork skills content.

Hypotheses 3 to 5 expand upon the basic ques- tions addressed in the earlier hypotheses because, assessed in their entirety, they provide preliminary estimates for determining which configurations of team training content are better than others. How- ever, given the absence of a priori expectations for

this set of hypotheses, the investigation into the tnoderating impact of traitiing content is explora- tory. Nonetheless, the following hypotheses are advanced to investigate these questions:

Hypotheses 3a to 3d: Taskwork training is pos- itively related to enhancements in team-level (a) cognitive outcomes, (b) affective outcomes, (c) behavioral processes, and (d) performance outcomes.

Hypotheses 4a to 4d: Teamwork training is pos- itively related to enhancements in team-level (a) cognitive outcomes, (b) affective outcomes, (c) behavioral processes, and (d) performance outcomes.

Hypotheses 5a to 5d: Team training that includes a combination of teamwork and taskwork con- tent is positively related to enhancements in team-level (a) cognitive outcomes, (b) affective outcomes, (c) behavioral processes, and (d) per- formance outcomes.

One truism of collectives is that teams of ex- perts often fail to evolve into expert teams (Salas, Cannon-Bowers, & Johnston, 1997). In other words, the possession of copious amounts of task knowledge is insufficient to overcome the perfor- mance decrements that can result from the coor- dination demands teamwork imposes on team members.

In order to assist teams in fulfilling their poten- tial, it is often necessary to move beyond the pro- vision of mere task-specific content to feature teamwork and team interaction training as well (Hollenbeck, DeRue, & Guzzo, 2004). For exam- ple, a team of medical professionals that possesses a high degree of skill in mutual performance mon- itoring will not only have the task expertise to of- fer suggestions but will also know when to do so in a timely manner. Therefore, team training that targets both task-specific content and generic teamwork competencies is expected to provide the greatest benefit in terms of enhanced team out- comes. Given this line of thinking, the following hypothesis is advanced:

Hypothesis 6: Team training that includes a com- bination of teamwork and taskwork content will be more effective than interventions that target either phenomenon in isolation.

Team Membership Stability

The aforementioned research questions will help illuminate the importance of team training as

910 December 2008 - Human Factors

a lever for facilitating change in a variety of team- level phenomena. However, it is important to move beyond these preliminary questions about the main effects of team training to begin to address other pertinent questions, such as, "Under what conditions is team training most effective as a planned intervention?" Thus, in the remainder of this section, questions are posed and hypotheses are advanced to address additional variables that may moderate the effect of team training on team- level outcomes.

Of primary importance to the present meta- analysis are the potential moderating roles of team membership stability and team size. Membership stability addresses the length of time team mem- bers have worked together interdependently to solve joint challenges, and team size indexes the number of team members assigned to a given col- lective.

In regard to membership stability, studies of team training interventions generally consist of two varieties: (a) studies of training interventions con- ducted with intact teams, whose members have a shared history as a result of a commonly held as- signment to a given collective operating inside an organization (e.g., an energy exploration team); and (b) studies of interventions delivered to a group of ad hoc strangers purposively assembled to con- duct either basic or applied research in a contrived setting (e.g., a tank crew composed of randomly selected and assigned students performing in a university lab). In comparison with their ad hoc counterparts, intact teams tend to have relatively stable membership and a shared history of work- ing together (Devine et a l , 1999). The total sam- ple of primary studies included in the present meta-analysis is representative of both intact and ad hoc teams.

In their recent meta-analysis on team training. Salas, Nichols, et al. (2007) made a conscious decision to examine the effects only of interven- tions that were delivered to intact teams. How- ever, these researchers also suggested that future meta-analytic efforts should examine levels of team membership stability as a potential modera- tor variable. We accept that call and will investi- gate whether team training works better for intact or ad hoc teams.

Based upon prior research addressing the stages of team development, it seems plausible that team training will be more effective for intact teams. This is because intact collectives presumably have

already navigated at least some of the process chal- lenges that characterize the preliminary, and often rocky, stages of group development (e.g., storm- ing, norming [Tuckman, 1965]) that newly formed teams must overcome to develop cohesion. The baseline level of shared expectations, competen- cies, and emergent states developed from a shared history will likely free up cognitive and interper- sonal resources, allowing intact teams to focus more of their resources on skill acquisition during team training. Given these arguments, the follow- ing hypothesis is advanced:

Hypothesis 7: Team training will be more effective for intact teams than for ad hoc teams.

Team Size

This research also examines the relative impact of team training for teams of varying sizes. It has been noted by several researchers that teams are highly effective when they have a sufficient, but not greater than sufficient, number of members to perform team tasks (Guzzo & Shea, 1992; Hack- man, 1990). The definition of sufficient, however, is somewhat unclear, in part because the research findings on the impact of team size on effective- ness have been mixed.

Some studies have found that larger teams are more effective (Magjuka & Baldwin, 1991 ; Yetton & Bottger, 1982), whereas other studies have re- ported that larger teams suffer from coordination and process losses (Gooding & Wagner, 1985; Mullen, Symons, Hu, & Salas, 1989). Kozlowski and Bell (2003) summarized these mixed results, concluding that the benefits of larger teams are dependent on the nature of the task and the team's environment. For example, one study that evalu- ated team size and its impact on the performance of tasks that required innovativeness found a negative correlation between team size and team processes, r = -.18 to r = -.51 (Curral, Forrester, Dawson, & West, 2001).

Also bearing on this issue is the seminal work of Steiner (1972), who examined how increasing the size of the team impacted team processes. Steiner's (1972) findings suggested that an increase in team size resulted in increases in productivity but also introduced inefficiencies (e.g., decreased motivation, poorer decision making, poorer coor- dination, and higher levels of conformity). Other studies have also found additional detrimental effects attributable to large-sized teams - for

TEAM TRAINING EFFECTIVENESS 911

example, an in-group bias effect (Mullen, Brown, & Smith, 1992), a participation leadership effect (Mullen, Salas, & Driskell, 1989), a decrease in group liking (Indik, 1965), and a decrease in per- formance (Mullen, 1987).

A common undesirable group phenomenon is social loafing. It has been found that larger groups are more prone to social loafing (Latané, 1981). Because larger groups lose the connection between their inputs and the rewards that are received, in- dividuals in large teams become less motivated to perform (Kidwell & Bennett, 1993). Karau and Williams (1993) discussed what they termed the "dilution" effect. This effect manifests itself in a variety of dysfunctional ways (e.g., free riding, get- ting lost in a crowd, shirking group work), all of which are associated with an increase in team size.

A study by Hackman and Vidmar ( 1970) set out to find the perfect size of a team. Based on ques- tions asked of teams large and small, they found the optimal number of team members to be 4.6. Moreover, in a recent study by De Dreu (2007), a weak correlation was found between team size and the measured outcomes of learning (r = . 15) and team effectiveness (r = . 16). De Dreu also found a negafive correlation between team size and infor- mafion sharing ( r = - . 11). These results show that the key to enhancing team functioning does not necessarily lie in increasing the size of the team.

The current research categorized teams into three groups; small teams (n = 2), medium teams (2 < n < 5), and large teams (n a 5). A recent meta- analysis on team building interventions found that large teams benefited the most from team building intervenfions, resulting in a mean true score corre- lation of .66 (Klein et al., in press). Given this find- ing and past findings suggesting that medium-sized teams will already be performing at a high level, we suggest that team training interventions will show greater effects for dyads and large teams. In other words, team training, whether it is geared to- ward teamwork or taskwork, will aid small and large teams more than medium-sized teams in im- proving their inefficiencies. Given this thinking, the following hypothesis is proposed:

Hypothesis 8: Team training will be more benefi- cial for small and large teams than for medium- sized teams.

METHOD

This study applied meta-analytical techniques

to previously conducted research in an effort to obtain a numerical estimate of the relationships between team training and team outcomes. Analy- ses were also conducted in order to determine the existence of moderators. The following sections describe in detail the identification, selection, and coding procedures for primary studies. In addition, a description of the method for calculating effect sizes is provided.

Literature Search

More than a half century ago, B. Glass (1955) observed that "no problem facing the individual scientist is more defeating than the effort to cope with the fiood of published scientific research, even within one's own narrow specialty" (p. 583). And even within the relatively narrow specialty of team training, there is indeed a rising tide of empirical research. Surfing this tide required a calculated line of attack. In total, four distinct and diverse ap- proaches were used to identify studies for poten- tial inclusion in the meta-analytic database.

First, comprehensive electronic searches of computerized databases were conducted using multiple combinations of appropriate keywords (e.g., teams, cross-training, team performance; a full list of keywords used can be obtained from the authors). Specifically, the search engines, electronic databases, abstracting services, and proceedings of Google Scholar, ScienceDirect, EBSCOhost, Academic Search Premier, Business Source Premier, PsycINFO, PsycARTICLES, Mil- itary and Government Collection, SPORTDiscus, the Interservice/Industry Training, Simulation and Educadon Conference, and the Defense Technical Information Center were searched for articles pub- lished through August 2008.

Second, a targeted electronic search of the fol- lowing journals was conducted, using the key- words team training and group training, with no restriction on dates: Journal of Applied Psychol- ogy, Personnel Psychology, Military Psychology, Organization Development Journal, Academy of Management Journal, Small Group Research, and Human Factors. Third, an ancestry approach was employed, whereby the bibliographies and refer- ence sections of relevant studies that had already been retrieved were searched to locate earlier rel- evant studies. Einally, an informal network of col- leagues was queried for potential unpublished articles, manuscripts, and conference papers. The complete study identification and search process

912 December 2008 - Human Factors

resulted in more than 500 artieles for possible in- clusion.

In the next phase of the literature review, two authors reviewed abstracts and online versions of the identified árdeles and came to a consensus con- cerning whether the full text document should be obtained. At this stage, articles containing the fol- lowing characteristics were excluded from further analyses: (a) master's theses; (b) use of clinical populations; (c) use of children as participants; (d) collectives and groups characterized by a defi- cient degree of task interdependence; (e) studies failing to report a usable test statistic (e.g., F, t, y}, z, d, r) or the raw data necessary to calculate these statistics (e.g., means, standard deviations, sample size); and (f) studies failing to utilize an actual team intervention.

As a result of this effort, 168 relevant published and unpublished papers were identified. These articles, book chapters, dissertations, technical reports, and conference proceedings were subse- quently obtained in full text and coded by study authors. A discussion of the specific coding strat- egy employed for these articles is presented next.

Coding Procedure

Various meta-analytic integrations often use a similar coding scheme to quantify study char- acteristics and results (Lipsey & Wilson, 2001; Stock, 1994). The coding strategy used in the present research included capturing 20 pieces of information from each study: (a) publication type, (b) study setting, (c) study design type, (d) nature of the organization and participant sample, (e) team type, (f) team membership stability, (g) task interdependence, (h) number of teams, (i) average team size, (j) predictor level of analysis, (k) crite- rion level of analysis, (1) training content, (m) name and description of training intervention, (n) method of training intervention, (o) criterion report type, (p) criterion description, (q) reliability of predic- tor measures, (r) reliability of criterion measures, (s) effect size or sizes, and (t) recommendation for inclusion.

For the criterion level of analysis coding, the initial database consisted of 19 effect sizes at the individual level and 93 at the team level. Although it is desirable to include in the subgroup analyses the largest possible number of effect sizes, re- searchers have strongly recommended against mixing levels of analysis in research integrations (e.g.. Gully, Devine, & Whitney, 1995; Hunter &

Schmidt, 1990). For the purposes of this research, only team-level outcomes were considered eligi- ble for analyses.

Rater Reliability

One or more study authors independently coded each of the 168 articles originally identified for possible inclusion in the meta-analysis. The coders for this research had previously pilot tested the coding scheme and possessed an adequate level of expertise in the substantive areas being coded. Moreover, before coding began, the au- thors met to discuss and develop detailed instruc- tions for coding the 20 categories of information from primary studies. More specifically, a resource manual was developed in order to provide the coders with standardized information to code the primary studies. Coders met in order to discuss the coding manual and ensure a shared under- standing of the concepts being coded. Furthermore, the coders were trained by coding two articles and then holding a meeting to discuss the information extracted from the articles. Once a satisfactory level of understanding of the categories was reached by all coders, the coding for the present meta-analysis began.

The coding was done in stages, with two checks performed to assess interrater reliability. First, two authors and one of their colleagues (who was blind to the study hypotheses) each independently coded 15 articles from the database. At this point, the in- terrater reliability for the three coders was assessed by calculating intraclass correlation coefficients (ICCs; Nunnally, 1978; Shrout & Fleiss, 1979). The ICC is a measure of reliability among two or more raters. An ICC can be conceptualized as the ratio of between-group variance to total variance. When an ICC approaches 1 it can be interpreted that the variance between the raters is essentially zero; thus the raters give the same ratings. Further- more, the variation in the ratings or measurements is attributable solely to the target being rated.

The reliability at this stage was deemed satis- factory (ICCs ranged from .75 to .85) and the full coding procedure continued. The second check on interrater reliability was performed near the end of the coding process, when 10 more articles were chosen to be coded by each of the three coders. The reliability of the coding for these final 10 arti- cles was assessed at the end of the coding process and again was acceptable (ICCs ranged from .85 to 1.0). Taken together, 25 of the 168 articles

TEAM TRAINING EFFECTIVENESS 913

were coded by two authors and a colleague. These 25 articles represented approximately 15% of the total number of articles.

After the 25 reliability sample articles were coded, the combined codings were assessed for rater reliability. We were particularly interested in the coding for the data that would be contributing to the investigation of the hypotheses. We were also keenly concerned with whether the coders re- liably coded the same effect sizes and whether they made the same decisions in regard to including the article in the final database. Upon examination, the coding of the data used for the hypotheses was sat- isfactory.

Specifically, for the criterion description cate- gory, the ICC (3, k) = .89. Similarly, for training content, team membership stability, and team size, the ICCs (3, k) were equal to .77, .94, and 1.00, re- spectively. Moreover, the coding for the effect size or sizes and inclusion/exclusion decisions were highly reliable (ICCs = .99 and .89, respectively). These estimates of the reliability with which 25 common articles were coded gave us confidence that the coding for this research was performed reliably. The remaining 143 articles were coded independently by one of the two authors who also served as coders for the reliability sample. Any initial ambiguity or uncertainty about whether to include a particular study result or characteristic in the final meta-analytic database was resolved through a consensus discussion by two or more study authors.

Meta-Analysis Procedure

The data analysis for this research was aided by the Hunter-Schmidt Meta-Analysis Programs 1.1 (Schmidt & Le, 2005). Effect sizes were first sorted within their associated subgroups (i.e., outcome type, training content, team membership stability, team size), and then a combined effect size esti- mate was generated for each subgroup and/or level of each moderator (e.g., small, medium, and large teams).

This software utilizes a random effects model, rather than a fixed effects model, to analyze the data. This model allows the true effect sizes to vary, instead of assuming that the true effect sizes have fixed values. Field (2001) noted that the ran- dom effects model is considered to be "more real- istic than the fixed-effect model on the majority of occasions" (p. 162). The data output for the pro- gram includes (but is not limited to) the mean true

score correlation, the standard deviation and vari- ance of true score correlations, credibility inter- vals, and the percentage of variance attributable to observed correlations after the removal of arti- facts. Confidence interval estimates were calcu- lated separately in Excel using formulas provided by Hunter and Schmidt (2004).

Effect Size Calculations

In order to aggregate findings across studies, all effect size estimates culled from primary studies were converted to a common metric, r (correla- tion). Formulas found in Hunter and Schmidt (2004) were used to transform primary study ef- fect sizes that had been reported as other statistics (e.g., t, F, d, x^, or Z). Upon being placed on this common metric of effect size (i.e., r), the primary study results can be pooled and evaluated for fit with the predicted hypotheses in this research.

An important issue that presented itself was that several studies contained more than one effect size estimate. This circumstance occurred for two rea- sons. First, several studies reported findings for separate samples under investigation. In this in- stance, it is acceptable to include the findings as separate effect sizes in the database. Alternatively, a large number of studies reported multiple effect sizes from single samples that were either (a) for the same outcomes or (b) for different outcomes. For example, a study might have reported two process outcomes and one performance outcome.

To be clear, however, stochastically dependent effect sizes were not included in the current data- base (which, if included, would patently violate the assumption of independent effect sizes). In other words, findings generated on the same (or similar) outcomes from the same sample were averaged prior to entry in the database. In contrast, when studies reported findings from the same sample on different outcomes, these findings were included separately in the overall database but were not included in any of the same subgroup analyses. Effect size estimates from primary studies that were combined prior to entry in the meta-analytic database are denoted with an asterisk in Table 1.

Corrections for Unreliability and Range Restriction

Meta-analysis is especially useful for determin- ing whether confiicting results in the literature are attributable to artifactual or actual variation (G. V. Glass, 1976; G. V. Glass, McGaw, & Smith, 1981;

text continues on page 917

ro XI

ro Û u

% ro c

¿ d>

0

T! 0

T>

c

TI c ro

0 T3

m

0 1

ç̂ ro

^

Q.

Ic tf)

0 JD

E 0

c

c 0 U

li ty

lo ro

: LO

c o

c '+: O Q.

•C 0 UQ

0 0.N en LO £ E 5 ro

o

. — . — . — CÍ

o o 5 § E E ro ro

u u O O

u u O o

u u O O

O O

E E ro ro 0 0

u u O O

O O O O O 5 5 5 5 5 E E E E E ro fü (TI ro ro roro >< >< >< >< ><

,d) , 0 • ^ - ^ - ^ "=c "s;

0

E ro

0 5 E ro

o 5 E ro

O 5 E ro

O

i tf)

t;« tí u u u O O O

u u O O

u u o O

u u O O

•O"D"Ö •!2 << < ^

tj tj ro ro

u o o o

O D Ö ! T 3 O C Ö Ö D ] i { i i o o O T 3 Ö T 5 O ö Ö X ! O ö ;

<< < ̂ << << << £££££ <<< << << < << SS << s

0 0 u u c c ro ro

Si ê R.P o

CL Q. Q.

0 0 U U C C ro ro

0 2 0 CL CL CL

0 U C ro Qj

tf) F > Sí! E '-P 0 O t! o'^ i?

CL CL <

0 u c ro E 0

0 CL

0 >

0

d) d) u u c c ro ro £ E

0

0 uc ro0

0 uc ro

0 uc ro

0 u

^ iV \\J ^ %\J ^ IV «W »W ^

I ^ E ^ Ê I Ê :l Ê E s E :|

Ü . Q . CL û .

C d)

tf)

0 2 ^ o^^ 0 ^ 0 ^ 2 0 0^2 2 C L C L C L U Q - U C L Q - Q - Q - U D - D -

00 o o 00 00 00 00000 000 00 00 o 00 00 00 o 00 o o 00 oouiin 00000 000 00 00 o 00 00 00 o csicsi cô o roro csicN

o 0 0 0000 r-

r-^ró r o r o r o rô en ro rôro iriuS rg

roronroro O O n roo

n r o oor̂ inm o

OO

r Loro

* * * KX rooinooo ocMO 00001^0 î roro

moo c\ic\i 00 roro 0 0 0 * o»—

o ir)ir) roro o o o o

ro o o

i c g o o o o o ^ C M C M C v i c g

ooooco o o ô

OO »-«— o o o o o o o o C M C M C M C M

roro o o o o C N C M

o o o o C N I C N

sn ic

& B

re e ri.

"ro

O c ro

is ti

U

an .

J 0

T l <

0 0 en

o8

er .

a 0

0 TI c

0

<

Í o8 t/í"

re s:

tío u

-a ro

X

ik s .

ro CQ

a la

s

LO

o8

0

0 QQ

O

c ro U 0

0 "0 tf)

0 _ ^ u

la s

ro LO o8

ce .

c oí

u

n n ii

ro CQ

c

2 m

c _ro 0 0

08 _0 O) ro

N

CQ

0

o m 08

I 0

Tl tf) c 0

m

ro co

if d>

O CQ C

o c ro U

O U

le lm

e l,

& H

0

0"

O o u

2 0 0 0

ro

0 di

(C o o k

al .

+-' 0 0

0 0 U

ro

0 0

O o U

ro +.1 d) c ro c D

Q

c 0

—

o8

b e ck

.

c _ 0

, H o l

o yh

a rt

,

^ " 0 GQ

tfT 1 =

LU

ro 0

LO

an , &

u ro

E n tin

, f

- .— c

UJ

cd LÜ

914

II ro o

^^ u

Q.

0 ¿̂

s CO

0

ite r

U

c O Q .

ti 0

û

in

D C

e:

i _

Q .

0

C ro E

rf o t

0 Q .

0

C ro E

rf o r

0 Q .

0

c ro E

rf o i

0) CL.

U)

0 U O t:

Q .

0

c ro E

0

c ro E

0 0 tt0 Q .

0 Q..

0

C ro E

rf o r

0 Q-

0

c ro E

rf o r

0

in 0uo

Q_

0

c ro E

rf o r

0 Q .

0 c ro E

rf o r

0 Q .

0

c ro E

rf o r

0 Q .

0 c ro E

rf o r

0

0 c ro E

rf or

0 Q .

0 c n E

rf o r

0 CL

0 0 .ü

« c 1 o u

CO <

I ro

O

E ro

ii E E m ro

o O

E E ro .0

o o o

ra ro ro . > < . > < . > < .>< .>< .>< ,0 .0 ,0 *^ "ïi '^ "^ "^ *^

O O "Í "£

i i o o

u u u 0 0 0

u u u o o o ëë o |

T! "O Tl T! t; t; ro ro

u u u O O O

u u O O

t; t; ro ro

u u u O O O

u u u O O O

^ ^ ^ +;4i:"ÖT3T3'ÖT3"D ti+i'D'O'DTJ'O'D

0 U) U)

0 u 2CL

0

1.ffe <

0

l _ CL

0 U C ro

^0 û_

0

••g

! t <

0

2 CL

0 U C ro

t: 0CL

0

•g 0 i t <

m 0 u 2Q .

0A iq

' c

o u

0

ti v

' c

o u

0

ti v

•g

o u

0 >

•g

O U

l/l

1 ^

O Û .

o o o o o o o o r o o o o o o o o o o o o o o o o o o o o o o o o p o p p o p p r - . r o p p p r - . i n p p o p p p p p p p p p p p p o p p i / S u S L Ó i n c v J r v i c s j ^ L n u i u S c \ i ^ c s i c g c s Í M M C \ i C N Í c v i C N Í C N C v i ( \ i c \ i r o r o r o c g c \ i c N

CM CM

- o • ro ro ro 00 1 ^ » -

o in C M C M T - T - » - T - » -

ro LnmLn m in -OLnr^ o

<- o * * T— » — i n o c o ^ o o c M ^ o o - O ' - C M i n r - ' ^

o o o • * 00 00 00 o* ^N ^N O^ ^^ O- ^^ O O O o o o o

T~" r^ ^" ^" r^ r^ o^ ^~~ C7> r̂ ^ r^

Qs ^ s ^S ^^ ^s Qs ^s <^ ^s ^3 ^3 » - » - T - V - . - T - T - C N I T - C V J C N I

00 CO in ^̂ in D̂ -O -O »i ^ 1̂ in in m

CO o ^ 1̂ in o "* CO ' i o CO T-

C\l CM o o o o o o o o o o o o o o o o CM c g CNI CM CM CM

o8 ^ 00

_0 Q. a o

Q_

o8

0 l/l o8 c

' 4 - ' c

LU

LU

0 CO

c" '4-»

UJ

LU

e n .

(~ 0 U c" ro

0 0

(/>

E ro

W ill

i

0

c m

4 ro

_u ro u

_c

08

> ro

Q

J5 ro

08

c ro

0

0 CL

08

O

0 é ^ 0 0

É ro CD - C 4-' ^

0 0 JP O c >

XX j |

N :=. Ü 0 ro ^ E ^

CO .SI c CO

X o u~ c

_ro CQ

0

c o

iñ 08 E o

T3 0 0

0 CO

c8

E n tin

, E

n tin

, 1

_ra

u ro

0

:c a i

N

rk e ,

&

CQ

" 0 SI ro to

rk s.

ro

D 0

M a th

i ca

ro ,

&

u ro

N

rk s.

ro

08

J2 ro to

. |

TI 0 0

(J3

e ff n e r, '

X

th ie

ro

-B o w

e r

c 0

C an

e p re

s ri

sk (

01

w ith

a n

d e n o te

d

-e

íe s

3 C

tS Í2

ar u ]

915

. E <u

ro o

g. le

Elu <u ro

2 i!n

c g

i in

UÛ

(1) 4) Cd

• o O) 3

I O U

LU

m <

O O O O O O O " g

E E E i - r o r o

5 5 5 5 E E E E ro ro (Ü ro

,<l> ,(U ,0) ,0) .<u . « u ,

u u u u O O O O *j *j

"O "O "O "O +-• -M <<<< ^Ç

,<D

u O

îî E E ro ro

.(U .(V

0

E ro

0

E ro

0

E ro

O

E ro

.(U

O 5 E ro

.0) X

"^

o E ro

0 5 E ro

.0)

0 § E ro

.0) x

0 5 E ro

.9)

0

E ro

.0)

O 5 E ro

.0)

0

E ro

.0)

O o

tj u o o

o u u u O O O O

u u o o

"O "O "O T3"O "D

U U U tJ rorororo _ ç ^ ^ ^ JE < < ,£ ,£

u u u u u u O O O O O O

c t> ¡u O) (U y Ott 2 u < Q-

(U u c ro Ê

b (U

0 .

0)

ff e e t <no!

u

2 Q.

(U U C ro Ê

b (U

0 .

(U u c ro E

e rf

o r

(U u c

d) ro .> E

ff e

e t

e rf

o r

< Ü-

!u6o

U

^ Q)

o g n i

ff e e tI :l -B a

u c ro

i o S (Uuc

(V (V (U (U > > > > u u tj tj

•$ SE s i :l -B a i a i 0)

2 ^ 2 Q . Q . Q . U '

U >

. a . Q. Q. Q.

OOOO o o o o o o o OOOO oo OOOO o o o o o OOOO o o C3C3OCD o o o o oc> O oocDO ro L/iLoiniT) O OO o o OOOO o o rorororo rôrooo ro ' i ' * • * rorororo ro CMCNCNCN Ô L/SL/Î i/i >o CMCMCMOSÎ CVJCM

o CN o o 00 0-.

'O

i o r N ^ o o T — ' — r o 0 0 0 0 ^

CM CM ^f ^0 ̂ D ̂ D *O LO ^f LO CO I

•t- h

o CM

CM r- r- CNI ^ T- ̂ .- ro CM CM CM CM

0 0 C )

O CN

CN «O

* ro O

ro CN J— 'S- 0 0

*

o •o ro

.K

ro

CMCMCM in ^ » - Q\ Q\ Q\ Qs ^5 C7̂ C^ O* « - » - r - r - CNCNCN T-

O O O O O O ' - ' - CN C M C N C N C M CN

O CM

O CN

O

M 0)

o >

O) g) m o8

lo r

ro Z

V)

o ro

m

& w

. is

.

LU ^

15

ro (U

Q .

. e

h Ie

ig

< o8 " Ö ro c -

rie l

Q .

3 <J

OJ (U

l e O)

i 1 a J ro ~5

ce, c£

é ta

l

v_ d)

e h re

i

to

c

o —) o8

n tin

.

LU

e rf a ty

.

to

ta l.

a> 0

h a p ii

to

c ro E

= ed

et

0 0

ie g e l

to

<u

oLL

as . &

ro to

to u t.

to

o

t; (U Q.

to o8 (D

to

<u

I m o c c ro O

9 1 6

TEAM TRAINING EFFECTIVENESS 917

Hunter & Schmidt, 2004; Hunter, Schmidt, & Jack- son, 1982). Part of this determination stems from an assessment of the amount of unreliability and range restriction that is present in the data. For ex- ample, it is standard practice for authors of meta- analyses to attempt to correct obtained reliability coefficients for unreliability in measures of the predictor and criterion (e.g.. Hunter & Schmidt, 2004). For the current research, it was an unfor- tunate reality that primary studies often failed to report all of the auxiliary information necessary to perform corrections for study artifacts. Criterion reliability estimates were available for only 44% of the effect sizes in the database (41 of 93). Thus, in order to best deal with this issue, we utilized artifact distribution meta-analysis, rather than per- forming corrections for unreliability on each indi- vidual effect size.

In addition to making corrections for the unre- liability of predictors and criteria in primary stud- ies, researchers using meta-analytic techniques often attempt to account for the effects of direct or indirect range restriction. In a recent article. Hunter, Schmidt, and Le (2006) argued that cor- recting for range restriction will generally result in more accurate combined estimates of the rela- tionships between variables. However, the articles included in the current database did not report the necessary information on restricted and unre- stricted samples that would be needed to perform corrections for range restriction. As a result, we could not make these corrections in the current re- search, and our effect size estimates may be con- servative. That is, they may underestimate the true nature of the relationships between team training and team functioning if the biasing effects of range restriction could be corrected.

Weighting

It is often recommended that studies with larger sample sizes receive more weight in meta-analyses in order to increase the precision of the estimate of the average effect size across studies (Hunter & Schmidt, 2004; Shadish, Cook, & Campbell, 2002). Although estimates obtained from studies with larger samples are not necessarily more valid, they are presumed to be more stable (i.e., accurate). At the same time, recent simulation studies have found that one can obtain a very accurate com- bined effect size through procedures employed to weight studies by sample size (e.g.. Field, 2005). Taking these considerations into account, the pri-

mary study effect sizes that were utilized in the current series of meta-analyses were weighted by their sample sizes.

RESULTS

Description of the Database

The final database for this research consisted of a total of 93 correlations obtained from 45 primary studies. These 93 effect sizes represented 2,650 teams. Of these, 1,660 teams were teams from the lab or classroom setting, 762 teams were from the military sector, 138 teams were aviation teams, 80 teams were medical teams, and 10 teams were from business organizations. It should be noted these 93 effect sizes were not all from independent samples (there were 52 independent samples). However, every subgroup analysis that was per- formed included only independent samples. Of the 45 studies included in the meta-analytic database, 31 were published (25 journal articles, 4 confer- ence proceedings, and 2 book chapters) and 14 were unpublished (3 conference papers, 6 techni- cal reports, and 5 dissertations). Table 1 summa- rizes some of the key information that was recorded from each primary study.

Meta-Analytic Results

The meta-analytic results for the four primary areas of investigation are reported in Tables 2 through 5. In each of these tables, key pieces of in- formation from each analysis are displayed. This information includes the number of teams in each analysis (N), the number of independent effect sizes (correlations) in each analysis (k), the mean weighted observed correlation (r ), and the 80% confidence interval for that correlation. In addi- tion, the tables display the estimated true score cor- relation (p), the standard deviation of this true score correlation (5Dp), the 80% credibility inter- val (10% CV and 90% CV), and the percentage of variance accounted for by statistical artifacts.

It is important to explain why both confidence and credibility intervals are reported with this re- search. In general, they each provide information that contributes to the process of estimating the true nature of the relationships between two vari- ables (Whitener, 1990). However, confidence in- tervals are applied to observed scores, are centered upon a single mean score, and take into account the effects of sampling error. Alternatively, cred- ibility intervals provide information concerning

text continues on page 922

o

o o

U

o

1̂ o o CO o 00 q q q̂ q •^ Ö Ö -ó Ö co o o 00 o

o '* m ro

0 0 IT) > * I r - r o •>* I

o o o o o o o o

CM m • ^ r o •

o 'S- ro ro

(U

E o u

•4-«

D

O o a

T3

<u in ro m O) c

' c ro E ro

^ ro ' í ro ro

U

O

Ö

^D sO ^f O"* T*~ ro CM ro CM ro

00 c\i (3^ ro " * ro ro ro ro ro

CM - o lO o CNI T— T— CM " ^ L/í

in 1̂ "̂ r 00 o o CM o "it vo o m

(U

ro c <

CQ <

(U

e o

ëE ro o

O t t 2 aj = U < ü. Q. <

0) _

E S

o s .

Si 8 CQ U

CN « Qj

918

roô ôôô'i CNO o o o m

u <

ooo o o o o ^ i ^ o o i n o o o o o co CNi o o r̂

I T - T - m »- ro ro

1 ^ T— o m CN «— »— CN ro m '

o o ô o o CNi CNJ r o LT) CM

Q LO

^3 o * c^ cr? «- o o o

(^ Ç^ (^ 00

o o o o in o CN o

o »— oo m CN »— r o »— CNJ r o i n • *

U

c 0

O

u 01 c ro

c O Q.

ro CQ

O ) c

E ro

r^ o r^ '— 00 r o i ^ CN OO OO ^ oo r O ' — r o ^ in^'«3"'<i* i n ' ^ m r o

00 D̂ D̂ 00 D̂ CN rO 00 T—^T—CN " i r o r o c N

ooor^m CNi^oin i n - o i ^ o CN»—CNjco inco^ro »iro-^ro

' — r o ^ o

CN 3̂ in f—' 3̂ 3̂ D̂ ^f D̂ ^^ ^3 f~) CNi r - c N rocNjrocNj C N ' t

ro c

co m m

0 o.

0

o u

4..«

o

c 0 +J c o u Ol

0 u c ro

Sic tjO) 0 ott

.> 5

u •

0 u c ro E o 0

'.p > e u

U <

tf)

P ro

0 u c ro E Ö

CL

o O

E ro

9 So K

Ml

= -s ^ c

919

T) (D

J3 ro 4-' t/) Q.

0

0

E ro

o a

T l 91 tn (0

CQ O) c

'c ro

E ro

0 0 >

é 0

tn

ro c

111

U

b

Q LO

O

o

Ö

u

Elu SB2 to

o o o r̂ ô Ul <3 CO o o o

00 o oo o ^ o m roro o o 00

. CO o o I CN ^ in

T— 00 ^ 00 00 o m ^ ^ Ul r- CN ro CN

o o o o o •̂ o o o o T- o o »-

r - CO ^ CVJ CO « * 0 0 ^ ^ i n • * CN ^ ro

m -o m m o

o o <- ro ro ^

O 00 •iT •« ro »- ro CN

p̂ ro o 00 ^ o '— ro ^ * i ro CM ro ro

o 00 00 CM I . 1 ^ CM «- ro

I ^ uo CM •<*• T - CM CN m r̂ CN m T— ro o

' CN »- m CM m o

.> s; tn

c 'S ?| o

u < Ü. Q.

91 U

m a n

tiv e

iv e

0 uc ro

« E 'c t! <u o g'iëo^ u < Q- il.

u O

<

Í -s

£ £ of tis ti

§1

8 1 - i-S"

920

reren) tore n r e c o o o o ro ĉ o o <3 o o o o o o ^ CM O O Ô I O O o o o o T- LO o o I o o o <3 ro o o I

ooo ooo

b o o

u

Q to

ro CM O O O r O O l O O O O

to O C M C O ^ O ^ L O

o o

Ö

ro

O

o.

ro CQ

0 1

E ro

u c a> >

.o o^ o^ LO ro 00 ro ro i oo o^ o » - r O L O T - CMCMrOCN l o ^ L O

ooooo ooooo^ I o o o OOOT- «-OOO OOO

s O O ^ O ^ O ^ * O O O r O ^ l O O O ^ O

»—roioro ^ C M r o r o | 0 " * L O

0 r 0 ' a • ^ 0 O ' O I ^ O iCM<3»- r O ' í L o r o LororO"* OJLOLO

ro ' í csj o ^ 00 ̂ 00 I ô ô r̂ o r O ' í C N rov-cNCM o c M r o

>o ô oo 00 CN ^ «— ^ I 1^ ro ^ T - r o ^ C M ^CMroro O ^ ' *

r o 00 r o

^^c^rooo î îo

ou 4-»

o

u c

O) u c

:l -.̂ a Ë 5 -I a § s •$.

u • o

ro

<

ro . 1 '

ro

to

U

E

XI (U

/r- (11 M-

S S

921

922 December 2008 - Human Factors

the distribution of effect sizes after other research artifacts have been removed. That is, credibility in- tervals use the estimated true correlation between constructs as their basis. Moreover, credibility in- tervals, along with the estimate for the percentage of variance attributable to statistical artifacts, can provide information on whether moderators may be operating (Whitener, 1990). As the percentage of variance attributable to artifacts increases, the more certain it is that additional moderators are not operating.

A brief discussion explaining the percentage of variance accounted for by artifacts is appropriate. The meta-analytic procedure that we used is based on the hypothesis that much of the variation in study results is often based on statistical and methodological artifacts, rather than true under- lying population relationships. These artifacts may distort study findings in many ways. For exam- ple, sampling error distorts study findings ran- domly, whereas the effect of measurement error is to systematically bias study results. Thus, this meta-analytic procedure corrects for the artifactual differences in study results. The amount of vari- ance accounted for by study artifacts is based on calculating the ratio of the variance attributable to artifacts to the total variance.

This index, as calculated by the Hunter-Schmidt Meta-Analysis Program 1.1 (Schmidt & Le, 2005), represents the amount of variance in the observed correlations (those gathered from the primary studies) attributed to artifacts. In other words, this number indicates the amount of variance that is accounted for by sampling error and unreliability. This index allows one to judge whether substan- tial variation attributable to moderators exists. A number greater than 100% is not uncommon and is an indication that the variation in effect sizes from the primary studies could have been attribut- able to second-order sampling error and not exist- ing moderators (Hunter & Schmidt, 2004).

If the percentage of variance accounted for by artifacts is calculated to be 100 or greater, whether the number is 150 or 300, the correct conclusion would be that all the total variance could be ac- counted for by artifacts (Hunter & Schmidt, 2004). The magnitude of the number does not reveal any- thing about the quality of the analysis. Rather, what is meaningful is if the number is greater than 100% or less than 100%. A number under 100% indicates that some of the variance left to account for may be attributable to unhypothesized moderators.

whereas a number over 100% indicates that the total variance could be accounted for by artifacts such as sampling error or measurement error.

There is no relationship between this number and the confidence in the estimated true correla- tion. Rather, this is another piece of information that is used in order to interpret the findings. More- over, when the percentage of variance accounted for is greater than 100%, the credibility interval be- comes a point estimate. The credibility interval is yet another indicator of the existence of modera- tors. When a credibility interval is a point estimate, it is unlikely that moderators exist based on the analysis.

In the following sections, the results derived from our investigations into the four primary areas of interest in this research are presented. In meta- analysis there is no criterion that illustrates the minimum number of effect sizes to include in an analysis. We do acknowledge and understand that conducting an analysis with a small number of effect sizes does increase the possibility of second- order sampling error (Arthur, Bennett, & Huffcutt, 2001; Hunter & Schmidt, 2004). In order to be com- plete and to fully illustrate the database used in this meta-analysis, we conducted all possible analyses with one or more effect sizes. However, we do em- phasize, as did Arthur, Bennett, Edens, and Bell (2003, see p. 238), that any meta-analyzed effect size based on fewer than five effect sizes should be interpreted with caution.

Does team training work? The overall results support our hypothesis that team training does work. For the combined (and averaged, when nec- essary) set of independent outcomes, team train- ing was shown to have a moderate, positive effect on team functioning (p = .34; 10% CV = .34; 90% CV - .34). This analysis was based on 52 effect sizes representing 1,563 teams and provided sup- port for Hypothesis 1.

Criterion description. The results of our analy- ses revealed team training to have a positive effect on each of the four outcomes under investigation (see Table 2). For the 12 available effect sizes for team-level cognitive outcomes, the estimated true score correlation (p) was .42. The 80% credibility interval (10% CV and 90% CV) ranged from .18 to .67. For affective outcomes, the analysis of 16 effect sizes (representing 465 teams) revealed an estimated true score correlation of .35. Next, team training appeared to work best for process out- comes (p = .44; 10% CV = .44; 90% CV = .44);

TEAM TRAINING EFFECTIVENESS 923

this analysis was based on 25 independent effect sizes representing 607 teams. However, given the overlapping confidence and credibility intervals, this assertion is merely suggestive and not defin- itive. Finally, team training was also shown to be effective for performance outcomes (p = .39; 10% CV = .27; 90% CV = .50). Overall, these findings provide support for Hypotheses 2a through 2d.

Training content. The content of training in- terventions was also investigated as a potential moderator variable (see Table 3). For this analysis, the content of study interventions was coded as primarily taskwork, teamwork, or both. Overall, the analyses provided support for Hypotheses 3 through 5 but not necessarily Hypothesis 6. That is, although each type of intervention was shown to be useful, there is insufficient evidence to con- clude that mixed content interventions are supe- rior to taskwork or teamwork interventions alone.

Moreover, there was very little difference among taskwork (p = .35), teamwork (p = .38), or mixed content (p = .40) interventions for the im- provement of perfonnance. However, for process outcomes, taskwork training (p = .28) did not work as well as teamwork (p = .44) or mixed content training (p = .56); caution should be exercised in the interpretation of the taskwork effect size, as it is based on three effect sizes. Similarly, teamwork- focused training appeared to result in enhanced affective outcomes in comparison with taskwork training (p = .41 and .11, respectively). We again caution the reader in the interpretation made on the estimated correlation for taskwork content on affective outcomes, as it was based on one point estimate.

Team membership stability. The results for the analyses on team membership stability provided partial support for Hypothesis 7 (see Table 4). Spe- cifically, team training appeared to work just as well (if not better) for intact teams. This was es- pecially the case for performance outcomes, for which p = .54 and .38 for intact and ad hoc teams, respectively. At the same time, the findings for process outcomes were highly similar across the two team membership configurations (p = .48 and .44 for intact and ad hoc teams, respectively).

Team size. For the analyses addressing the po- tential moderating effect of team size, we divided studies into three categories based on the average size of the teams under investigation: small, me- dium, and large. The most informative comparison here concerns perfonnance outcomes. Our results

suggest that team size impacts the benefit from team training in terms of improved performance (p = .39, .34, and .50, respectively, for small, me- dium, and large teams; see Table 5). These results are based upon having 12,15, and 13 effect sizes in each group, respectively. The findings from the other three outcomes with team size as a potential moderator are not as clear. For example, team train- ing appeared to influence cognitive outcomes to a larger extent in teams of medium size (p = .46). Conversely, small teams benefited the most in terms of improvements in affective (p = .39) and process (p - .59) outcomes. This diverse set of find- ings does not provide any appreciable level of sup- port for Hypothesis 8.

Post hoc analyses. We conducted some post- hoc meta-analyses on effect sizes based on domain and strategy (see Tables 6 and 7). The five primary domains that are represented in our sample are studies conducted in the aviation, military, med- ical, and business sectors and those conducted in laboratories with student samples. The training strategies that were represented in our database consisted of coordination training/crew resource management, cross-training, communication train- ing, critical thinking training, self-guided training, self-correction training, team adaptation and coor- dination training (TACT), stress training, tactical training, and team knowledge training. These re- sults should be interpreted with caution but none- theless can be used to paint a picture of not only team training in each of these domains but also where the research and quantifiable data on train- ing effectiveness are lacking.

Most of the primary studies could be catego- rized into two domains: studies conducted in the military and those conducted in the lab with stu- dent samples. These analyses were based on 39 and 37 effect sizes, representing 762 teams in the military domain and 1,660 student teams. Team training appeared more effective in military set- tings (p = .59; 10% CV = .59; 90% CV = .59) than in laboratory settings (p = .33; 10% CV = .22; 90% CV = .43).

The other three domains represented by the pri- mary studies in our analyses were from the med- ical, aviation, and business settings. These three domains did not render a significant number of pri- maiy studies that reported the necessary quantifi- able data to use in meta-analysis. Six independent effect sizes were utilized to conduct the analysis for medical and aviation studies, and five independent

text continues on page 926

! § 8

o o

U

• o o ^ —̂ . o o m o

I o» m 00 m in 00 m ro

o- m 00 • m 00 o •

Q LO

ro

O Q

O a

"S tf) ro

CQ O l c c ro

E ro

0 c 0 >

LLJ 0

ro

< u o

oa.

CQ

ro O m ro ro ro in co ro CNJ

ro ro r̂ ro ^^ ro in r̂ ^ ro

U

o

u CN '

CN o «- t ^ CN CN

m >o o

O CNJ O OO O >o - o > - r o 00 >o ^ —̂

ç 'ro

O Q

IÍ.ÍII CQ •

Í 8

II i

c a í

tf)

T h i

d o ÍD c n]

xp l

0) "O

%l

^ 2

924

U) 0

o Q.

Z) -o 0 (/) ro

CO O) c 'ç

I E ro

-z 0 c

£

^ ë

O CL

DQ

/a r.

jn te

d b y

a ct

s

o ^ u ^

u

90 "

> U ô̂ O

'as

CL

90 %

U

S? o ~̂

Ö

1^

2

S tr

a te

g y

ra O o o o

1 ^

00"

.4 6

o ro

ro

ro ro

1*

in g

ai n

M tr

L Ío

n /C

R C

o o rd

in a l

m o o m

CM

CM

ro ro

O

. ^

^~

CM

ro

n in

g C

ro ss

-t ra

i

, - TO

ro ro TO

ro CM

CM O

TO O

ro

TO CM

CM O

1

ro

ro

-oTO

CT

ra in

i ca

tio n t

C o m

m u n i

CM

O

o •o

ro

T—

ro

ro TO

CT c

ra in

i in

ki n g t

00 =

o o

o ro

ô ro

8

• o ro

o ro

O ro

ro

CM

CM TO

• o

, m

TO

o o

o

CM

ro

m o

00 =

o o

•o m

•o in

.0 0=

o o

U l o

in o

.0 0=

o o

o

1 ^ -o

888

•o U l

[o

ro

o CM U l

CM

"^

CT

Ul'ç C 'm

îd t

ra in

i C

tio n t n

C ri tic

a l th

S e lf-

g u id

i S

e lf-

co rr

e

vO

•o

T A

C T

o

o

,_ o

1

o

CM

TO U l

n in

g S

tr es

s tr

a i

îo

ro

,_ >O

O

CM

O CM

.0 0=

o o

TO

^_ TO

8

TO

TO

-O

TO

CM

in

' d

tr a il

a in

in g

w ie

d g e

T a ct

ic a l t

r T

ea m

k n o

b se

rv e d r ;

0

in te

i

lu a p

co n fi

0

n d

3

1 I 0

tio n ;

(0 0)

th e p

re -

r-

O _Q Ç

ro r

in te

r fo

r ea

ch

— i

in te

ity id

ib il

th e

( m

e a s

id o

f 1

H- O -o-Qto tj 01

co n

(f u lly

it e ri o n

c

_SJ

th e

i rv

e d c

o r

u ct

a n d

• m

e a n o

b se

i

II l t . .

•jS Qi IA (0

.a

Jt io

n w

a s

d is

tr ib

i

c

e a ci

1 w

h ic

h

o

c ie

n t:

!e u (/) c 0 ra a>

o f

co rr

S Si

c

.^ iÀ

te a

m d ic

to r

c o n s tr

S a

i c

I c

ro Si O u Q)

ti m

a te

d

S II o. L-

o b s e rv

t in

te rv

a

S

je n

1 0)

_c

d o

ft

(1)aa D

Ô

§

u ti

rib 1 d is

t ea

ch irv

al f

o r

re d ib

ili ty

in te

u 0)

'S T>

o w

e r

b o u r

II

Ö

re la

tio n

8

ro

ro

tic a

.!í!

ñ

rd f

o r

by s

' ic

c o u n te

ic e

Ï 'd

v a ri a r

tr u e

o b se

io n o

f t

ro

d a

rd d

ta n

-D

m a te

e st

i

II

â

u m

b e r o f

)p e r

b o u n

ite ri o n ); S

N o te

. N

= n D

II

0 di ct

o r

a n d c

r

o

e n ta

g e

p e l

a ct

s =

rt if

c O (J

% V

an A

c d is

tr ib

u tio

n ;

!e s

fr o m

s ii

tí

1c 1 X I

ne e

va ri a

0)

li lt

ro ro

(J

ed s

u te

rp re

t ca

n

d g (D C fD

s-

T h is

le d

E an

1 0 0 .

er g

re a te

r th

_o Ê C (D

IS r

e su

lte d

(D

1c D

§

ne e a

ria |e

o f

va

ro

re d

e ta

il fo

r a

m o

:io n

13 (Ü in

a $

A n a ly

tic R

M e ta

-

ï

to t t

re a d e r

ir e ct

t ac

t. W

e

t ; ro

e p

e rc

i /

s tu

d y

lic h

1 c Í 3 c

Ï a Si (A (2)

1 Q) 0) 01

3

in te

d

o (J

<u

ie s

co u ld

p ri m

a ry

s tu

d

925

926 December 2008 - Human Factors

effect sizes were found from studies conducted in business settings. The results indicate that team training conducted with business groups (p = .85) resulted in greater effect sizes than did studies conducted with medical teams (p = .23; 10% CV= .11; 90% CV = .35) and aviation teams (p = .33; 10% CV= .08; 90% CV= .58). Again, we caution the reader in interpreting these analyses.

Our last post hoc analysis was based on the training strategy used. The two most popular strat- egies represented in our database were coordina- tion training and cross-training. Thirty-three effect sizes were meta-analyzed in order to assess the impact that coordination training had on team out- comes. The results indicated that this type of train- ing had a moderate positive effect on outcomes (p = .47; 10% CV = .47; 90% CV = .47). Cross- training was represented in our database by 14 ef- fect sizes and 432 teams. These results suggest that cross-training does have a moderate positive effect on outcomes (p = .44; 10% CV = .26; 90% CV = .62). We direct the reader to Table 7, which in- cludes all the analyses based upon strategy. And again, we strongly caution the interpretation of these analyses and can state only that more re- search is needed based on strategy.

DISCUSSION

Meta-analysis is a valuable tool for navigating the research literature when conflicting findings are the norm rather than the exception. Most areas of research fall prey to this issue, and the team training literature is certainly not immune. A meta- analysis of this area is beneficial so that the find- ings across studies can be integrated and reveal some simpler patterns of relationships. These pre- liminary patterns can then provide a basis for theory development. In addition to this benefit, meta-analysis can be particularly advantageous for research on teams because of the small-sample studies that are typical of this area. The current in- vestigation therefore provides an overdue empir- ical accounting of the team training literature and, in turn, identifies important patterns of results.

What Works and Why?

The findings from the present study provide in- sight into the extent to which team training inter- ventions relate to team outcomes. That is, across a wide variety of settings, tasks, and team types, team training efforts were successful. Although

"a positive relationship between training and per- formance in team settings is and has been clear for some time" (Hollenbeck et al., 2004, p. 360), this research is, to the best of our knowledge, the first to comprehensively and quantitatively evaluate the relationships between these interventions and outcomes.

These findings suggest that team training inter- ventions are a viable approach for organizations to take in order to enhance team outcomes. They are useful for improving cognitive outcomes, affec- tive outcomes, teamwork processes, and perfor- mance outcomes. That is, team training accounted for approximately 12% to 19% of the variance in the examined outcomes. This percentage, though statistically correct, is an underestimate of the prac- tical and theoretical significance of the relation- ships between the variables investigated in the present meta-analysis (e.g.. Hunter & Schmidt, 1990). Thus, the utility of team development tech- niques is likely understated by the present find- ings. Further, the large number of studies in the database and the large overall observed effect size make these findings robust to disconfirmation, ex- cluding those analyses that included fewer than five effect sizes.

Across all outcomes, team training interven- tions were more effective for team processes than for the other outcome types. As a primary expla- nation, in dynamic input-throughput-output mod- els it is generally understood that team-member and team-level processes such as communication and coordination are determinants of team perfor- mance and effectiveness. Although the training interventions reviewed in the present study did appear to have a nearly equal impact on outcomes as processes, it stands to reason that improving processes will also positively influence perfor- mance outcomes.

For example, training teams to communicate, coordinate, and make decisions should ultimately increase the quantity of products produced. Finally, it is also recognized that team performance is shaped by additional environmental and/or orga- nizational characteristics and contingencies that are out of the control of team members; team mem- bers actively execute team processes but are often at the mercy of a larger organizational system to produce more distal performance outputs. Of course, team processes may also be influenced by the larger organizational system, but probably not to the same degree as team performance outcomes.

TEAM TRAINING EFFECTIVENESS 927

In contrast to our hypothesis, team training in- terventions with a mixed training content (i.e., teamwork and taskwork) were not superior to those with a taskwork or teamwork focus in isolation. Team training was deemed useful for improving team performance outcomes, regardless of the type of training content. This finding could be inter- preted such that team training is helpful but that combining teamwork and taskwork content does not necessarily make the training incrementally more beneficial. The problem could potentially be that too much information was included in the training given to teams. As such, the teams ex- posed to mixed-content training may have been overwhelmed and could not successfully grasp the intended concepts and information that were pre- sented to them.

An additional explanation is available, but it merits more research. Specifically, Salas, Burke, and Cannon-Bowers (2002) identified the impor- tance of both taskwork and teamwork training. They argued that it is best to develop a team's task- work knowledge prior to the team's acquisition of teamwork skills. Alternatively, the results from a recent study by Ellis et al. (2005) suggested that enhancing teamwork skills prior to taskwork knowledge leads to greater performance improve- ments. Although these explanations do not com- bine to provide a clear explanation for the lack of support for Hypothesis 6, they do serve as a call to researchers to further examine the impact of tem- poral factors when designing training that contains both teamwork and taskwork components.

Our research also showed that membership stability moderated the relationship between team training and team outcomes, such that intact teams that underwent training improved the most on process and performance outcomes. Our hypoth- esis that intact teams would benefit most from team training interventions was only partially sup- ported, however. There was clear support, given the nonoverlapping confidence intervals, for our assertion that intact teams would benefit the most from training interventions when the focus was on performance outcomes. This finding was consis- tent with our belief that those teams that had an op- portunity to work with one another would benefit the most from team training.

Intact teams should have already overcome some of the maturational challenges that newly formed teams would not have had the opportunity to navigate. Ad hoc teams are likely forced to at-

tend to many more factors during their training session than intact teams, and thus intact teams should benefit the most from team training. An additional explanation could be that intact teams may have seen the benefits of training more so than ad hoc teams and, thus, were more likely to attend to the important aspects of training, whether taskwork or teamwork focused in nature. In other words, intact teams may have perceived the train- ing as being more instrumental to their team's per- formance improvement and, ultimately, to the attainment of valued outcomes in the workplace.

The moderator analysis on team size helped to further clarify when team training is most helpful for collectives. Team performance, as expected, improved the most for large teams. Moreover, team processes improved the most for small teams. These findings partially supported our hypotheses. Intuitively, if the members of a small team are behaving in a way that impedes their performance, once they are trained how to properly communi- cate and coordinate with one another, they may have an easier time maintaining those process en- hancements.

A surprising finding was the extremely low ef- fect size (p = .08) for affective outcomes in large ' teams. However, it is important to interpret this finding with caution because there were only three effect sizes to combine for this calculation. Addi- tionally, it speaks to the need for more research on these phenomena in order to appropriately deter- mine the effect of team training on affective out- comes for large teams. Moreover, as there were no effect sizes available to meta-analyze for cognitive outcomes for large teams, more research is neces- sary in this area as well.

Implications for Practitioners

We believe that much progress has been made with respect to team training, both in research and in practice. The meta-analytic findings are encour- aging in that they show team training to be ef- fective. However, these findings highlight some practical issues for organizational change agents to consider when designing and implementing training. First, practitioners should relish the fact that team training is effecfive. The implications of this are critical. To know that team training can ex- plain 12% to 19%ofthe varianceofateam'sper- formance (i.e., better-performing teams will exhibit more effective team behaviors than their counter- parts) can mean reducing medical errors (in health

928 December 2008 - Human Factors

care), saving an aircraft (in aviation), increasing the bottom line (in business), or saving lives (in the military).

Second, given the heightened interest in team training in health care, change agents in health care institutions should utilize this information to bol- ster their argument for implementing such training.

Third, in deciding on the specific type and fo- cus of team training, practitioners need to be cog- nizant of the ultimate outcome they are interested in, as our findings indicated differential effects. Sometimes these effects were rather small, but at other times they were rather large, depending on the specific outcome targeted (i.e., cognition, be- havior/process, affect, performance).

In conclusion, the data we present, as we have stated several times, are encouraging. Of most im- portance, however, is what individuals do to im- prove the training and development initiatives that they manage. We hope that practitioners and re- searchers alike find useful and practical advice to follow based on the findings presented.

Study Limitations

In general, a large number of methodological tools are available to researchers who attempt to discover the meaning of organizational phenom- ena (McCall & Bobko, 1990); meta-analysis is just one such tool. The validity of inferences con- cerning any relationship examined through meta- analysis is a function of several factors, including (a) the representativeness of the sample of primary studies considered by the meta-analyst, (b) the de- gree of validity of each of the primary studies, and (c) the number of primary studies (Bobko & Stone- Romero, 1998).

Despite the strengths and contributions of the current study, several potential limitations should be noted. First, although extensive efforts were made to be inclusive and a large number of pri- mary studies were included, definitive conclusions regarding the validity of each included study can- not be made. To the extent that the primary stud- ies lacked proper control mechanisms, either via design features or statistically, it is not possible to unequivocally conclude that team training in- terventions cause improvements in team func- tioning.

Second, as Rosenthal and DiMatteo (2001) sug- gested, every meta-analysis has some inherent bias attributable to the selection of inclusion/exclusion

criteria and the methods chosen to review the literature. To the extent that the studies in the data- base come from populations that are more homog- enous and nonrepresentative than is typical, the more likely it is that the external validity of these findings may be threatened. For example, the over- whelming majority of the studies included herein were conducted using college students acting as action and performing teams in simulated en- vironments (e.g., aviation, military). Clearly, this focus has been influenced by years of generous funding by the Department of Defense and var- ious military institutions (e.g., U.S. Army, U.S. Navy). Although these studies have obvious merit in advancing the science of this area, they often do not capture the unique dynamics and context in which other teams perform.

Finally, in our attempt to drill down and gauge relationships at more finite levels, we unfortunately left many of our subgroups lacking a robust num- ber of effect sizes to analyze. This was both a nec- essary evil and a cogent assessment of the current state of research on teams. It was necessary because we deemed it important to provide more prescrip- tive findings than simply saying, "team training works to improve team outcomes." The goal of ad- dressing when and where it works best is an im- portant undertaking. This research takes an initial step in that direction, but it is worth repeating that more and better evaluations of the impact of team training in organizations are needed.

In addition, we ask researchers and practition- ers, in any domain, to consider the data that they present when reporting their findings. Whereas p values and percentage improvement provide an indication of the impact of team training, they leave the reader with no true measure of the mag- nitude with which team training improved desired outcomes. We call for researchers (and editors of journals) to report (and to demand) informative statistics, such as the group means and standard deviations, rather than solely p values or percent- ages. Researchers need to examine diverse sam- ples and settings and to elicit more published data, which would allow for additional meta-analyses in the future.

Recommendations for Future Research

The results of the present meta-analysis sug- gest several avenues for future research. First, most of the articles contained in the database

TEAM TRAINING EFFECTIVENESS 929

focused on teams working jointly on a task in the same location. However, many organizations have operations in multiple countries. In multinational organizations, globally distributed teams are often responsible for making and implementing impor- tant decisions. Research is needed that examines whether team training improves team outcomes in distributed teams. Distributed teams have to resolve the same problems that other teams en- counter; however, these challenges are often more pronounced when team members are distributed across temporal and geographic boundaries (Stagl, Salas, Rosen, et al., 2007). This type of research would provide useful information to managers and human resource professionals regarding the in- creasing use of globally distributed teams.

More than a decade ago. Stout, Salas, and Fowlkes (1997) noted, "there are few systematic evaluations of the effectiveness of particular team training efforts" (p. 169). Unfortunately, the large amount of published literature that had to be ex- cluded from the meta-analytic database bears evi- dence that this critique still merits attention. That is, although the study of work teams has increased in the past 2 decades (Cohen & Bailey, 1997; Niel- sen, Sundstrom, & Halfhill, 2005; Sundstrom, Mclntyre, Halfhill, & Richards, 2000), relatively few researchers properly report their findings. In short, there is a plethora of team training but little systematic evaluation and dissemination of results. Practitioners who facilitate team training inter- ventions and researchers who study teams and team training should make increased efforts to col- laborate and empirically evaluate and publish their findings in peer-reviewed scientific outlets.

Another area meriting further attention involves the extent to which the team leader is incorporated as a critical component of team training design. Research has demonstrated the beneficial influence of team leader support, by way of leader briefings, for posttraining recall and the performance of train- ees (e.g., Mathieu et al., 2000; Smith-Jentsch, Salas, & Brannick, 2001 ). Following this further, a recent meta-analysis found that person-focused and task- focused behaviors by the leader were related to team effectiveness and team productivity (Burke, Stagl, Klein, et al., 2006). However, it was an un- fortunate fact that very few studies in the database included any discussion of the role of the leader in team training. This is in contrast to the vast liter- ature on team leadership in general. Given the identification of a sufficient number of empirical

articles on the topic, a moderator analysis inves- tigating the presence or absence of team leader support during training may be an informative avenue of future research.

Also highlighted within the current findings was the potentially important role of time in inter- vention delivery. This area has been recently high- lighted as one that needs more attention in the team's arena by several researchers. For example, with regard to developmental interventions. Hack- man and Wageman (2005) noted the importance of timing. Specifically, they discussed how team coaching interventions should be made: "at times when the team is ready for them and able to deal with them" (p. 283). They suggested that coaching be done at the beginning of the team's task cycle for effort-related interventions, near the midpoint for strategy-related interventions, and at the end of a task cycle for interventions that address knowl- edge and skill (Hackman & Wageman, 2005). We join with those authors and others in arguing for research that examines issues related to time and developmental interventions - in our specific case, team training interventions.

Last, our meta-analysis did not consider the impact of team composition on the effectiveness of team training. Most of the primary studies in the database did not consider team composition. Thus, considering the dearth of reported data on team composition variables in team training stud- ies, we make a call to researchers. Specifically, al- though much research has been done to evaluate the moderating effects of individual characteristics (e.g., goal orientation, motivation to learn, consci- entiousness) on individual learning, how does a team's composition impact team learning?

Concluding Remarks

Team training works ! The findings from this re- search are encouraging. In general, our results also suggest that training content, team membership stability, and team size moderate the effectiveness of team training interventions. This study was the first to thoroughly and quantitatively summarize the team training literature. We hope our findings provide valuable information to those stakehold- ers charged with making informed decisions re- garding the planning, design, development, and delivery of team training interventions. We also hope it inspires more and richer team training eval- uations from a variety of sectors.

930 December 2008 - Human Factors

ACKNOWLEDGMENTS

This work was supported by funding from the U.S. Army Research Institute for the Behavioral and Social Sciences (Contract W74V8H-04-C- 0025). The views, opinions, and/or findings con- tained in this paper are those of the authors and should not be construed as an official position, pol- icy, or decision of the Department of the Army or the University of Central Florida.

We would like to thank Huy Le, John Mathieu, and Florian Jentsch for their insightful guidance during the course of this project. We would also like to acknowledge Shannon Scielzo and Samuel Wooten for their assistance during the early cod- ing portions of the project.

REFERENCES

References marked with an asterisk indicate studies inciuded in the meta-anaiysis.

*Adelman, L., Christian, M., Gualtieri, J., & Bresnick, T. A. ( 1998). Ex- amining the effects of communication training and team compo- sition on the decision making of patriot air defense teams. IEEE Transactions on Sy.stems. Man, and Cybernetics - Pan A: System.? and Humans, 28, 729-741.

»Alexander, L. T., Kepner, C. H., &Tregoe, B. B. (1962). The effective- ness of knowledge of results in a military system-training program. Journal of Applied P.'iychoiogy, 46, 202-211.

Arthur, W., Jr., Bennett, W., Jr., Edens, P. S., & Bell, S. T. (2003). Effec- tiveness of training in organizations: A meta-analysis of design and evaluation features. Journal of Applied Psychology. 88, 234-245.

Arthur, W., Jr., Bennett, W., Jr., & Huffcutt, A. I. (2001). Conducting mela-analysis using SAS. Mahwah, NJ: Erlhaum.

Baker, D. P., Salas, E., Barach, P., Battles, J., & King, H. (2007). The re- lation between teamwork and patient safety. In P. Carayon (Ed.), Handbook of human factors and ergonomics in health care and patient safety (pp. 259-271). Mahwah, NJ: Erlbaum.

Banker, R. D., Held, J. M., Schroeder, R. G., & Sinha, K. K. (1996). Impact of work teams on manufacturing performance: A longitudi- nal field iludy. Academy of Management Journal, 39, 867-890.

»Banks, J. H., Jr., Hardy, G. D., Scott, T. D., Kress, G., & Word, L. E. (1977). Realtrain validation for rifle squads: Mission accomplish- ment (Research Rep. 1192, Army Project No. 2Q763743A773 and 2Q763743A775, Tactical Team Performance). Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.

»Blickensderfer, E., Cannon-Bowers, J. A., & Salas, E. (1997, April). Training teams to self-correct: An empirical investigation. Paper pre- sented at the 12th Annual Meeting of the Society for Industrial and Organizational Psychology, St. Louis, MO.

Bobko, P., & Stone-Romero, E. L. ( 1998). Meta-analysis may be another u.seful tool, but it is not a panacea. Research in Personnel and Human Resources Management, 16, 359-397.

Boehm-Davis, D. A., Holt, R. W., & Seamster, T. L. (2001 ). Airline re- source management programs. In E. Salas, C. A. Bowers, & E. Edens (Eds.), Improving teamwork in organizations (pp. 191-215). Mahwah, NJ: Erlbaum.

»Brannick, M. T., Prince, C , & Salas, E. (2005). Can PC-based systems enhance teamwork in the cockpit? International Journal of Aviation Psychology, 15, 173-187.

*Brown, T. C. (2003). The effect of verbal self-guidance training on col- lective efficacy and team performance. Personnel Psychology, 56, 935-964.

Burke, C. S., Stagl, K. C , Klein, C , Goodwin, G. F., Salas, E., & Halpin, S.M. (2006). What type of leadership behaviors are functional in teams? A meta-analysis. Leadership Quarterly, 17, 288-307.

Burke, C. S., Stagl, K. C , Salas, E., Pierce, L., & Kendall, D. (2006). Understanding team adaptation: A conceptual analysis and model. Journal of Applied Psychology, 91, 1189-1207.

»Buzaglo, G., & Wheelan, S. A. (1999). Facilitating work team effec- tiveness: Case .studies from Central America. Small Group Research, 30, 108-129.

Campbell, J. P, & Kuncel, N. R. (2001 ). Individual and team training. In N. Anderson, D. S. Ones, H. K. Sinangil, & C. Viswesvaran (Eds.), Handbook of work and organizational psychology (pp. 278-312). London: Blackwell.

*Cannon-Bowers, J. A., Salas, E., Blickensderfer, E., & Bowers, C. A. ( 1998). The impact of cross-training and workload on team func- tioning: A replication and extension of initial findings. Human Fac- tora, 40, 92-101.

Cannon-Bowers, J. A., Tannenbaum, S. I., Salas, E., & Volpe, C. E. (1995). Defining team competencies and establishing team train- ing requirements. In R. Guzzo, E. Salas, & associates (Eds.), Team effectiveness and decision making in organizations (pp. 333-380). San Francisco: Jossey-Bass.

*Cobb, M. G. (2000). The impact of environmental complexity and team training on team processes and performance in multiteam environ- ments. Dissertation Abstracts International. 60, 4281-4411.

Cohen, S. G., & Bailey, D. (1997). What makes teams work: Group effectiveness research from the shop floor to the executive suite? Journal of Management, 23, 239-290.

Cohen, S. G., & Ledford, G. B. (1994). The effectiveness of self- managing teams: A quasi-experiment. Human Relation.?, 47, 13-43.

»Cooke, N. J., Cannon-Bowers, J. A., Kiekel, P. A., Rivera, K., Stout, R. J., & Salas, E. (2000). Improving teams' interpersonal knowl- edge through cross-training. In Proceedings of the XlVth Triennial Congress of the International Ergonomics Association and 44th Annual Meeting of the Human Factors and Ergonomics Society (pp. 2.390-2.393). Santa Monica, CA: Human Factors and Ergonomics Society.

•Cooke, N. J., Kiekel, P A., & Helm, E. E. (2001). Measuring team knowledge during skill acquisition of a complex task. International Journal of Cognitive Ergonomics, 5, 297-315.

»Cooke, N. J., Kiekel, P A., Salas, E., Stout, R., Bowers, C , & Cannon- Bowers, J. (2003). Measuring team knowledge: A window to the cognitive underpinnings of team performance. Group Dynamics: Theory. Research, and Practice. 7, 179-199.

Curral, L. A., Forrester, R. H., Dawson, J. F , & West, M. A. (2001 ). It's what you do and the way that you do it: Team task, team size, and innovation-related group processes. European Journal of Work and Organizational Psychology, 10, 187-204.

Davies, J. M. (2(X)1). Medical applications of crew resource manage- ment. In E. Salas, C. A. Bowers, & E. Edens (Eds.), Improving team- work in organizations (pp. 265-281). Mahwah, NJ: Eribaum.

De Dreu, C. K. W. (2007). Cooperative outcome interdependence, task refiexivity, and team effectiveness: A motivated information pro- cessing perspective. Journal of Applied Psychology, 92, 628-638.

Denson, R. W. (1981). Team training: Literature review and annotated bibliography (Final Rep. AFHRL-TR-80-40). Brooks Air Force Base, TX: HQ Air Force Human Resources Laboratory.

Devine, D. J., Clayton, L. D., Philips, J. L., Dunford, B. B., & Meiner, S. B. ( 1999). Teams in organizations: Prevalence, characteristics, and effectiveness. Small Group Research, 30, 678-711.

Driskell, J. E., Salas, E., & Johnston, J. (1995, April). Is stress training generalizable to novel settings? Paper presented at the 10th Annual Conference of the Society for Industrial and Organizational Psychol- ogy, Orlando, FL.

»Duncan, P C , Rouse, W. B., Johnston, J. H., Cannon-Bowers, J. A., Salas, E., & Bums, J. J. (1996). Training teams working in complex systems: A mental model-based approach. In W. B. Rouse (Ed.), Human/technology interaction in complex systems (Vol. 8, pp. 173-231 ). Greenwich, CT: JAI Press.

Dyer, J. L. (1984). Team research and team training: A state-of-the-an review. In F. A. Muckler (Ed.), Humanfactors review (pp. 285-323). Santa Monica, CA: Human Factors and Ergonomics Society.

»Ellis, A. P J., Bell, B. S., Ployhart, R. E., Hollenbeck, J. R., & Ilgen, D. R. (2005). An evaluation of generic teamwork skills training with action teams: Effects on cognitive and skill-based outcomes. Person- nel Psychology, 58, 641-672.

»Entin, E. B., Entin, E. E., MacMillan, J., & Serfaty, D. (1993). Struc- turing and training high-reliability teams (AD No. ADA3O2385). Buriington, MA: Alphatech.

TEAM TRAINING EFFECTIVENESS 931

*Entin, E. E.. & Seifaty, D. ( 1999). Adaptive team coordination. Human Factors, 41, 312-325.

»Entin, E. E., Serfaty, D., & Decken, J. C. ( 1994). Team adaptation and coordination training (TR-648-1 ). Burlington, MA: Alphatech.

Field, A. P. (2001 ). Meta-analysis of correlation coefficients: A Monte Carlo comparison of fixed- and random-effects methods. Psycho- logical Methods, 6, 161-180.

Field, A. P. (2005). Is the meta-analysis of correlation coefficients accu- rate when population correlations vary? Psychological Methods, 10, 444-467.

Flin, R. H. ( 1995). Crew resource management for teams in the offshore oil industry. Journal of European Industrial Training, 19, 23-27.

Flin, R. H., & O'Connor, P. (2001). Applying crew resource manage- ment on offshore oil platforms. In E. Salas, C. A. Bowers, & E. Edens (Eds.), Improving teamwork in organizations (pp. 217-233). Mahwah, NJ: Eribaum.

Foushee, H. C. ( 1984). Dyads and triads at 35,000 feet: Factors affecting group process and aircrew performance. American Psychologist, 39, 885-893.

»Freeman, J. T., Cohen, M. S., & Thompson, B. (1998). Effects of deci- sion support technology and training on tactical decision making. In Proceedings of the 1998 Command and Control Research & Technology Symposium [CD-ROM]. McLean, VA: Command and Control Research Program.

Gagné R. M. (1962). Military training and principles of learning. American Psychologist, 17, 83-91.

•Ganster, D. C , Williams, S., & Poppler, P ( 1991 ). Does training in prob- lem solving improve the quality of group decisions? Journal of Applied Psychology, 76, 4 7 9 ^ 8 3 .

George, G. C , Hannibal, M. J., & Hirsch, W. L. (2004). Building the brand through people. WorldatWork Journal, 13(1), 3 9 ^ 5 .

Glass, B. (1955). News and notes. Science, 21, 583-596. Glass, G. V. ( 1976). Primary, secondary, and meta-analysis of research.

Educational Researcher, 5, 3-8. Glass, G. V, McGaw, B., & Smith, M. L. ( 1981 ). Meta-analysis in social

research. Beverly Hills, CA: Sage. *Glickman, A. S., Zimmer, S., Montero, R. C , Guerette, P. J., Campbell,

W. J., Morgan, B. B., et al. (1987). The evolution of teamwork skills: An empirical assessment with implications for training (Tech. Rep. Contract No. N00014-86-K-0732). Ft. Belvoir, VA: Defense Tech- nical Information Center.

Goldstein, I. L., & Ford, J. K. (2002). Training in organizations: Needs assessment, development, and evaluation (4th ed.). Belmont, CA: Wadsworth.

Gooding, R. Z., & Wagner, J. A., III. ( 1985). A meta-analytic review of the relationship between size and performance: The productivity and efficiency of organizations and their subunits. Administrative Science Quarterly, 30, 462-4?,l.

*Green, L. R. ( 1994). The effectiveness of tactical adaptation and coor- dination training on team performance in tactical scenarios. Un- published master's thesis. Naval Postgraduate School, Monterey, California.

Gully, S. M., Devine, D. J., & Whitney, D. J. (1995). A meta-analysis of cohesion and performance: Effectsof levels of analysis and task interdependence. Small Group Research, 26, 497-520.

Guzzo, R. A., & Shea, G. P. ( 1992). Group performance and intergroup relations in organizations. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (Vo\. 3, pp. 269-313). Palo Alto, CA: Consulting Psychologists Press.

Hackman, J. R. (1983). A normative model of work team effectiveness (Tech. Rep. No. 2). New Haven, CT: Yale University.

Hackman, J. R. (1987). The design of work teams. In J. Lorsch (Ed.), Handbook of organizational behavior (pp. 215-342). New York: Prentice Hall.

Hackman, J. R. (Ed.). ( 1990). Groups that work (and those that don 't¡: Creating conditions for effective teamwork. San Francisco: Jossey- Bass.

Hackman, J. R. (2002, August). New rules for team building. Optimize, pp. 50-62.

Hackman, J. R., & Vidmar, N. (1970). Effects of size and task type on group performance and member reactions. Sociometry, 33, 37-54.

Hackman, J. R., & Wageman, R. (2005). A theory of team coaching. Academy of Management Review, 30, 269-287.

*Heffner, T. S. (1997). The influence of team training methods on simu- lator performance. IEEE Transactions on Systems, Man, and Cyber- netics, 27, 4]65-AnO.

Helmreich, R. L., & Foushee, H. C. (1993). Why crew resource man- agement? Empirical and theoretical bases of human factors train- ing in aviation. In E. L. Weiner, B. G. Kanki, & R. L. Helmreich (Eds.), Cockpit resource management (pp. 3-45). San Diego, CA: Academic Press.

*Henry, L. J. ( 1997). The effect of group process training on team effec- tiveness. Unpublished doctoral dissertation, Texas Women's University, College of Health Sciences, Dentón, TX.

Hollenbeck, J. R., DeRue, D. S., & Guzzo, R. (2004). Bridging the gap between I/O research and HR practice: Improving team composi- tion, team training, and team task design. Huimm Resource Manage- ment, 43, 353-366.

Hunter, J. E., & Schmidt, F. L. ( 1990). Methods of meta-analysis: Cor- recting error and bias in research finditxgs. Beverly Hills, CA: Sage.

Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Cor- recting error and bias in research findings (2nd ed.). Newbury Park: Sage.

Hunter, J. E., Schmidt, F L., & Jackson, G. B. (1982). Meta-analy.sis: Cumulating research findings across studies. Beverly Hills, CA: Sage.

Hunter, J. E., Schmidt, F. L., & Le, H. (2006). Implications of direct and indirect range restriction for meta-analysis method and findings. Journal of Applied Psychology, 91, 594-612.

*lkomi, P. A., Boehm-Davis, D. A., Holt, R. W., & Incalcaterra, K. A. (1999). Jump seat observations of advance crew resource manage- ment (ACRM) effectiveness. In R. S. Jensen, B. Cox, J. D. Callister, & R. Lavis (Eds.), Proceedings of the lOth International Symposium on Aviation Psychology (pp. 292-297). Columbus: Ohio State University.

Ilgen, D. R. (1999). Teams embedded in organizations: Some implica- tions. American Psychologist, 54, 129-139.

Indik, B. P. (1965). Operational size and member participation: Some empirical tests of altemative explanations. Human Relations, 18, 339-350.

Judge, T. A., Cable, D. M., Colbett, A. E., & Rynes, S. L. (2007). What causes a management article to be cited—Article, author, or journal? Academy of Management Journal, 50, 491-506.

Karau, S. J., & Williams, K. D. ( 1993). Social loafing: A meta-analytic review and theoretical integration. Journal of Personality and Social Psychology 65, 681-706.

Kidwell, R. E., & Bennett, N. ( 1993). Employee propensity to withhold effort: A conceptual model to intersect three avenues of research. Academy of Management Review, 18, 429-456.

Klein, C , DiazGranados, D., Salas, E., Le, H., Burke, C. S., & Lyons, R. (in press). The effect of team building interventions on team out- comes: An update and extension. Small Group Research.

Klein, C , Stagl, K. C , Salas, E., Parker, C , & Van Eynde, D. F (2007). Returning to flight: Simulation-based training for the U.S. National Aeronautics and Space Administration's Space Shuttle Mission ManagemenlTeam. International Journal of Training and Develop- ment, II, 132-138.

Kozlowski, S. W. J., & Bell, B. S. (2003). Work groups and teams in orga- nizations. In W. C. Borman, D. R. Ilgen, & R. J. Klimoski (Eds.), Comprehensive handbook of psychology: Industrial and organiza- tional psychology (Vol. 12, pp. 333-375). New York: Wiley.

Kozlowski, S. W. J., & Klein, J. K. (2000). A multi-level approach to theory and research in organizations: Contextual, temporal and emer- gent processes. In J. K. Klein & S. W. J. Kozlowski (Eds.), Multi- level theory research, and methods in organization,'/: Foundations, extensions, and new directions (pp. 3-90). San Francisco: Jossey- Bass.

Kraiger, K. (1985). On learning from the past: A meta-analytic fable. Personnel Psychology, 38, 799-801.

Kraiger, K., Ford, J. K., & Salas, E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology, 78, 311-328.

»Lampton, D. R., McDonald, D. P, Rodriguez, M. E., Cotton, J. E., Morris, C. S., Parsons, J., et al. (2001). Instructional .arategiesfor training teams in virtual environments (Tech. Rep. No. 1110). Alexandria, VA: U.S. Army Research Institute for the Behavioral Sciences.

»Lassiter, D. L., Vaughn, J. S., Smaltz, V. E., Morgan, B. B., Jr., & Salas, E. ( 1990). A comparison of two types of training interven- tions on team communication performance. In Proceedings of the Human Factors Society 34th Annual Meeting (pp. 1372-1376). Santa Monica, CA: Human Factors and Ergonomics Society.

932 December 2008 - Human Factors

Latané. B. ( 1981 )• The psychology of social impact. American Psychol- ogist, 36, 343-356.

*Le Blanc. P. M., Hox, J. J.. Schaufeli. W. B.. Taris, T W., & Peeters, M. C. W. (2007). Take care! The evaluation of a team-based burnout intervention program for oncology care providers. Journal of Ap- plied Psychology, 92, 213-227.

*Leedom, D. K., & Simon. R. (1995). Improving team coordination: Acase for behavior-based training. A/i7/raryPjycAo/og3'. 7,109-122.

LePine, J. A., Piccolo, R. F., Jackson. C. L., Mathieu, J. E., & Saul, J. R. (2008). A meta-analysis of teamwork processes: Tests of a multi- dimensional model and relationships with team effectiveness criteria. Personnel Psychology, 61, 273-307.

Lipsey, M. W., & Wilson, D. B. (2001 ). Practical meta-analysis. Thou- sand Oaks, CA: Sage.

»MacMillian, J., Entin, E. E., Entin, E. B., & Serfaty, D. (1994). Struc- turing and training high reliability teams (Year 2 Interim Tech. Rep. TR-632). Fort Rucker, AL: U.S. Army Research Institute/Aviation Research and Development Activity.

Magjuka, R. J., & Baldwin, T. T ( 1991 ). Team-based employee involve- ment programs: Effects of design and administration. Personnel Psy- chology, 44, 793-812.

Marks, M. A. (2(X)6). The science of team effectiveness. Psychological Science in the Public Interest, 7(3), i.

Marks, M. A., Mathieu, J. E., & Zaccaro, S. J. (2001). A temporally based framework and taxonomy of team processes. Academy of Manage- ment Review, 26, 356-376.

•Marks, M. A., Sabella, M. J., Burke, C. S., & Zaccaro, S. J. (2002). The impact of cross-training on team effectiveness. Journal of Applied Psychology, 87, 3-13.

•Marks, M. A., Zaccaro, S. J., & Mathieu, J. E. (2000). Performance im- plications of leader briefings and team-interaction training for team adaptation to novel environments. Journal of Applied Psychology, 85,971-986.

»Mathieu, J. E., Heffner, T. S., Goodwin, G. F., Salas, E., & Cannon- Bowers, J. A. (2000). The influence of shared mental models on team process and performance. Journal of Applied Psychology, 85, 273-283.

Mathieu, J. E., & Leonard, R. L. (1987). Applying utility concepts to a training program in supervisory skills: A time-based approach. Academy of Management Journal, 30, 316-335.

McCall, M., & Bobko, P. (1990). Research methods and discovery in industrial/organizational psychology. In M. Dunnette & L. Hough (Eds.), Handbook of industrial and organizational psychology (pp. 381-418). Palo Alto, CA: Consulting Psychologists Press.

McGrath, J. E. (1984). Groups: Interaction and performance. Engle- wood Cliffs, NJ: Prentice Hall.

Mclntyre, R. M., & Salas, E. ( 1995). Measuring and managing for team performance: Emerging principles from complex environments. In R. Guzzo & E. Salas (Eds.), Team effectiveness and decision mak- ing in organizations (pp. 149-203). San Francisco: Jossey-Bass.

•Minionis, D. P. ( 1995). Enhancing team performance in adverse con- ditions: The role of shared team mental models and team training on an interdependent task (Doctoral dissertation, George Mason University, \995). Dissertation Abstracts International, 56, 1139.

Mohammed, S., & Ringseis, E. (2001). Cognitive diversity and con- sensus in group decision making: The role of inputs, processes, and outcomes. Organizational Behavior and Human Decision Processes, «5,310-335.

»Morey, J. C , Simon, R., Jay, G. D., Wears, R. L., Salisbury, M., Dukes, K. A., et al. (2002). Error reduction and performance improvement in the emergency department through formal teamwork training: Evaluation results of the MedTeams project (quality of care). Health Services Research, 37, 1553-1582.

Mullen, B. (1987). Self-attention theory. In B. Mullen& G. R. Goethals (Eds.), Theories of group behavior (pp. 125-146). New York: Springer-Veriag.

Mullen, B. (1989). Advanced BASIC meta-analysis. Hillsdale, NJ: Erlbaum.

Mullen, B., Brown, R. J., & Smith, C. (1992). Ingroup bias as a function of salience, relevance, and status: An integration. European Journal of Social Psychology, 22, 103-122.

Mullen, B., & Rosenthal, R. (1985). BASIC meta-analysis. Hillsdale, NJ: Brlbaum.

Mullen, B., Salas, E., & Driskell, J. E. (1989). Salience, motivation, and artifact as contributions to the relation between participation rate and leadership. Journal of Experimental Social Psychology, 25, 545-559.

Mullen, B., Symons, C , Hu, L., &. Salas, E. (1989). Group size, leader- ship behavior, and subordinate satisfaction. Journal of General Psy- chology, 116, 155-170.

•Naylor, J. C , & Briggs, G. E. ( 1965). Team-training effectiveness under various conditions. Journal of Applied Psychology, 49, 223-229.

Nielsen, J. M., Sundstrom, E. D., & Halfhill, T. R. (2005). Group dynam- ics and effectiveness: Five years of applied research. In S. A. Wheelan (Ed.), The handbook of group research and practice ( p p . 285—3II). Thousand Oaks, CA: Sage.

Nieva, V. F., Fleishman, E. A., & Rieck, A. (1978). Team dimensions: Their identity, their measurement and their relationships (Contract No. DAHC 19-78-C-OOOl). Washington, DC: Advanced Research Resources Organization.

Noe, R. A. (2002). Employee training and development. Boston: McGraw-Hill.

Nunnally, J. C. (1978). Psychometric theory. New York: McGraw-Hill. Orasanu, J., & Fischer, U. (1997). Finding decisions in natural environ-

ments: The view from the cockpit. In C. E. Zsambok & G. Klein (Eds.), Naturalistic decision mailing expertise: Research and appli- cations (pp. 343-357). Mahwah, NJ: Erlbaum.

Ora.sanu, J., & Salas, E. (1993). Team decision making in complex envi- ronment. In G. Klein, J. Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in action: Models and methods (pp. 327-345). Norwood, NJ: Ablex.

Oser, R. L., Salas, E., Merket, D. C , & Bowers, C. A. (2001 ). Applying resource management training in naval aviation: A methodology and lessons learned. In E. Salas, C. A. Bowers, & E. Edens (Eds.), Im- proving teamwork in organizations (pp. 283—304). Mahwah, NJ: Eribaum.

•Parsons, K. L. ( 1981). The effects of load sharing system training upon team performance (Doctoral dissertation, Michigan State University, 1980). Dissertation Abstracts International, 42(6-A), 2876-2877.

»Pearsall, M. J., Ellis, A. P. J., & West, B. J. (2004, August). Reducing the negative effects of stress in team contexts: The impact of cross- training. Paper presented at the Annual Conference of the Academy of Management, New Orleans, LA.

Pfeffer, J., & Sutton, R. I. (2000). The knowing-doing gap: How smart companies turn knowledge into action. Boston: Harvard Business School Press.

»Prichard, J. S., & Ashleigh, M. J. (2007). The effects of team-skills train- ing on transactive memory and performance. Small Group Research, 38, 696-726.

•Rapp, T. L., & Mathieu, J. E. (2007). Evaluating an individually self- administered generic teamwork skills training program across time and levels. Small Group Research, 38, 532-555.

•Rollins, M. L., & Angelcyk, J. J. (1995). A descriptive study of crew resource management attitude change. In N. Johnston, R. Fuller, & N. McDonald (Eds.), Aviation psychology: Training and selection: Proceedings of the 21st Conference of the European Associationfor Aviation Psychology (Vol. 2, pp. 45-50). Aldershot, UK: Avebury Aviation.

Rosenthal, R. (1991). Meta-analytic procedures for .mcial research. Newbury Park, CA: Sage.

Rosenthal, R., & DiMatteo, M. R. (2001). Meta-analysis: Recent devel- opments in quantitative methods for literature reviews. Annual Review of Psychology, 52, 59-82.

Saavedra, R., Earley, P. C , & Van Dyne, L. (1993). Complex interde- pendence in task-performing groups. Journal of Applied Psychology, 78, 61-72.

Salas, E., Burke, C. S., Bowers, C. A., & Wilson, K. A. (2001). Team training in the skies: Does crew resource management (CRM) train- ing work? Human Factors, 43, 641-674.

Salas, E., Burke, C. S., & Cannon-Bowers, J. A. (2002). What we know about designing and delivering team training: Tips and guidelines. In K. Kraiger (Ed.), Creating, implementing, and managing effective training and development: State-of-the-art lessons for practice (pp. 234-259). San Francisco: Jossey-Bass.

Salas, E., & Cannon-Bowers, J. A. ( 1997). Methods, tools, and strategies for team training. In M. A. Quiñones & A. Ehrenstein (Eds.), Train- ingfora rapidly changing workplace (pp. 249-279). Washington, DC: American Psychological Association.

Salas, E., & Cannon-Bowers, J. A. (2000). The anatomy of team train- ing. In S. Tobias & J. D. Fletcher (Eds.), Training and retraining: A handbookfor business, industry, government, and the military (pp. 312-335). New York: Macmillan Reference.

Salas, E., Cannon-Bowers, J. A., & Johnston, J. H. ( 1997). How can you turn a team of experts into an expert team? Emerging training

TEAM TRAINING EFFECTIVENESS 933

strategies. In C. E. Zsambok & G. Klein (Eds.). Naturalistic decision making (pp, 359-370), Mahwah, NJ: Erlbaum,

Salas. E„ DiazGranados, D,, Weaver, S, J,, & King, H, (2008), Does team training work? Principles for health care. Academic Emergency Medicine. 15, 1002-1009,

Salas, E,, Dickinson, T, L,, Converse, S, A,, & Tannenbaum, S, I, (1992), Toward an understanding of team performance and training. In R, W, Swezey & E, Salas (Eds,), Teams: Their training and performance (pp, 3-29), Norwood, NJ: Ablex,

Salas, E,, Nichols, D, R,, & Driskell, J, E, (2007), Testing three team training strategies in intact teams: A meta-analysis. Small Croup Research. 38,A1\-Ai%.

Salas, E,, Rozell, D,, Mullen, B,, & Driskell, J, E, (1999), The effect of team building on performance: An integration. Small Group Re- search. 30. 309-329,

Salas, E,, Stagl, K, C , & Burke, C, S, (2004), 25 years of team effec- tiveness in organizations: Research themes and emerging needs. In C, L. Cooper & I, T, Robertson (Eds,), tntemalional review of indus- trial and organizational psychology (pp, 47-91), New York: Wiley,

Salas, B,, Stagl, K, C, Burke, C, S,, & Goodwin, G, F, (2007), Fostering team effectiveness in organizations: Toward an integrative theoret- ical framework of team performance. In R, A, Dienstbier (Series Ed,) & B, Shuan, W, Spaulding, & J, Poland (Vol, Eds,), Modeling com- plex systems. Vol. 52: Current theory and research in motivation (pp, 185-243), Lincoln: University of Nebraska Press,

Salas, E,, Wilson, K, A,, Burke, C, S,, & Wightman, D, (2006), Does crew resource management training work? An update, an extension, and some critical needs. Human Factors. 48, 392-412,

Schmidt, F L,, & Le, H, (2005), Hunter-Schmidt Meta-Analysis Pro- grams ÇVersion 1,1) [Computer software), Iowa City: University of Iowa, Department of Management & Organizations,

•Schreiber, M, A,, Holcomb, J, B,, Conaway, C, W,, Campbell, K, D,, Wall, M,, & Mattox, K, L, (2002), Military trauma training per- formed in a civilian trauma center, Joumai of Surgical Research, 104, 8-14,

»Serfaty, D,, Entin, E, E,, & Johnston, J, H, ( 1998), Team coordination training. In J, A, Cannon-Bowers & E, Salas (Eds,), Making deci- sions under stress: Implications for individual and team training (pp, 221-245), Washington, DC: American Psychological Association,

Shadish, W, R,, Cook, T, D,, & Campbell, D, T, (2002), Expérimentai and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin,

»Shapiro, J, J,, Morey, J, C , Small, S, D,, Langford, V,, Kaylor, C, J,, Jagminas, L,, et al, (2004), Simulation based teamwork training for emergency department staff: Does it improve clinical team perfor- mance when added to an existing didactic teamwork curriculum? Quality and Safety in Health Care, / J, 417-421,

Shrout, P E,, & Fleiss, J, L, (1979), Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428,

»Siegel, A, I,, & Federman, P, J, (1973), Communications content train- ing as an ingredient in effective team performance. Ergonomics, 16, 403-416,

Smith-Jentsch, K, A,, Salas, E„ & Brannick, M, T, (2001 ), To transfer or not to transfer? Investigating the combined effects of trainee char- acteristics, team leader support, and team climate, Joumai of Applied Psychology, 86, 279-292,

Stagl, K, C , Salas, E,, & Fiore, S, M, (2007), Best practices in cross train- ing teams. In D, A, Nembhard (Ed,), Wort^orce cross training hand- book (pp, 156-175), Boca Raton, FL: CRC Press,

Stagl, K, C , Salas, E,, Rosen, M, A,, Priest, H, A,, Burke, C, S,, Goodwin, G, F , et al, (2007), Distributed team performance: A multi- level review of distribution, demography, and decision making. Research in Multi-Level Issues. 6, 11-58,

Steiner, I, D, (1972), Group process and productivity. Orlando, FL: Academic Press,

Stock, W, A, (1994), Systematic coding for research synthesis. In H, Cooper & L, V, Hedges (Eds,), The handbook of research syn- thesis (pp, 97-110), New York: Russell Sage Foundation,

»Stout, R, J,, Salas, E,, & Fowlkes, J, E, (1997), Enhancing teamwork in complex environments through team training. Group Dynamics: Theory, Research, and Practice, I, 169-182,

Sundstrom, E,, Mclntyre, M,, Halfhill, T,, & Richards, H, (2000), Work groups: From the Hawthome studies to work teams of the 1990s and beyond. Group Dynamics, 4, 44-67,

Tannenbaum, S, (2002), A strategic view of organizational training and learning. In K, Kraiger (Ed,), Creating, implementing, and manag-

ing effective training and development (pp, 10-52), San Francisco: Jossey-Bass.

Tannenbaum, S, I,, Beard, R, L,, & Salas, E, (1992), Team building and its influence on team effectiveness: An examination of conceptual and empirical developments. In K, Kelley (Ed,), Issues, theory, and research in industrial/organizational psychology (pp, 117-153), Amsterdam: Elsevier,

Tuckman, Bruce W, (1965), Developmental sequence in small groups. Psychological Bulletin, 63, 384-399,

»Volpe, C, E,, Cannon-Bowers, J, A,, Salas, E„ & Spector, P E. (1996), The impact of cross-training on team functioning: An empirical investigation. Human Factors, 38, 87-100,

Whitener, E, M, (1990), Confusion of confidence intervais and credi- bility intervals in meta-analysis, Joumai of Applied Psychology, 75,315-321,

Wiener, E, K,, Kanki, B, G,, & Helmreich, R, L, (Eds,), ( 1993), Cockpit resource management (pp, 479-501), San Diego, CA: Academic Press,

Yetton, P W,, & Bottger, P C, (1982), Individual versus group problem solving: An empirical test of best-member strategy. Organizational Behavior and Human Performance, 29, 307-321,

Eduardo Salas is a Pegasus Professor and university trustee chair in the Department of Psychology and In- stitute for Simulation & Training at the University of Central Rorida, Orlando, He received a Ph.D, in indus- trial/organizational psychology from Old Dominion University in 1984,

Deborah DiazGranados is a doctoral candidate and research associate in the Department of Psychology and Institute for Simulation & Training at the University of Central Florida, Orlando, where she received her M.S. in industrial/organizational psychology in 2005.

Cameron Klein is a doctoral candidate in the Depart- ment of Psychology at the University of Central Florida, Orlando, and a consultant at Kenexa in Lincoln, Nebraska. He received his M.S. in industrial/organiza- tion psychology from the University of Central Florida in 2004.

C. Shawn Burke is a research scientist in the Institute for Simulation & Training, University of Central Florida, Orlando. She received her Ph.D. in industrial/organiza- tional psychology from George Mason University in 2000.

Kevin C. Stagl is a senior talent advisor at Talent Thresh- old LLC, Orlando, and an adjunct professor at the Uni- versity of Central Florida, where he received his Ph.D. in industrial and organizational psychology in 2006.

Gerald F. Goodwin is a research psychologist in the Basic Research Unit at the Army Research Institute, Arlington, Virginia. He received his Ph.D, in industrial/ organizational psychology from Pennsylvania State University in 1999.

Stanley M. Halpin is chief of the Fort Leavenworth Research Unit, the leader development unit of the Army Research Institute, Fort Leavenworth, Kansas, He received his Ph.D, in social psychology from Purdue University in 1970.

Date received: June 18, 2008 Date accepted: November 25, 2008