Synthesis Paper

Leader cognition under threat: “Just the Facts”

William B. Vessey, Jamie Barrett, Michael D. Mumford ⁎ The University of Oklahoma, USA

a r t i c l e i n f o a b s t r a c t

It has been argued that leader cognition is a particularly important influence on team performance under conditions of crisis or threat. The goal of the present effort was to assess the merits of processing performance information, as opposed to processing social information, for leader performance in terms of creativity of solutions and the quality of solutions with respect to domain specific performance variables. Undergraduates were asked to assume the role of a leader in a marketing firm and provide solutions to three marketing problems. It was found that providing training instrategies for working with performance information (e.g.,causes, resources, restrictions, and contingencies) resulted in higher levels of performance than providing training in strategies for working with social information (e.g., actors, affect, goals, and social systems). Moreover, training in strategies for working with performance information proved especially beneficial when more elements of the problem situation were under leader control. The implications of these findings for understanding leader cognition and leader performance are discussed.

© 2011 Published by Elsevier Inc.

Keywords: Leadership Cognition Problem-solving Crises

Few scholars would dispute the point that leader performance is a complex, unusually complex, phenomenon (Bass, 1990; Yukl, 2009). Nonetheless, in recent years, we have begun to see a new understanding emerge of many of the variables contributing to leader performance. For example, the key behaviors evidenced by leaders, consideration, initiating structure, participation, and change management, have been identified (Yukl, 2009) along with the conditions that moderate the impact of these behaviors on performance (Vroom & Jago, 2007). We are now gaining an understanding of how leaders formulate and articulate viable visions (Shipman, Byrne, & Mumford, 2010). We have begun to identify the mechanisms by which leaders interact with followers, motivate their followers, and share leadership responsibilities (Avolio & Gardner, 2005; Henderson, Liden, Glibkowski, & Chaudhry, 2009; Friedrich, Vessey, Schuelke, Ruark, & Mumford, 2009). Additionally, we have begun to gain an understanding of when leadership really counts with respect to team and organizational performance (Day, Gronn, & Salas, 2006).

Despite these advances in our understanding of leadership, and leader performance, many questions remain unanswered (Mumford, Friedrich, Caughron, & Byrne, 2007). One critical question which has not been clearly answered pertains to how people think about social or organizational problems that call for leadership (Lord & Hall, 2005). Answers to this question are important for two reasons. First, the evidence gathered in studies by Connelly et al. (2000), Hedlund et al. (2003), Lord, de Vader, and Alliger (1986), Marta, Leritz, and Mumford (2005), and Mumford, Campion, and Morgeson (2007) indicates that cognitive capacities such as intelligence, planning, and problem definition are powerful influences on leader performance. Second, understanding the nature of leader cognition might provide a basis for the development of new interventions likely to improve leader performance under conditions where cognition is critical to performance (Marcy & Mumford, 2010).

One situation where cognition is held to be critical to leader performance is when a team or organization is presented with a crisis (Drazin, Glynn, & Kazanjian, 1999; Weick, 1995) — a complex, novel, ill-defined problem associated with high stakes, high risk, outcomes (Hunt, Boal, & Dodge, 1999). Recently, Mumford, Friedrich et al. (2007) proposed a model of how leaders think about crisis situations based on existing research into leader cognition and decision making. Our intent in the present effort was twofold. First, we hoped to show that training in certain key capacities embedded in the model would lead to differential effects in

The Leadership Quarterly 22 (2011) 710–728

⁎ Corresponding author at: Department of Psychology, The University of Oklahoma, Norman, OK 73019, USA. E-mail address: [email protected] (M.D. Mumford).

1048-9843/$ – see front matter © 2011 Published by Elsevier Inc. doi:10.1016/j.leaqua.2011.05.011

Contents lists available at ScienceDirect

The Leadership Quarterly

journal homepage: www.elsevier.com/locate/leaqua

how people in leadership roles think about crises. Second, we hoped to examine attributes of the situation that would moderate the impact of this training. Identifying key capacities affecting leader performance in crisis situations and the specific situations in which these capacities come into play is a critical step in developing training that may be applied in real world settings. To this end a study was conducted in which participants were presented with training on the key capacities embedded in the model, and were then presented with a set of low-fidelity leadership problem-solving scenarios containing different potential moderators of the effects of these capacities on leader performance. Overall, the intended contributions of our work are 1) to increase understanding of the way leaders think during crises, 2) to identify strategies that may increase leader performance during crises, and 3) to identify situational factors that may influence the effect of these strategies on leader performance. In the sections that follow we first describe the research into leader cognition during crises and potential interventions to improve leader cognition, we then discuss the methods used in this study of leader cognition, and concluding with a discussion of the results of this study and the potential implications of this study for our understanding of leader cognition during crises.

1. Leader cognition

Crises are often used to understand leader cognition because it is under crisis conditions that organizations value leader problem-solving (Hunt et al., 1999; Mumford, 2006). A model of leader cognition developed to specifically address leader cognition under crisis conditions was recently proposed by Mumford, Friedrich et al. (2007). The basis of this model of leader cognition is sensemaking, a process through which people frame experiences as being meaningful in some specific way (Weick, 1995). The importance of sensemaking to leader problem-solving under crisis conditions becomes apparent when it is recognized that novel, complex, ill-defined problems can be construed, or understood, in many different ways. As a result, the structure imposed on the crisis situation is important because it both frames, or defines, the problem at hand and provides a structure in which the leader can formulate a plan for addressing the crisis (Mumford, Schultz, & Osburn, 2002).

The model proposed by Mumford, Friedrich et al. (2007) holds that leader sensemaking begins with scanning of the internal and external environment vis-à-vis monitoring mental models in an effort to identify emerging novel, high risk, high reward problems (Koberg, Uhlenbruck, & Sarason, 1996). These mental models provide a framework for working with information in a particular way. When a crisis event has been identified, information gathering will be initiated to determine the nature and significance of the event (Weick, 1995). Information gathering, in turn, leads to activation of descriptive mental models that may be used to understand, or make sense of, the crisis situation (Weick, 1995). These descriptive mental models are noteworthy because they provide a basis for identifying critical causes and salient goals. These causes, goals, and the descriptive mental models they are based on, in turn activate relevant case-based knowledge that might be used in problem-solving (Kolodner, 1997; Strange & Mumford, 2005). Case-based knowledge is knowledge based on past experiences, where an individual recalls a past experience and applies the lessons, both positive and negative, learned to a related event. For example, if an individual must make a decision regarding changing from one job to another they may remember their past experiences changing jobs. They might then apply this knowledge to the current situation by assuming similar actions to those they took in the past would have similar results in the present. Subsequent analysis of those cases with respect to more objective performance information, information pertaining to likely causes, resources, restrictions, and contingencies, and more subjective social information, information pertaining to actors, affect, goals, and social systems, is analyzed to formulate a prescriptive mental model (Strange & Mumford, 2005).

This prescriptive mental model, a model providing a template plan for actions in the situation at hand (Mumford et al., 2002), provides a basis, or framework, for forecasting the outcomes of various actions that might be taken (Byrne et al., 2010). With reflection on the outcomes arising from these forecasts, both self reflection and reflection on social systems, plausible alternative plans can be identified giving rise to the formation of plans and backup plans (Xiao, Milgram, & Doyle, 1997). Opportunistic execution of these plans (Patalano & Siefert, 1997) in turn gives rise to adaptive responses to the problems arising in crisis situations (Mumford, Friedrich, Caughron, & Antes, 2009).

Althoughthis modelof leadercognitionis complex, four linesofevidence suggestthatit might provide a plausiblebasis fordescribing how leader think, and presumably, think about crises. One key implication of this model is that planning will be an important influence on leader problem-solving. In a study examining the influence of planning on performance during crises, Marta et al. (2005) asked teams to formulate plans for turning around a failing car company. When the viability of plans proposed by these teams was evaluated, it was found that the best performance was obtained from teams where leaders emerged who possessed requisite planning skills. However, it was also found that people possessing planning skills did not consistently emerge as leaders of these teams.

In a second study, Shipman et al. (2010) examined the influence of forecasting on leader vision formation. Participants in this study were asked to assume the role of a principal of a new experimental school and were asked to prepare a speech to be read to students, parents, and teachers, describing their vision for this school. Prior to preparing their vision statements, participants were asked to work through a set of exercises, putatively provided by a consulting firm where forecasting was required. It was found, in keeping with Mumford, Friedrich et al. (2007) model, that the amount of forecasting was strongly, positively, related to (r≈0.35) to the quality of the vision statements produced by people working in this leadership role.

Not only has evidence been found for key processing operations proposed in this model, Strange and Mumford (2005) have also provided evidence pointing to the importance of case-based knowledge, a central component of the model. Again, in this study participants were asked to assume the role of a principal of a new experimental secondary school and were asked to formulate a speech to be given to students, parents, and teachers where they were to describe their vision for this school. Exercises were provided to help participants prepare their speech, where the nature of these exercises allowed participants to consider either good or poor cases and strategies for analyzing these cases with respect to causes, goals, both causes and goals, or neither causes or goals. It was found that case

711W.B. Vessey et al. / The Leadership Quarterly 22 (2011) 710–728

models provided a basis for vision formation, with analysis of weak cases with respect to goals and strong cases with respect to causes resulting in production of the strongest vision statements.

In still another study along these lines, Marcy and Mumford (2010) examined the contribution of causal analysis to the performance of people working in leadership roles. In this study, undergraduates were asked to work on a computer simulation of a university president's job. Prior to starting work on the simulation, participants were given instruction in strategies for think about causes — for example, “think about causes that have big effects,” “think about causes that have direct effects,” and “think about causes that affect multiple outcomes.” It was found that this instruction in causal analysis resulted not only in better performance on the leadership simulation, but that it also resulted in stronger mental models being applied to the leadership problems broached by this simulation. Based on the mounting evidence for the plausibility of the model, and the importance of leader performance in crises to leaders and organizations, this model was chosen as the framework for the present effort.

2. Improving leader cognition

Although none of the studies described above provided a complete test of Mumford, Friedrich et al. (2007) model of leader cognition, the findings obtained in these studies do suggest this might be a plausible model of leader cognition, at least under conditions where planning, vision formation, and decision-making are required. The plausibility of this model, however, broaches another question. Specifically, what does this model tell us about the kind of interventions that might prove useful in enhancing leader cognition and improving leader performance in solving crisis problems?

In fact, the Marcy and Mumford (2010) study provides a partial answer to this question. In this study, people assuming leadership roles were provided with a series of self-paced instructional modules. These instructional modules provided training in strategies that might improve the analysis of causes embedded in case-based knowledge. For example two strategies trained were “identify causes having large effects on what is occurring in the problem” and “identify causes that may affect multiple aspects of the problem.” The findings obtained in the Marcy and Mumford (2010) study indicated that this particular strategy intervention was beneficial with regard to leader problem-solving on an organizational simulation task. However, key causes represent only one way of working with case-based knowledge. For example, strategies for working with resources might also prove useful (Nohria & Gulati, 1996).

More centrally, the model of leader cognition proposed by Mumford, Friedrich et al. (2007) suggests there are two types of information leaders might work with in solving crisis problems. As mentioned earlier, when a leader is working through a problem they often turn to their case-based knowledge. Case-based knowledge is inherently complicated and involves both objective and subjective information regarding past performance (Hammond, 1990). First, leaders might work with more objective, clearly defined, information pertaining to performance in the situation at hand. In other words they might work with information bearing on causes, resources, restrictions, and contingencies bearing on the problem at hand. Second, leaders might work with more subjective, less defined, social information pertaining to the problem at hand. In other words they might work with information bearing on actors, affect, goals, and social systems. Although it seems plausible to argue that leaders' solutions to crisis problems might consider both performance and social information it is open to debate whether it is more valuable to work with performance information or social information in attempts to resolve the problems arising in a crisis.

On the one hand, leadership ultimately involves the exercise of influence (Yukl, 2009). The terms “exercise influence” are noteworthy because they imply that leadership involves, either directly or indirectly, social interaction. The importance of social interaction to leader performance would, in turn, seem to suggest that leader problem-solving performance would improve when leaders work with, or are provided strategies for working with, social information.

Although this argument seems plausible when taken at face value at least two other considerations provide a potential counter argument. First, in crisis situations followers, groups, and organizations become unreliable, or unpredictable, entities as a result of stress and threat (Hunt et al., 1999; Weick, 1995). The unpredictability of social reactions to crisis events makes it difficult for leaders to employ social information in crafting problem solutions. Additionally, the rapidly changing nature of social information in a crisis situation may serve as a distraction, as the leader may need to adjust problem solutions to account for changes among followers, groups, and organizations. As a result, leaders might prefer to rely on performance relevant information when crafting problem solutions. Under these conditions strategies for working with performance relevant information might prove especially valuable. Additionally, training in social strategies may encourage leaders to focus on the social aspects of a problem when they otherwise would have preferred to consider morestableaspectsof theproblem due totheunpredictablenatureof socialreactionsinacrisis.This shift infocus could resultin the consideration of less relevant or controllable aspects of the problem, decreasing problem-solving performance.

Second, Mumford, Zaccaro, Harding, Jacobs, and Fleishman (2000) have argued that a leader's primary objective is to solve organizational problems. Within this pragmatic framework, followers, groups, and organizational systems are viewed as tools for solving problems and a context inwhich problem-solvingoccurs. However, the focus of leaders is on generating a solution which will work within this social context (Mumford & Van Doorn, 2001). This pragmatic focus on solution generation should, in turn, result in a preference for working with performance information in resolving crises and, one would expect, that the use of stronger strategies for working with performance relevant information would contribute more to problem-solving performance than strategies for working with social information. When these considerations about predictability and pragmatism are taken into account they lead to our first hypothesis;

Hypothesis One. Training leaders in strategies for working with performance information will result in better problem-solving in crisis situations, in terms of creativity and business performance, than training leaders in strategies for working with social information.

712 W.B. Vessey et al. / The Leadership Quarterly 22 (2011) 710–728

The value of performance information, and strategies for working with this information, in leaders' attempts to solve crisis problems depends, in part, on the situation in which the problem-solving is occurring (Marta et al., 2005). Working with performance information is useful only when the variables shaping performance can be influenced by the leaders' actions (Mumford, 2006). This observation is noteworthy because it suggests that the value of training strategies for working with performance information will depend on the extent to which causes, resources, restrictions, and contingencies can be influenced by the leader.

Some support for this proposition may be found in a study by Thomas and McDaniel (1990). They asked hospital chief executive officers to describe the extent to which they saw issues in hospital administration cases as positive versus negative, implying gain versus loss, and as controllable versus uncontrollable. The intensity of information processing the top management team engaged in was the critical dependent variable. They found that the perceived controllability of variables, for example causes, resulted in better information processing among top management teams. Moreover, controllability exerted stronger effects than gain versus loss or positive versus negative attributes of the situation. The tendency of leaders to invest cognitive resources in working with controllable aspects of the situation, in turn, implies that the value of training strategies for working with performance information would be greater when more attributes of the situation might be under the leader's control. Thus, our second hypothesis;

Hypothesis Two. Instruction in strategies for working with performance information, as opposed to social information, will result in better problem-solving, in terms of creativity and business performance, when more, as opposed to fewer, attributes of the situation might be controlled by leaders.

In addition to control, another situational variable that might influence leader problem-solving is the framing of the situation (Kahneman & Tversky, 1979). Framing has been studied with respect to a number of variables such as gain versus loss framing (Kahneman & Tversky, 1979), entity maintenance versus enhancement framing (Shamir, House, & Arthur, 1993), or social versus individual accountability framing (Yukl, 2009). However, Ward, Patterson, and Sifonis (2004), in a study of creative problem-solving, found that when problems are understood in abstract terms more creative problem solutions are generated. Because crisis problems call for novel responses on the part of leaders, these findings suggest that framing problems in abstract terms rather than in terms of concrete past experience, such as best practices, would result in better leader performance presumably due to more flexibility and creative use of strategies for working with performance information. Hence;

Hypothesis Three. Framing problems in terms of abstract features rather than concrete experience will result in better use of performance information strategies and thus better leader problem-solving, in terms of creativity and business performance, than framing problems in terms of concrete experience.

3. Method

3.1. Sample

The sample used to test these hypotheses consisted of 170 undergraduates attending a large southwestern university. The 90 men and 80 women who agreed to participate in this study were recruited from undergraduate psychology classes providing extra-credit for participation in experimental studies. Participants received a list of the studies being offered in each semester this study was being conducted and then selected the experiments in which they wished to participate. The average age of participants who agreed to participate in this study was 18 years. Their academic ability, as assessed by scores on the American College Test (ACT) lay roughly a quarter of a standard deviation above national norms for freshmen entering four year institutions. Virtually all study participants had some real world work experiences.

3.2. General procedures

To minimize demand characteristics, participants were recruited to participate in what was purported to be a three hour study of problem-solving. During the first half hour of this study, participants were asked to complete a set of timed covariates providing assessments of intelligence and divergent thinking skills. During the last half hour of this study, participants were asked to complete a set of untimed covariate controls examining personality variables and leadership styles that might affect either learning or task performance.

During the second hour of this study participants were asked to work through a series of self-paced instructional exercises. This self-paced instructional program, based on the procedures developed by Marcy and Mumford (2007), provided participants with training in the application of strategies for working with different types of information. In the case of the performance information condition, participants were provided with instruction in two strategies relevant to working with causes, resources, restrictions, and contingencies respectively, for a total of eight strategies. In the case of the social information condition, participants were provided with instruction in two strategies relevant to working with actors, affect, goals, and social systems considerations respectively, for a total of eight strategies. In the combined training condition, participants completed both the performance information and social information exercises with instruction in a total of sixteen strategies. The specific procedures used in this training are included in our discussion of experimental manipulations. Each of the single training conditions took an average of 45 min to complete, while the combined training condition took an average of 1 h to complete. It was found in follow-up questionnaires during pilot testing and through researcher observation that participants increased their speed in completing these training modules as they familiarized

713W.B. Vessey et al. / The Leadership Quarterly 22 (2011) 710–728

themselves with the instructional format. No significant differences in performance on the training itself were found between the conditions.

Following training, participants were asked to assume the role of a branch manager working in a mid-size, entrepreneurial, marketing firm. After reading through a description of this firm and their role in the firm, participants were asked to provide solutions to three marketing problems. To invoke a sense of crisis as these leaders prepared their solutions, based on the findings of Barrett et al. (2011-this issue), five negative consequences of poor solutions, two for the client, two for the marketing firm, and one for society in general, were noted. The manipulation of control occurred just after the general introductory scenario where it was noted either 4 of 6 key variables were subject to control by the leader or only 2 of those 6 key variables were subject to control by the leader. The manipulation of framing occurred as each of the problems was presented. In the abstract framing condition key concepts that should be considered in solution generation were listed while in the concrete condition the same concepts were presented by embedding them in the problem at hand. This task took an average of 45 min to complete. The average time for completion of the entire study was 2 h and 40 min.

3.3. Covariates

Because the performance task required problem-solving, participants were asked to complete measures of intelligence and divergent thinking, which have both shown significant effects on creative problem-solving performance (Vincent, Decker, & Mumford, 2002). The measure of intelligence administered was the Wonderlic Personnel Test. The Wonderlic presents 50 verbal and mathematical problems where the selected answer is scored as correct or incorrect. This test yields split half reliabilities above 0.80. Evidence for the construct validity of this measure has been provided by Frisch and Jessop (1989) and Hawkins, Faraone, Pepple, Seidman, and Tsuang (1990).

To measure divergent thinking, an ability held to improve performance on tasks calling for creative thinking, participants were asked to complete Merrifield, Guilford, Christensen, and Frick's (1962) consequences test. On this test, people are asked to list as many consequences of unlikely events (e.g., what would happen if gravity was cut in half) as they can think of. Overall, five problems are presented in a 10 minute period. When scored for fluency, the number of consequences generated, this measure yields internal consistency coefficients above 0.70. Vincent et al. (2002) have provided evidence for the construct validity of fluency scores as predictors of leader performance.

In addition to these cognitive measures, participants were also asked to complete a background data scale intended to provide a measure of interest and engagement in marketing tasks. As this is a marketing task expertise in the area may have an effect on performance. This eight item scale asks questions such as “How likely is it you will go into advertising or marketing as a career” or “How often do you discuss current advertisements with your friends.” The internal consistency coefficient obtained for this scale was above 0.70. Scott, Lonergan, and Mumford (2005) have provided evidence for the construct validity of using these kinds of background data scales as a basis for assessing interest and expertise in professional fields.

The next two measures participants were asked to complete, both untimed, were intended to take into account the instructional manipulation and the nature of the performance task. Motivation to invest resources on cognitive tasks was measured using Cacioppo and Petty's (1982) need for cognition scale. Given the length of the study it was important to consider participant motivation to invest cognitive resources in completing the task. This measure presents 34 self description questions where people are asked to indicate their investment in cognitive activities — for example “I prefer my life to be filled with puzzles I must solve.” This need for cognition scale yields internal consistency coefficients above 0.80 while evidence for the validity of this scale has been provided by Scott et al. (2005). In addition, participants were asked to complete Pintrich, Smith, Garcia, and McKeachie's (1993) measure of learning goals. The eight questions that make up this scale ask people to describe their investment in educational activities such as “I prefer courses that arouse my curiosity, even if they are difficult” or “It is important for me to understand the content of a course.” Participants are asked to rate on a 5-point scale the extent to which those statements describe them. Learning goals may influence the amount of effort participants put into training, an effect we would expect to see in their training performance. This learning goals scale yields internal consistency coefficients above 0.70 and McClendon (1996) have provided evidence for the construct validity of this measure.

In addition these task specific personality measures, participants were also asked to complete two omnibus personality inventories. The first general personality inventory was Goldberg's (1990) measure of introversion, conscientiousness, agreeableness, neuroticism, and openness. On this inventory people are presented with 100 adjectives (e.g., nice, aggressive, responsible) and they are asked to rate, on a 9-point scale, the extent to which each adjective describes them. The internal consistency coefficients obtained for these five scales is above 0.80. Evidence for the construct validity of these scales has been provided by Becker, Billings, Eveleth, and Gilbert (1997), Conway and Peneno (1999), Reysen (2005), and Saucier (2002). The second general personality inventory participants were asked to complete was Watson, Clark, and Tellegen (1988) PANAS inventory which provides scales measuring positive and negative effects. On this inventory participants are asked to rate, on a 5- point scale, the extent to which 20 emotional adjectives (e.g., excited, guilty, nervous) describe them. These scales yield internal consistency coefficients above 0.80. Crawford and Henry (2004), Crocker (1997), and Kercher (1992) have provided evidence for the validity of these scales as measures of trait affect.

The final two untimed covariate measures examined preferred leadership styles. Participants were asked to complete Fleishman and Harris (1962) measures of consideration and initiating structure. Consideration and initiating structure may influence the way leaders approach a problem and thus were included asa potential covariate. The 20 items included in this inventory presentbehavioral statements illustrative of each style. For example an initiating structure question states “Assign people under you to particular tasks”

714 W.B. Vessey et al. / The Leadership Quarterly 22 (2011) 710–728

while a consideration question states “Asks for sacrifices from people for the good of the entire department.” Participants are asked to rate on a 5-point scale how typical these behavioral statements are of their behavior in a leadership role. The resulting measures of consideration and initiating structure yield internal consistency coefficients above 0.80. Evidence for the validity of these scales as measures of consideration and initiating structure has been provided by Bass (1990) and Judge, Colbert, and Ilies (2004).

The second measure of leadership style participants were asked to complete focused on cognitive style in solving leadership problems. This measure, drawn from Bedell-Avers, Hunter, and Mumford (2008) is intended to provide assessments of charismatic, ideological, and pragmatic styles. Similar to initiating structure and consideration, leadership style may influence the manner in which an individual approaches a leadership problem. On this 12 item measure participants are presented with three one paragraph abstracts, describing incidents of leadership illustrating each style, and they are asked to indicate the incident most similar to their own style. These scale yield split-half reliabilities in the 0.70 s. Bedell-Avers et al. (2008) have provided evidence for the construct validity of the resulting scales measuring charismatic, ideological, and pragmatic stylistic differences.

3.4. Experimental task

The experimental task participants were asked to work on was intended to provide a low-fidelity simulation of the work performed by mid-level leaders in marketing firms (Motowidlo, Dunnette, & Carter, 1990). On this task participants were asked to assume the responsibility of a branch manager — a mid to high-level leader. They were informed that in this leadership role they were responsible for 5 project teams consisting of 7 to 10 members with each team working on a different contract. As the leader they were informed that their main goal was to help the firm build a strong client base. To accomplish this they were expected to help arrange contracts, supervise project work, and assist team leaders in their work as indicated by the demands of the situation.

Prior to reading through this description of theirresponsibilities inthis leadership role, participants were asked to read througha four paragraph description of the marketing firm itself — Aperture. This descriptive material noted that the firm had been formed in 1995 by an entrepreneur who was interested in media. Over the next ten years the firm had grown to one of the20 leading marketing companies in the United States. The success of the firm was attributed to the close relationships maintained with clients and the extensive services provided to clients. It was noted that the firm provided a variety of marketing campaigns to clients of varying size, employing a staff that included copy writers, graphic designers, multi-media specialists, photographers, market researchers, and technology specialists.

Subsequently, these leaders were presented with three marketing problems calling for a leadership response — automotive, snack food, and sports drink marketing problems. Each problem was initiated through a simulated email. These emails followed a consistent structure. First, the email provided a description of the client firm. Next, the product to be marketed was described. Subsequently, current marketing initiatives being pursued with respect to this product were described. Following this description of current marketing activities, the problem confronting the client firm was presented. In all cases, these problems were structured to present a novel, complex, ill-defined problem involving marketing strategy. Leaders asked to provide ideas that would help resolve this problem. Fig. 1 illustrates one of the three problems leaders were asked to solve.

With regard to these problems two further points should be noted. First, the material presented in each problem was drawn from marketing case studies (Swayne & Ginter, 1993). The material presented in these cases was abstracted by a psychologist such that critical aspects of case context and case content were reflected in the email. However, no information on case solutions or case outcomes was provided. The accuracy of these case summaries was assessed by two other psychologists and any indicated revisions in adequate case material added, based on revision comments, were made.

Second, prior to starting work on problem-solutions an addendum to the initial email was presented. This addendum, putatively provided based on company policy, was intended to invoke a sense of threat with respect to the need for a viable problem solution. Each addendum noted two risks to the client, two risks to the marketing firm, and one general social risk associated with the marketing effort. Again, these risks were all drawn from relevant case material. These risks were presented to ensure that participants, when solving these problems, experienced an aspect of crisis situations — threat. Threat as opposed to time pressure was used to invoke a sense of crisis based on the findings of Barrett et al. (2011-this issue) indicating that threat enhances cognitive processing among people working in leadership roles in contrast to time pressure which diminishes cognition (Fiedler & Garcia, 1987). Fig. 2 provides an illustration of this addendum.

3.5. Dependent variables

Participants, in their role as a leader in this marketing firm, were asked to provide a page to a page and a half written solution to each of these three problems. Solutions were appraised by four judges, all doctoral students in industrial and organizational psychology, familiar with the source cases and marking principles but unaware of hypotheses underlying the present study. These judges were asked to rate the three problem solutions provided with respect to creativity and business performance criteria.

With regard to creativity, judges were asked to rate each solution with respect to quality, originality, and elegance based on the findings of Besemer and O'Quin (1999). Quality was defined as a logical, workable, solution to the problem, originality was defined as a novel, surprising, solution to the problem, and elegance was defined as a solution where all parts worked effectively as a whole. These ratings were made on a set of benchmark rating scales. Benchmark rating scales present illustrations of good, average, and poor performance with respect to the dimension under consideration. Prior research by Redmond, Mumford, and Teach (1993) indicates that the use of benchmark rating scales is attractive when complex products are being evaluated with respect to both reliability and validity of the resulting ratings. Fig. 3 illustrates the benchmark rating scales developed for appraising the quality, originality, and elegance of problem solutions.

715W.B. Vessey et al. / The Leadership Quarterly 22 (2011) 710–728

With regard to business performance, these judges were asked to rate the solutions provided with respect to five marketing performance dimensions identified by Lonergan, Scott, and Mumford (2004). These business performance dimensions included 1) feasibility of the proposed solution, 2) efficiency of the proposed solution, 3) cost effectiveness of the proposed solution,

From: Erin Ship Sent: Wed 4:17 PM To: [email protected] Subject: Puma Account

Boss,

Our team is working on the recent contract we’ve acquired working on the Puma sports car marketing for Flight Autos and needs some assistance from you. Flight is launching an advertising campaign to promote their new inexpensive sports car, the Puma. Let me give you a refresher on what is going on with this client.

Flight Autos is one of the top 10 employers in the United States. The automobile market has very complex segmentation because of the number of things that go into the decision to purchase a car. Product features and price vary wildly. The market we’re dealing with is high-performance sports cars. The cars used to be produced in small numbers and required a major investment for the buyer. Flight is hoping to change the way this market works by introducing the Puma, a high-performance sports car that will appeal to middle and upper-middle class car buyers. If the Puma can gain ground as a new option for middle and upper-middle class car buyers Flight will basically have one area of the market entirely to itself for a time. The revenue generated by this market dominance could help save Flight from its recent financial trouble.

The Puma has shown in tests, conducted at Flight’s proving grounds, that it has performance comparable to other high- performance sports cars at four times the price, and has shown much higher performance than similarly priced sports cars. Early reviews by auto magazines have confirmed these findings with the early reviews being universally positive. Flight has figured out how to manufacture the Puma at minimal cost compared to their competitors allowing for a much lower price at the dealership.

We’ve already launched a campaign to advertise the introduction of the Puma. We’ve run advertisements in a variety of automobile magazines touting the technical aspects of the car. Television ads are now running which focus on comparisons between the Puma and other similarly priced sports cars. The Puma is also now being promoted with the publication of a magazine focused on the Puma. Finally, we’ve arranged for the Puma to be entered into a number of professional races around the country in an attempt to showcase its performance.

Flight needs us to come up with ways to track the effect of our advertising on sales as soon as possible. Some previous attempts at tracking the effectiveness of our advertising have been to give rebates if customers bring in an advertisement or to have contests associated with a product. We then tracked the number of people responding. The Puma will be sold by the Flight Autos network of over 3000 dealers across the United States. Because of the number of dealerships involved it will be difficult to get an idea of how early advertising is affecting car buyers without a concrete system in place. In addition, Flight has a limited time to take advantage of their new manufacturing process. As soon as the Puma is launched it will be only a matter of time before other companies are able to copy the technique. Flight needs to be able to take advantage of this window of opportunity so they need to be able to quickly change their advertising if it’s deemed necessary.

We need some ideas for how to track the effect of our advertising. Anything you can come up with would be helpful for the team. Thanks for the help, Erin

Fig. 1. Illustrative email describing marketing problem.

COMPANY POLICY REQUIRES US TO DETERMINE THE RISKS ENTAILED IN ANY ACTION TO BE TAKEN. OUR TEAM HAS IDENTIFIED THE FOLLOWING RISKS THAT MAY BE ATTACHED TO ANY ACTION TAKEN ON THIS MATTER.

Flight has recently fallen on some hard times. Between labor costs and lagging sales they’ve had financial difficulty for the last few years. If this campaign doesn’t work Flight will be losing a large amount of money. Flight is putting its future on the line with the launch of the new Puma and if sales goals are not met, they could be looking at major cutbacks including firing and plants shutting down permanently. This could have major negative impacts on the U.S. economy. We’ll also be losing a valuable contract for the firm and the money they bring in. Finally the firm’s reputation will suffer if our marketing fails because we’ll be blamed, to some degree, for the firings and plant closings at Flight Autos.

Fig. 2. Risk addendum to problem statement email.

716 W.B. Vessey et al. / The Leadership Quarterly 22 (2011) 710–728

Quality

1) Poor rating: The parts of this plan do not fit together well and are incomplete. It does not make sense and fails to provide an answer to the problem in a sound, rational, manner. The solution is very general with few details about how this solution would be implemented.

Ex: “I would quit.” “Develop a plan and stick to it.” “Decreased costs and increased revenues.”

2) Poor to average rating

3) Average rating: The parts of this plan fit somewhat well together. Though missing some specifics, most ideas are complete. The solution may contain one or two confusing elements, but makes general sense, overall.

Ex: “Try something new like a summer sale or coupon campaign. This could help with getting new customers during a profitable time of year, then you’d have more customers in the slower parts of the year. Name a flavor of the drink after a famous athlete.”

4) Average to excellent rating

5) Excellent rating: The parts of this plan fit together exceptionally well. The solution includes all necessary elements for establishing solving this company’s problems and is soundly complete. The plan is clear and exceptionally coherent.

Ex: “First they need to run some test trials with advertising starting locally and try to promote new products. If it works they could slowly expand nationally, possibly by picking up new investors who will back them while expanding. To help with expanding, look at letting other distributors pick up the work until the company is large enough to take over distribution again. Make sure to keep testing advertising to make sure it is working nationwide, not just in certain areas.”

Originality

1) Poor rating: The solution is very predictable and fails to provide any new or unique ideas.

Ex: “Advertise in sports.” “Advertise in non-sports magazines and shows.” “Run ads focusing on the heat and how the drink can cool you down.”

2) Poor to average rating:

3) Average rating: The solution has a few original and unique elements. The solution, however, still contains many predictable concepts.

Ex: “Use a different type of packaging than most companies. Do packs of 4, 14, and 20 instead of the usual 6, 12 and 24 people are used to. Update the packaging to match the new attitude.”

4) Average to excellent rating

5) Excellent rating: The solution is clearly unique. It has core elements that appear wholly original—particularly to the participant.

Ex: “Do a large advertising campaign but when it comes out make sure there are only a limited number of cars in each area. This will help generate buzz and word of mouth advertising that would save the company money on traditional advertising. Make sure there are enough cars in each area to be seen but to still be hard to get.”

Elegance

1) Poor rating: The elements of the solution do not fit well together. Most of the elements are unnecessary; there is little or no focus to the solution.

Ex: “Need to make manufacturing process unable to be copied. Come up with new commercials. Make a better packaging. Do a survey of buyers and ask at the end what made it more appealing. Make magazine ads have catchy things. Have multiple sponsorships.”

2) Poor to average rating.

3) Average rating: The solution has some elements that fit well together. The solution is somewhat focused. Many of the elements to the solution are necessary, though it still contains some unnecessary aspects.

Ex: “Revise current campaign to include radio spots and TV clips that show the Puma outperforming other cars. Poll which locations have the target audience of the car to identify hot-spots. Do more advertising in these areas. Research gas prices where Puma likely to be sold. Allow planning of marketing to change every few months.”

4) Average to excellent rating.

5) Excellent rating: The elements of the solution fit exceptionally well together. The solution is focused, with all of the elements included being necessary for the solution to work. The solution uses only the minimum number of elements to be effective.

Ex: “Start a new TV ad series that focuses on local and hometown networks and values, targeting both kids and adults. Advertise well-known brands along with new healthier products while focusing on ties to the local community efforts and outreach programs. Address national expansion goals with some limited advertising in national magazines.”

Fig. 3. Quality, originality, and elegance rating scales.

7 1 7

W .B . V essey

et al.

/ Th

e Lead

ersh ip

Q u arterly

2 2 (2 0 1 1 ) 7 1 0 – 7 2 8

4) reputation maintenance/enhancement of the proposed solution for the client firm, and 5) balance of investment in the solution. Again, these ratings were made using benchmark rating scales. Fig. 4 illustrates the benchmark rating scales developed for appraising solution feasibility and solution efficiency.

Development of these benchmark rating scales was based on the solutions provided to these marketing problems by a sample of 20 undergraduates during pilot testing. Judges, four doctoral students in industrial and organizational psychology, were asked to rate the solutions provided on a 5-point scale. Anchors were selected that had means near the high, medium, and low points on the scale and evidenced high levels of agreement among the judges.

Prior to rating solutions judges were asked to participate in a 20 hour training program. In this training program the nature of the rating task was described to the judges and the conceptual and operational definitions of each of the rating dimensions was presented. Subsequently, judges were asked to rate a set of sample problem solutions using the benchmark rating scales. After they making their ratings, the judges convened as a panel to discuss their ratings and resolve any observed discrepancies. Three such panel meetings occurred, with inter-rater agreement being checked using the ratings obtained for a set of problem solutions given to judges following each panel meeting.

The average interrater agreement coefficient obtained from judges following training was 0.85 for ratings of solution quality, 0.93 for ratings of solution originality, and 0.84 for ratings of solution elegance. For ratings bearing on business performance, the interrater agreement coefficient obtained for feasibility was 0.93, for efficiency 0.81, cost effectiveness 0.80, reputation maintenance 0.76, and balance 0.84. Inspection of the correlations among the quality, originality, and elegance ratings produced the expected (Scott et al., 2005) pattern of positive correlations with quality and originality being correlated 0.78, quality and elegance being correlated 0.66, and originality and elegance being correlated 0.58. This pattern of correlations points to the construct validity of the creativity ratings.

A similar pattern of findings emerged for the business performance ratings with all ratings displaying moderate to strong positive relationships. Again, however, the obtained pattern of correlations was substantively meaningful. Thus, feasibility ratings were strongly related to efficiency (r=0.67) and cost effectiveness (r=0.60) but proved to be less strongly related to reputation maintenance (r=0.49) and balance (r=0.46). Reputation maintenance, however, was found to be more strongly related to balance (r=0.56). Thus evidence is available for the construct validity of the business performance ratings as well as the creativity ratings. The correlations between the creativity and business performance variables and reliabilities can be found in Table 1.

3.6. Manipulations

3.6.1. Strategy training The first manipulation intended to provide strategies for working with performance information, social information, or both

performance and social information, was based on training interventions which occurred prior to participants beginning work on

Feasibility

1) Poor rating: The elements of the solution are unrealistic. It would be difficult for any organization to successfully implement the plan.

Ex: “Make magazine ads that can track how long someone looks at it so they can tell if the ad is working.”

2) Poor to average rating.

3) Average rating: The solution has some elements that are realistic. Some organizations would be able to implement the solution or specific elements of the solution.

Ex: “The can or container in which it comes should be distinct from all others in size and shape. Make it available at all restaurants, movies, and convenience stores.”

4) Average to excellent rating.

5) Excellent rating: The planned solution has most or all elements that are realistic. The elements of the solution could definitely be implemented by the specific organization in the case and most other organizations as well.

Ex: “Create a website for advertising that has coupons and interactive stuff like games that kids could play. Do limited advertising of the site and rely on word of mouth to spread. Put the website on the side of the packaging.”

Solution Efficiency

1) Poor rating: The plan requires more effort than it would return in terms of benefit for the company. The plan clearly has very few benefits versus the amount of time and energy required.

Ex: “Redesign the car and start making a new one.” “Have people go around neighborhoods with samples people can try for free.”

2) Poor to average rating:

3) Average rating: The plan clearly benefits the company, although it would be fairly time and energy intensive to implement. The benefits outweigh effort, but only by a small amount.

Ex: “People are obsessed with “natural” and “organic” products so change the colors of the packaging to seem more eco-friendly and health-conscious. They’d need to research what kinds of packaging would have this effect before doing it.”

4) Average to excellent rating

5) Excellent rating: The plan requires very little time or energy to implement while providing clear substantial benefits for the company. The benefits greatly outweigh effort.

Ex: “Form “street teams” of volunteers and offer prizes or rewards for doing the most or best advertising. These teams would basically make up their own advertising and would be essentially unpaid labor/advertising for the company that could be motivated with a few t-shirts.”

Fig. 4. Example business performance rating scales.

718 W.B. Vessey et al. / The Leadership Quarterly 22 (2011) 710–728

the experimental task. This training manipulation was intended to provide participants with strategies for working with performance and/or social information in solving leadership problems using a self-paced instructional format developed by Marcy and Mumford (2007, 2010).

This self-paced instruction consisted of three modules for each strategy being trained. In the first module the strategy was described, the ways in which effective execution of this strategy might influence problem-solving were noted, and an illustration of strategy application in solving a real-world problem was provided. The description of each strategy consists of a short definition of the strategy as well as alternate ways of framing or defining the specific strategy. The influence of the strategy on problem solving is described using a list of ways the problem and solution may be influenced by correctly and incorrectly applying the strategy to the problem at hand. The final portion of the first module consists of an example problem in which the application of the strategy might affect problem solving and an example of one way to successfully apply the strategy to the problem at hand.

In the second module participants were presented with a short, one paragraph, problem where application of the strategy would contribute to performance. These problems were designed to focus on problems students might regularly face in an effort to properly contextualize the use of these strategies. Each problem is followed by a request for a solution to the overall problem presented. Participants were asked to answer a series of three questions bearing on application of this strategy vis-à-vis the problem at hand. These questions are given in a multiple choice format, with three choices per question. The first question focuses on identification of the parts of the problem that may be affected by strategy application. The second question focuses on potential

Table 1 Correlations and reliabilities among dependent variable components.

1 2 3 4 5 6 7 8

1. Quality (0.85) 2. Originality 0.78 ⁎⁎ (0.93) 3. Elegance 0.66 ⁎⁎ 0.58 ⁎⁎ (0.84) 4. Feasibility 0.59 ⁎⁎ −0.22 ⁎⁎ 0.37 ⁎⁎ (0.93) 5. Efficiency 0.37 ⁎⁎ −0.12 0.72 ⁎⁎ 0.67 ⁎⁎ (0.81) 6. Cost effectiveness 0.20 ⁎ 0.14 0.12 0.74 ⁎⁎ 0.60 ⁎⁎ (0.80) 7. Reputation maintenance 0.34 ⁎⁎ 0.06 0.07 0.09 −0.06 0.56 ⁎⁎ (0.76) 8. Balance 0.42 ⁎⁎ 0.13 0.10 0.15 ⁎ −0.12 0.11 0.04 (0.84)

Note: ICCs appear on the diagonals. ⁎ Pb0.05. ⁎⁎ pb0.01.

CAUSES

Does this cause affect multiple aspects of the problem?

When thinking about causes in a problem, it is important to think about whether a cause affects multiple aspects of the problem. For example, will a change in this cause affect different parts of the problem? Or will changing this cause only have one effect?

Some other examples of this approach might be:

• Which cause changes the most aspects of the problem? • What cause changes the least aspects of the problem?

There are a number of reasons why it is important to think about whether changes in a cause affect multiple aspects of a problem. Here are just a few:

• Some causes may only affect one aspect of a problem. • Some causes may impact all aspects of a problem. • Focusing on causes that impact the most aspects of a problem allows for the generation of

solutions that achieve the greatest effect with the least amount of effort.

For example, Donna works in sales. She is responsible for retaining customers and notices that gradually the company’s customers have been leaving. She suspects that possible causes of this are that customers often have difficulty contacting customer service in addition to any other individual problems they may have.

Applying the strategy, Donna looks at each of the Causes in the problem, and asks herself, “Does this cause affect multiple aspects of the problem?" Donna realizes that if customers are able to reach customer service more easily, any of their other individual problems can be handled by customer service representatives. Because of the impact on any other problems that may arise, Donna focuses on improving customer service at her company.

Fig. 5. Illustrative material for first instructional module.

719W.B. Vessey et al. / The Leadership Quarterly 22 (2011) 710–728

outcomes of strategy application. The third and final question asks participants which solution they would choose from a set of three possible solutions. After completing these questions participants were provided with feedback concerning their answers. This feedback noted the correct answer to each question and provided justification as to why this answer was correct, or in the case of multiple correct answers why each might be considered correct.

In the third module, participants were again presented with short, one paragraph, real-world problems where application of this strategy would contribute to problem-solving. These problems were presented in the same format as those in module two. Participants were then asked to provide a written one paragraph answer to this problem using the strategy being trained. These responses were then reviewed by a researcher present in the room and brief feedback was given on application of the strategies to the problems. It is of note all instructional content was based on problems that might be encountered in day-to-day life by undergraduates to maximize relevance and minimize contamination of the experimental task vis-à-vis instructional materials. Figs. 5–7 illustrate the training material provided for one strategy — does the cause affect multiple aspects of the problem.

The model of leader cognition proposed by Mumford, Friedrich et al. (2007) holds that processing performance information should consider causes, resources, restrictions, and contingencies. Accordingly, participants were given instruction in two strategies bearing on each information element. More specifically, training content examined — for causes, 1) working with causes that have direct effects and 2) working with causes that have multiple effects; for contingencies, 3) can this contingency be manipulated and 4) are multiple contingencies affected by the same factors; for resources, 5) how critical is this resource for problem-solving and 6) how large of an effect does this resource have; for restrictions, 7) can the effect of this restriction be minimized 8) is the effect of this restriction contingent on other restrictions.

Mumford, Friedrich et al. (2007) model stresses four types of social information used in leader problem-solving. Specifically, information bearing on actors, affect, goals, and social systems is held to be used by leaders. Accordingly, the two strategies trained for working with each type of information examined — for actors, 1) how do actors affect other groups and 2) how large of an effect does an actor have on problem resolution; for affect, 3) how does affect effect problem solutions and 4) how could a change in affect influence problem solutions; for goals, 5) how do actions on one goal affect other goals and 6) how critical is the goal; and for social systems, 7) can the social systems be changed to improve problem-solving and 8) how closely related are system operations to problem-solving.

In the performance information condition participants were provided with training in this information type's 8 strategies. In the social information condition participants were provided with training in the 8 social strategies. In the combined trainings condition, participants were first provided with training in the 8 performance strategies and then the 8 social strategies. All training interventions occurred in a fixed sequence of causes, resources, contingencies, and restrictions and/or actors, affect, goals, and social systems. Pilot testing showed no significant effects of the ordering of training interventions on performance. This

Imagine the following problem: Andrea is trying to get out the monthly newsletter she publishes. She hasn’t received any of the articles her different writers contribute. The writers all work on a volunteer basis because of their interest in the newsletter’s topic. The writers normally email her their work about a week before the newsletter goes out so she has time to edit. In the last couple of months the newsletter has come out late for a variety of reasons meaning that some assignments have been sent out late to writers. Andrea needs to get the newsletter out but is unsure what to do with no content. How would you solve Andrea’s problem? Make sure to use the training you have received in this module when considering your options.

1. What is/are the key cause(s) in this scenario? (Circle all that apply)

• The workers are volunteers. • Andrea is inconsistent with when the newsletter comes out. • The writers have to email their work to her.

2. Which causes could affect multiple aspects of the problem? (Circle one)

• The volunteer nature of the work. • Andrea’s inconsistent deadlines. • The email system.

3. Which solution should you choose to apply in this situation? (Circle one)

• Start sticking to a set deadline every month and put out the newsletter regardless of who has submitted their articles.

• Begin paying the workers in hopes that it motivates them to get their work in on time.

• Talk with the writers and find out why they are not submitting their articles on time.

Imagine the following problem: Andrea is trying to get out the monthly newsletter she publishes. She hasn’t received any of the articles her different writers contribute. The writers all work on a volunteer basis because of their interest in the newsletter’s topic. The writers normally email her their work about a week before the newsletter goes out so she has time to edit. In the last couple of months the newsletter has come out late for a variety of reasons meaning that some assignments have been sent out late to writers. Andrea needs to get the newsletter out but is unsure what to do with no content. How would you solve Andrea’s problem? Make sure to use the training you have received in this module when considering your options.

1. What is/are the key cause(s) in this scenario? • The workers are volunteers. (This likely effects their motivation.) • Andrea is inconsistent with when the newsletter comes out. (They may

not know when their deadline is.)

2. Which causes could affect multiple aspects of the problem? • Andrea’s inconsistent deadlines. (By putting out the newsletter

inconsistently the writers may not know when to submit their articles and may not have motivation to submit them on time.)

3. Which solution should you choose to apply in this situation? (Circle one)

• Talk with the writers and find out why they are not submitting their articles on time. (This lets Andrea find out what the real cause of the problem is since there may be multiple causes.)

Fig. 6. Illustrative material second instructional module.

720 W.B. Vessey et al. / The Leadership Quarterly 22 (2011) 710–728

particular ordering was selected using a random number generator to determine ordering within training conditions and for the order in which training would occur in the combined training. The decision was made to pair like strategies (e.g., the two strategies focusing on causes) based on the increased amount of time required by participants in pilot testing when strategy pairs were split. Again, no significant effects on performance were found between paired and unpaired strategies.

3.6.2. Control The second manipulation occurred through the instructions given to participants after they had read through the description of

the marketing firm but before they began work on the three problems. In the high control condition, participants were instructed that they could control finances for the firm, distribution of the marketing material, scheduling, and personnel but that they could not control the product or the price of the product. In the low control condition, participants were instructed they could control finances for the firm and distribution of marketing material but they could not control scheduling, personnel, product, and price. Control items were developed based on key aspects of marketing problems identified in the marketing literature (Boyett & Boyett, 2003; Buskirk, 1961; Meldrum & McDonald, 2007). These items were then presented to participants in a short bullet point list identifying which aspects of the problem were controllable by the participant and which were not.

The specific items chosen for each condition were based on the ability of participants to solve the problems. In pilot testing it was found that participants had a difficult time generating solutions to the marketing problems without control of finances and distribution of the marketing materials, resulting in frustration toward the experimental task. Thus, these aspects of the problem were included in both the high and low control conditions to ensure participants could complete the task. Similarly, product and price were chosen as uncontrollable aspects of the problem in both conditions due to their effect on problem-solving. In pilot testing, when participants were allowed to control product and price it was found that they often changed the product and price in unrealistic ways based on their position (e.g., having a car manufacturer become a computer manufacturer), essentially ignoring the problem presented.

3.6.3. Framing The framing manipulation was intended to manipulate whether each problem participants were asked to solve was presented

in concrete terms or in abstract terms. In the concrete condition the abstracts derived from the case material were presented in describing each marketing problem. This material was illustrated in Fig. 1. In the abstract condition, the same content was

Sean is working on a school project for his high school physics class. He needs to make an enclosure for an egg that can be dropped from 20 feet and allow the egg to survive the fall. The enclosure can only weigh 2 pounds but Sean’s is consistently over that. Every time he removes a piece to try to make weight the enclosure stops functioning properly. Currently he’s using straws for most of his pieces but also has some spare parts available. He’s afraid that he doesn’t have time to redesign his project before the due date. How would you solve Sean’s problem? Make sure to use the training you have received in this module when considering your options.

Fig. 7. Illustrative material for the third module of instruction.

721W.B. Vessey et al. / The Leadership Quarterly 22 (2011) 710–728

presented, however the content was presented as key concepts abstracted from the case email. Fig. 8 illustrates the nature of the material presented in the abstract condition. In this regard it should be noted that all key concepts presented in the concrete condition appear and they are presented in the same order. Thus the concrete and abstract conditions were equated with respect to relevant information but not style of information presentation — concrete versus abstract.

From: Erin Ship Sent: Wed 4:17 PM To: [email protected] Subject: Puma Account Boss, Our team is working on the recent contract we’ve acquired working on the Puma sports car marketing for Flight Autos and needs some assistance. Flight is launching an advertising campaign to promote their new inexpensive sports car, the Puma. As usual we’ve conducted an analysis of the key factors in this case.

Client Background

• Many employees • Scarce resources • Manufacturers automobiles • Control of market segment (sports cars) • More resources from control of market segment

Market and Competitors

• Customers consider multiple factors • Product features and price vary • Competing in high performance sports cars • Product produced in small numbers • Requires major investment for the buyer

Product

• Product called the Puma • High-performance sports car • Target market is middle and upper-middle class • Features comparable to higher priced competitors • Better features than competitors at same price • Early reviews positive • Costs little to manufacture compared to competition • Lower price from production costs

Current Campaign

• Magazine advertisements focus on technical aspects • Television ads focus on comparisons with competitors • Magazine published focusing on product • Car used in professional competitions

Problem

• Need ways to track the effectiveness of advertising • Product sold at over 3000 locations in United States • Difficult to track effect of advertising on buyers • Limited time to secure market position • Manufacturing process can be copied • Need to have ability to quickly change advertising

Previous Attempts at Solving

• Previously gave rebates if customers bring in advertisement • Tracked the number of people responding to contests

We need some ideas for how to track the effect of our advertising. Anything you can come up with would be helpful for the team and possibly for Flight Autos as a company. Thanks for the help, Erin

Fig. 8. Illustration of problem presentation in abstract condition.

722 W.B. Vessey et al. / The Leadership Quarterly 22 (2011) 710–728

3.7. Analyses

The effects of the training, control, and framing manipulations on leader problem-solving were assessed in two multivariate analyses of covariance. Based on the sizeable positive correlations observed among the creativity ratings, which can be seen in Table 1, the dependent variables examined in the first multivariate analysis of covariance were the ratings of quality, originality, and elegance averaged across raters for the three problems presented. Again, given sizeable positive correlations observed among the business performance dimensions, the dependent variables examined in the second multivariate analysis of covariance were the ratings of feasibility, efficiency, cost effectiveness, reputation maintenance, and balance averaged across raters. A covariate was retained in these analyses only if it proved significant at the 0.05 level in initial analyses.

4. Results

A multivariate analysis of covariance was conducted examining the overall performance, in terms of creativity and business quality, of study participants on solving three leadership problems. As expected due to the nature of the problem-solving task, intelligence (F(3, 151)=14.58, p≤0.001) had a significant positive relationship with overall performance. It was also found that divergent thinking (F(3, 151)=3.02, p≤0.05) was positively related to overall performance on the problem-solving task. Finally, gender (F(3, 151)=6.37, p≤0.01) had a significant relationship with overall performance, with men performing better than women on the problem-solving task.

With regard to our manipulations, one significant interaction was found. A significant (F(3, 151)=4.47, p≤0.05) interaction was found between the training and leader control manipulations in terms of overall performance. Inspection of cell means indicated that overall performance was highest (M=2.976, SD=0.072) when training was given on performance strategies and the problem was framed in abstract terms. Though they must be interpreted in terms of the significant interaction found, we also found main effects for both the training and the leader control conditions. The training condition produced a significant (F(3, 151)=4.52, p≤0.05) main effect on overall performance, with inspection of cell means indicating that overall performance on the problem-solving task was highest (M=2.948, SD=0.49) when participants were trained on performance strategies as opposed to social strategies (M=2.804, SD=0.47) or both sets of strategies (M=2.790, SD=0.48). The leader control condition also produced a significant (F(3, 151)=3.74, p≤0.05) main effect on overall performance, with inspection of cell means indicating that overall performance was highest when participants were given a high level of control (M=2.914, SD=0.40) as opposed to a low level of control (M=2.781, SD=0.41).

4.1. Creativity

Table 2 presents the results obtained in the multivariate analysis of covariance examining the creativity of solutions provided to the three leadership problems. Consistent with the nature of the task, intelligence (F(3, 151)=11.30, p≤0.001) produced a positive relationship with creativity. It was also found that extraversion (F(3, 151)=3.30, p≤0.05) was negatively related to the production of more creative problem solutions. Finally, it was found that gender (F(3, 151)=3.90, p≤0.01) produced a significant relationship as a covariate, with men producing more creative solutions in these leadership problems than women.

A significant (F(3, 151)=4.84, p≤0.01) interaction was obtained between the training and leader control manipulations in terms of solution creativity. Inspection of the cell means indicated that when control was high, training performance strategies or both performance and social strategies resulted in production of higher quality (M=3.0, SD=0.108), more original (M=2.94, SD=0.096), and more elegant (M=2.78, SD=0.088) solutions to these leadership problems in contrast to the quality (M=2.84, SD=0.099), originality (M=2.85, SD=0.087), and elegance (M=2.42, SD=0.081) of solutions obtained when only social strategies were trained. In contrast, when control was low, training social strategies resulted in better performance with respect to solution quality (M=2.98, SD=0.106 versus M=2.89, SD=0.099), solution originality (M=3.01, SD=0.093 versus M=2.85,

Table 2 Multivariate analysis of covariance results for creativity of problem solutions.

F df p η2

Covariates Intelligence 11.30 151 0.000 0.181 Extraversion 3.30 151 0.022 0.061 Gender 3.90 151 0.010 0.071

Main effects Training condition 2.75 151 0.045 0.051 Control condition 1.96 151 0.122 0.037 Framing condition 0.98 151 0.402 0.019

Interactions Training*Control 4.84 151 0.003 0.086 Training*Framing 1.80 151 0.149 0.034 Control*Framing 1.03 151 0.379 0.020 Training*Control*Framing 2.67 151 0.049 0.048

Note: F = F-ratio, df = degrees of freedom, p = significance level using Roy's largest root, and η2 = eta-squared effect size.

723W.B. Vessey et al. / The Leadership Quarterly 22 (2011) 710–728

SD=0.087), and solution elegance (M=2.57, SD=0.086 versus M=2.42, SD=0.081) than when control was high. Thus social strategies appear useful to leaders when control is low while performance strategies are more useful when control is high.

A significant (F(3, 151)=2.67, p≤0.05) three-way interaction was also obtained between the training, control, and framing manipulations. The cell means indicated that when control was low, the problem was framed in concrete terms, and both performance and social strategies were trained, particularly poor solutions were obtained with respect to quality (M=2.69, SD=0.149 versus M=2.94, SD=0.148), originality (M=2.71, SD=0.149 versus M=2.91, SD=0.132) and elegance (M=2.42, SD=0.118 versus M=2.58, SD=0.122) as opposed to all other conditions. Thus low control, when working with multiple strategies and complex material, tends to undermine leaders' creative problem-solving.

Though found in the presence of significant interactions, we did find that the strategy training provided to study participants produced a significant (F(3, 151)=2.75, p≤0.05) main effect with respect to the creativity of the solutions resulting from these leadership problems. Inspection of the cell means indicated that solution quality was generally higher when performance (M=2.97, SD=0.078) or social (M=2.93, SD=0.073) strategies were provided than when both performance and social strategies (M=2.87, SD=0.075) were provided. Similarly, solution originality was higher when performance (M=2.92, SD=0.069) or social (M=2.93, SD=0.064) strategies were provided than when both performance and social strategies (M=2.86, SD=0.066) were provided. In contrast, solution elegance was better when only performance strategies (M=2.73, SD=0.064) were provided as opposed to social strategies (M=2.50, SD=0.059) or both performance and social strategies (M=2.51, SD=0.061).

4.2. Business performance

Table 3 presents the results obtained in the multivariate covariance conducted to examine the business performance ratings. As may be seen, intelligence (F(5, 154)=3.87, p≤0.01) proved to be a significant covariate producing a positive relationship with business performance. It was also found that negative affect (F(5, 154)=2.50, p≤0.05) was a significant covariate with negative affect proving to be positively related to the viability of the solutions provided with respect to business performance criteria. Finally, gender (F(5,154)=2.43, p≤0.05) also proved to be a significant covariate, with women producing better solutions than men.

A significant (F(5, 155)=2.90, p≤0.05) interaction was obtained between the training and leader control manipulations with respect to business performance. Inspection of the cell means indicated that the best solutions with respect to feasibility (M=3.35, SD=0.085 versus M=3.04, SD=0.079), efficiency (M=2.99, SD=0.051 versus M=2.82, SD=0.080), cost effectiveness (M=2.98, SD=0.071 versus M=2.78, SD=0.066), reputation maintenance (M=3.11, SD=0.096 versus M=2.87, SD=0.084), and balance (M=2.68, SD=0.10 versus M=2.46, SD=0.094) occurred when either performance training or both performance and social training were given, as opposed to social training, under conditions of high control. When control was low, however, all three types of training, performance (M=2.88, SD=0.088), social (M=2.82, SD=0.084), and both performance and social (M=2.72, SD=0.084) appeared equally effective and less effective than performance strategy training when control was high with respect to all five business criteria.

The final significant interaction (F(5, 155)=3.93, p≤0.01) to emerge in this analysis involved the training and framing manipulations. Inspection of the cell means indicated that these effects emerged on the cost effectiveness, reputation maintenance, and balance criteria. On these dependent variables, it was found that training in performance strategies, coupled with an abstract framing of the problem, resulted in better performance with respect to cost effectiveness (M=3.03, SD=0.075 versus M=2.81, SD=0.069), reputation maintenance (M=3.22, SD=0.094 versus M=2.90, SD=0.088), and balance (M=2.88, SD=0.105 versus M=2.48, SD=0.098) than all other conditions. Thus under crisis conditions, training in performance where leaders think about the problem in abstract terms appears to benefit some aspects of business performance.

Additionally, a significant main effect was obtained for the training (F(5, 155)=2.50, p≤0.05) manipulation. Examination of the cell means indicated that better performance was generally observed when training was given focusing on performance

Table 3 Multivariate analysis of covariance results for business effectiveness of problem solutions.

F df p η2

Covariates Intelligence 3.87 154 0.002 0.112 Negative affect 2.50 154 0.033 0.075 Gender 2.43 154 0.038 0.073

Main effects Training condition 2.50 155 0.033 0.075 Control condition 1.71 154 0.135 0.053 Framing condition 1.77 154 0.121 0.054

Interactions Training*Control 2.90 155 0.016 0.086 Training*Framing 3.93 155 0.002 0.112 Control*Framing 1.13 154 0.348 0.035 Training*Control*Framing 1.55 155 0.177 0.048

Note: F = F-ratio, df = degrees of freedom, p = significance level using Roy's largest root, and η2 = eta-squared effect size.

724 W.B. Vessey et al. / The Leadership Quarterly 22 (2011) 710–728

strategies than social strategies or both performance and social strategies with respect to feasibility (M=3.23, SD=0.061 versus M=3.05, SD=0.058), efficiency (M=2.96, SD=0.058 versus M=2.83, SD=0.056), cost effectiveness (M=2.96, SD=0.051 versus M=2.80, SD=0.049), reputation maintenance (M=3.11, SD=0.064 versus M=2.91, SD=0.062), and balance (M=2.66, SD=0.072 versus M=2.49, SD=0.070). Thus training in strategies for working with performance information embedded in knowledge, causes, resources, restrictions, and contingencies, appears critical for leaders if they are to shape better business solutions under crisis conditions. However this result must be interpreted in light of the significant interactions obtained.

5. Discussion

Before turning to the broader conclusions flowing from the present study certain limitations should be noted. To begin, although the present effort used a low-fidelity simulation task where participants were asked to solve multiple leadership problems, it should be recognized that both the experimental task and the problems presented were drawn from one domain — marketing. As a result, the question arises as to whether our findings can be extended to other performance domains in which leaders work. Moreover, the nature of the sample employed in this study, undergraduates, raises the question as to whether similar findings would be obtained in a more experienced population of leaders.

Along similar lines, it should also be recognized that the task presented to leaders was a series of “paper-and-pencil” problems arising in the context of a marketing firm. Though leadership is often defined as the exercise of influence over others (Yukl, 2009) the specific skills and processes that impact successful leadership can be evaluated without direct social interaction between a leader and subordinates (Sackett & Lievens, 2008). Low-fidelity paper and pencil assessments of leader performance have consistently shown high validity as assessments of leader performance (Motowidlo et al., 1990; Mumford, Zaccaro, Connelly, & Marks, 2000; Zaccaro, Mumford, Connelly, Marks, & Gilbert, 2000). Based on the generally high validity of this low-fidelity simulation methodology for assessing leader performance and the general acceptance in the field regarding its use in assessing leadership skills (Hazy, 2006; Mumford, Zaccaro, Connelly, & Marks, 2000) we felt this methodology was appropriate as an indicator of leader skills in the study at hand.

This point is of some additional importance because “paper-and-pencil” problems may lead towards a tendency to rely on abstract analytic strategies, our performance strategies, as opposed to more social strategies. However we would argue that the specific types of problems presented in this study as compared to others in which social information may play more of a direct role in problem solving (e.g., Barrett et al., 2011-this issue) is one of the strengths of the study. A number of authors have argued that the effects of situational variables on leader performance arethe area in need of increased study within the realm of leadership (Hunter, Bedell-Avers, & Mumford, 2007). While the specific task at hand may in fact not call for the use of social information to the degree that other tasks do, research showing that similar training may, in fact, have different outcomes for leader performance based on the problem domain is critical to the development and study of leadership training going forward. By acknowledging that not all problems or situations call for the same type of training we may be able to tailor training to be more effective based on the particular types of problems faced by a leader. For example, a leader in an engineering R&D group may employ much more performance information as compared to social information, while a high school principal may utilize much more social information due to the nature of the work. The question remains, though, as to whether similar findings would be observed if the present study had presented this task in a more distinctly social context and this is an area in which further research is called for.

It should also be noted that the manipulations presented in the present study were presented in a fixed order. Thus training manipulations occurred prior to presentation of the experimental task, leader control manipulations occurred just prior to participants starting work on the problems, and the framing manipulations occurred as people began work on the problems. This fixed ordering of experimental manipulations was necessary to maintain the realism of the experimental task. By the same token, however, the findings obtained in the present study have little to say about the potential effects arising from alternative orderings of the manipulations.

An additional limitation we must note from a statistical standpoint is that though a number of statistically significant effects were found, the effect sizes were generally small to moderate. For example, though we found a significant interaction between the training and control conditions in terms of creativity (p≤0.01) the effect size was relatively small (η2=0.086). These smaller effects sizes indicate a potential need for future research and must be born in mind when interpreting the results of this study.

A final potential limitation that should be borne in mind is that problem solutions were obtained under crisis conditions. In the case of the present study, a sense of crisis was involved on all problems through anticipation of potential negative consequences of failure. Prior research by Barrett et al. (2011-this issue) indicates that articulation of potential negative outcomes does induce a sense of crisis without inhibiting leader cognition (Fiedler & Garcia, 1987). Thus induction of a sense of crisis was not inappropriate, especially given prior studies (Hunt et al., 1999; Mumford, 2006) indicating that leader performance is especially significant under crisis conditions. By the same token caution is called for in generalizing the findings obtained in the present effort to non-crisis conditions.

Even bearing these limitations in mind, we believe that the results obtained in the present study have some important implications for understanding leader performance and leader cognition. Mumford, Friedrich et al. (2007) have argued that leaders, in formulating problem solutions, may focus on working with either performance information (causes, resources, restrictions, and contingencies) or social information (actors, affect, goals, and social systems). The findings obtained in the present study provide some support for this model by showing that training in strategies for working with performance information and social information affect leader performance differently under different leadership conditions. Thus this study, in keeping with the findings of Marcy and Mumford (2010) and Scott, Lonergan, and Mumford (2004), suggests that by providing

725W.B. Vessey et al. / The Leadership Quarterly 22 (2011) 710–728

leaders with better strategies tailored to working with their knowledge under specific conditions we may do much to enhance leader cognition and problem-solving, especially when leaders must solve problems under crisis conditions.

Before moving on to discussion of the findings in relation to our hypotheses we first need to address the differences in the results of this study as compared to another recent study of leader cognition in crisis (Barrett et al., 2011-this issue). While this study found that a focus on performance strategies resulted in better overall problem solutions as compared to a focus on social strategies, the study by Barrett et al. (2011-this issue) found that social strategies provided the greatest benefit for leader problem solving. Though there are a number of potential explanations for these findings, the difference in tasks between the two studies and the effects of these tasks on the sample populations of undergraduates may provide the best explanation. The study by Barrett et al. (2011-this issue) had participants work through a low-fidelity leadership task involving the administration of a school, with participants placed in the role of a principal. Undergraduate students are likely to be much more familiar with schools and the educational system than marketing firms, resulting in higher emotional investment in the task. Moreover, the task of running a school in itself may involve the need for more social awareness than the marketing task presented in this study. We would argue that our findings apply to more emotionally neutral tasks while the findings of the Barrett et al. (2011-this issue) study may apply to tasks in which a leader is more emotionally involved. When a leader is more emotionally invested in a task they may be more likely to miss important social information, leading to improvements in performance when trained on social strategies for problem-solving. In a more emotionally neutral task the leader may need less assistance in perceiving social information objectively, and would instead benefit from an increased focus on performance information.

The issue that arises at this juncture, however, is what kind of strategies we should be attempting to develop in leaders to help them resolve crises (Drazin et al., 1999). Two considerations led to our hypothesis that leader problem-solving under crisis conditions would improve more if developmental interventions focused on performance information strategies as opposed to social information strategies. First, people are not reliable under crises (Weick, 1995). Second, when confronted with crises leaders may prefer to employ a pragmatic problem-solving strategy (Mumford & Van Doorn, 2001) which stresses performance information. With regard to the business criteria our findings clearly support this hypothesis, with our findings indicating that the best problem solutions emerged when leaders were given training in strategies for working with performance information across all conditions.

In this regard, however, the findings obtained with regard to creativity point to an important caveat on this general conclusion. Training social information processing strategies could result in more creative problem solutions being produced by leaders as evidenced in evaluations of solution quality, originality, and elegance. Moreover, the interaction of training strategies with control indicated that when control was low, social information processing strategies were likely to result in problem solutions of higher creativity. Thus leaders may use social forces to compensate for a lack of control, resulting in training social information processing strategies proving beneficial to generation of creative solutions, at least under conditions where leaders lack control. As a result, there might, under conditions where leader control will be limited, prove to be some value in training social information processing strategies.

Our second hypothesis held that when leaders had more control, information processing would be focused on controllable variables resulting in the production of better problem solutions (Thomas & McDaniel, 1990). In fact, control did not produce significant main effects with respect to the creativity and business performance ratings. These effects were consistent and in the expected direction, indicating that both leader creativity and business performance may be better when leader control was high as opposed to low. Based on this consistency in the direction of the effect we would suggest further studies investigating the potential effects of leader control, and varying types of control, on leader performance in crises, as these findings suggest that maximizing leader autonomy and control in crisis situations may result in better performance (Yammarino, Mumford, Connelly, & Dionne, 2010).

However, the control by training type interaction suggests that control may as a phenomenon exert another pattern of effects. It was found that the business value of problem solutions and the creativity of problem solutions were particularly strong when leaders were provided with training in information strategies under conditions of high control. This pattern of findings might be accounted for on two bases. First, control of variables provides a set of conditions under which it is possible to apply performance information strategies. Second, control might increase information processing intensity. Although the findings obtained in the present study cannot allow us to disentangle these two alternative theoretical explanations, they do suggest that leader performance in resolving crisis problems might be improved when leaders are given training in performance strategies and then allowed to operate with few constraints being placed on them with regard to the approaches they use in resolving a crisis.

Our third hypothesis held that if leaders were encouraged to think more abstractly more creative and better business oriented problem solutions would result (Ward et al., 2004). In the present study abstractness was manipulated through problem framing — presenting the problems in concrete as opposed to abstract terms. The findings we obtained, at least with regard to the business criteria, suggest that at least some aspects of business outcomes improve when leaders are working with performance strategies and are thinking abstractly. This finding is of some importance because crisis situations often lead people to think concretely. However, the best leaders may be those who think about objective performance information in an abstract fashion despite the pressures placed on them by crises. This observation, in turn, suggests that training on environmental interventions that encourage leaders to think about cases in abstract terms when they are applying performance strategies may prove valuable (Mumford, Marks, Connelly, Zaccaro, & Reiter-Palmon, 2000).

More generally, the findings obtained in the present study paint a picture of how leaders think about crises under specific leadership conditions. Effective leaders will work through the problem using performance strategies, strategies for working with information bearing on causes, resources, restrictions, and contingencies, performing particularly well when they frame the

726 W.B. Vessey et al. / The Leadership Quarterly 22 (2011) 710–728

problem abstractly and have the latitude to act on the situation. This calls to mind that old saying of Joe Friday in the television show Dragnet — “Just the Facts.” Unfortunately many interventions intended to improve leader performance, often interventions used to improve leader performance in crises, stress social information processing at the expense of other types of processing that may be occurring (Avolio, Reichard, Hannah, Walumbwa, & Chan, 2009; Boyatzis, Bilimoria, Godwin, Hopkins, & Lingham, 2006; Eden et al., 2000). Hopefully, the present study will serve to remind us that it may be as, if not more, important to get leaders to think about the objective performance setting at hand as they attempt to resolve crises.

Acknowledgments

We would like to thank Tamara Friedrich, Jay Caughron, and Alison Antes for their contributions to the present effort.

References

Avolio, B., & Gardner, W. (2005). Authentic leadership development: Getting to the root of positive forms of leadership. The Leadership Quarterly, 16, 315–338. Avolio, B., Reichard, R., Hannah, S., Walumbwa, F., & Chan, A. (2009). A meta-analytic review of leadership impact research: Experimental and quasi-experimental

studies. The Leadership Quarterly, 20(5), 764–784. Barrett, J., Vessey, W. B., & Mumford, M. D. (2011). Getting leaders to think: Effects of training, threat, and pressure on performance. The Leadership Quarterly, 22,

729–750 (this issue). Bass, B. M. (1990). Handbook of leadership: A survey of theory and research. New York, NY: Free Press. Becker, T., Billings, R., Eveleth, D., & Gilbert, N. (1997). Validity of scores on three attachment style scales: Exploratory and confirmatory evidence. Educational and

Psychological Measurement, 57, 477–493. Bedell-Avers, K. E., Hunter, S. T., & Mumford, M. D. (2008). Conditions of problem-solving and the performance of charismatic, ideological, and pragmatic leaders: A

comparative experimental study. The Leadership Quarterly, 19, 89–106. Besemer, S., & O'Quin, K. (1999). Confirming the three-factor creative product analysis matrix model in an American sample. Creativity Research Journal, 12,

287–296. Boyatzis, R., Bilimoria, D., Godwin, L., Hopkins, M., & Lingham, T. (2006). Effective leadership in extreme crisis. 9/11: Mental health in the wake of terrorist attacks

(pp. 197–214). New York, NY US: Cambridge University Press. Boyett, J. H., & Boyett, J. T. (2003). The guru guide to marketing. Hoboken, NJ: Wiley & Sons. Buskirk, R. H. (1961). Principles of marketing. New York, NY: Holt, Rinehart, and Winston. Byrne, C. L., Shipman, A. S., & Mumford, M. D. (2010). The effects of forecasting on creative problem-solving: An experimental study. Creativity Research Journal, 22,

119–138. Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42, 116–131. Connelly, M. S., Gilbert, J. A., Zaccaro, S. J., Threlfall, K. V., Marks, M. A., & Mumford, M. D. (2000). Predicting organizational leadership: The impact of problem-

solving skills, social judgment skills, and knowledge. The Leadership Quarterly, 11, 65–86. Conway, J., & Peneno, G. (1999). Comparing structured interview question types: Construct validity and applicant reactions. Journal of Business and Psychology, 13,

485–506. Crawford, J. R., & Henry, J. D. (2004). The positive and negative affect schedule (PANAS): Construct validity, measurement properties and normative data in a large

non-clinical sample. The British Journal of Clinical Psychology, 43, 245–265. Crocker, P. R. E. (1997). A confirmatory factor analysis of the positive affect negative affect schedule with a youth sport sample. Journal of Sport & Exercise

Psychology, 19, 91–97. Day, D., Gronn, P., & Salas, E. (2006). Leadership in team-based organizations: On the threshold of a new era. The Leadership Quarterly, 17, 211–216. Drazin, R., Glynn, M., & Kazanjian, R. (1999). Multilevel theorizing about creativity in organizations: A sensemaking perspective. Academy of Management Review,

24, 286–307. Eden, D., Geller, D., Gewirtz, A., Gorden-Terner, R., Inbar, I., Liberman, M., et al. (2000). Implanting Pygmalion leadership style through workshop training: Seven

field experiments. The Leadership Quarterly, 11, 170–210. Fiedler, F., & Garcia, J. (1987). New approaches to effective leadership: Cognitive resources and organizational performance. Oxford England: John Wiley & Sons. Fleishman, E., & Harris, E. (1962). Patterns of leadership behavior related to employee grievances and turnover. Personnel Psychology, 15, 43–56. Friedrich, T., Vessey, W., Schuelke, M., Ruark, G., & Mumford, M. (2009). A framework for understanding collective leadership: The selective utilization of leader

and team expertise within networks. The Leadership Quarterly, 20, 933–958. Frisch, M., & Jessop, N. (1989). Improving WAIS—R estimates with the Shipley-Hartford and Wonderlic Personnel Tests: Need to control for reading ability.

Psychological Reports, 65, 923–928. Goldberg, L. R. (1990). An alternative “description of personality”: The Big-Five factor structure. Journal of Personality and Social Psychology, 59, 1216–1229. Hawkins, K., Faraone, S., Pepple, J., Seidman, L., & Tsuang, M. (1990). WAIS—R validation of the Wonderlic Personnel Test as a brief intelligence measure in a

psychiatric sample. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 2, 198–201. Hedlund, J., Forsynthe, G. B., Horvath, J. A., Williams, W. M., Snoot, S., & Sternberg, R. J. (2003). Identifying and assessing tacit knowledge: Understanding the

practical intelligence of military leaders. The Leadership Quarterly, 14, 117–140. Henderson, D., Liden, R., Glibkowski, B., & Chaudhry, A. (2009). LMX differentiation: A multilevel review and examination of its antecedents and outcomes. The

Leadership Quarterly, 20, 517–534. Hunt, J., Boal, K., & Dodge, G. (1999). The effects of visionary and crisis-responsive charisma on followers: An experimental examination of two kinds of charismatic

leadership. The Leadership Quarterly, 10, 423–448. Judge, T., Colbert, A., & Ilies, R. (2004). Intelligence and leadership: A quantitative review and test of theoretical propositions. The Journal of Applied Psychology, 89,

542–552. Kahneman, D., & Tversky, A. (1979). On the interpretation of intuitive probability: A reply to Jonathan Cohen. Cognition, 7, 409–411. Kercher, K. (1992). Assessing subjective well-being in the old–old. The PANAS as a measure of orthogonal dimensions of positive and negative affect. Research on

Aging, 14, 131–168. Koberg, C. S., Uhlenbruck, N., & Sarason, Y. (1996). Facilitators of organizational innovation: The role of life-cycle stage. Journal of Business Venturing, 11, 133–149. Kolodner, J. L. (1997). Educational implications of analogy: A view from case-based reasoning. The American Psychologist, 52, 57–66. Lonergan, D. C., Scott, G. M., & Mumford, M. D. (2004). Evaluative aspects of creative thought: Effects of idea appraisal and revision standards. Creativity Research

Journal, 16, 231–246. Lord, R., de Vader, C., & Alliger, G. (1986). A meta-analysis of the relation between personality traits and leadership perceptions: An application of validity

generalization procedures. The Journal of Applied Psychology, 71(3), 402–410. Lord, R. G., & Hall, R. J. (2005). Identity, deep structure, and the development of leadership skill. The Leadership Quarterly, 16, 591–615. Marcy, R., & Mumford, M. (2007). Social innovation: Enhancing creative performance through casual analysis. Creativity Research Journal, 19, 123–140. Marcy, R. A., & Mumford, M. D. (2010). Leader cognition: Improving leader performance through causal analysis. The Leadership Quarterly, 21, 1–19. Marta, S., Leritz, L. E., & Mumford, M. D. (2005). Leadership skills and the group performance: Situational demands, behavioral requirements, and planning. The

Leadership Quarterly, 16, 591–615.

727W.B. Vessey et al. / The Leadership Quarterly 22 (2011) 710–728

McClendon, R. (1996). Motivation and cognition of preservice teachers: MSLQ. Journal of Instructional Psychology, 23, 216–220. Meldrum, M., & McDonald, M. (2007). Marketing in a nutshell. Burlington, MA: Elsevier. Merrifield, P. R., Guilford, J. P., Christensen, P. R., & Frick, J. M. (1962). The role of intellectual factors in problem solving. Psychological Monographs, 76, 1–21. Motowidlo, S., Dunnette, M., & Carter, G. (1990). An alternative selection procedure: The low-fidelity simulation. The Journal of Applied Psychology, 75, 640–647. Mumford, M. D. (2006). Pathways to outstanding leadership: A comparative analysis of charismatic, ideological, and pragmatic leadership. Mahulah, NJ: Erlbaum. Mumford, T. V., Campion, M. A., & Morgeson, F. P. (2007). The leadership skills stratplex: Leadership skill requirements across organizational levels. The Leadership

Quarterly, 18, 154–166. Mumford, M. D., Friedrich, T. L., Caughron, J. J., & Antes, A. (2009). Leadership research: Traditions, developments and current directions. In D. A. Buchanan & A.

Bryman (Eds.), Handbook of Organizational Research Methods (pp. 111–127). Thousand Oaks, CA: Sage Publications Ltd. Mumford, M. D., Friedrich, T., Caughron, J., & Byrne, C. (2007). Leader cognition in real-world settings: How do leaders think about crises? The Leadership Quarterly,

18, 515–543. Mumford, M., Marks, M., Connelly, M., Zaccaro, S., & Reiter-Palmon, R. (2000). Development of leadership skills: Experience and timing. The Leadership Quarterly,

11, 87–114. Mumford, M., Schultz, R., & Osburn, H. (2002). Planning in organizations: Performance as a multi-level phenomenon. In F. Yammarino & F. Dansereau (Eds.), The

many faces of multi-level issues (pp. 3–65). US: Elsevier Science. Mumford, M., & Van Doorn, J. (2001). The leadership of pragmatism: Reconsidering Franklin in the age of charisma. The Leadership Quarterly, 12, 279–309. Mumford, M., Zaccaro, S., Harding, F., Jacobs, T., & Fleishman, E. (2000). Leadership skills for a changing world: Solving complex social problems. The Leadership

Quarterly, 11, 11–35. Nohria, N., & Gulati, R. (1996). Is slack good or bad for innovation? Academy of Management Journal, 39, 1245–1264. Patalano, A. L., & Siefert, C. M. (1997). Opportunistic planning: Being reminded of pending goals. Cognitive Psychology, 34, 1–36. Pintrich, P., Smith, D., Garcia, T., & McKeachie, W. (1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire. Educational and

Psychological Measurement, 53, 801–813. Reysen, S. (2005). Construction of a new scale: The Reysen likability scale. Social Behavior and Personality, 33, 201–208. Saucier, G. (2002). Orthogonal markers for orthogonal factors: The case of the Big Five. Journal of Research in Personality, 36, 1–31. Scott, G., Lonergan, D., & Mumford, M. (2005). Conceptual combination: alternative knowledge structures, alternative heuristics. Creativity Research Journal, 17,

79–98. Shamir, B., House, R., & Arthur, M. (1993). The motivational effects of charismatic leadership: A self-concept based theory. Organization Science, 4, 577–594. Shipman, A., Byrne, C. L., & Mumford, M. D. (2010). Leader vision and forecasting: The effects of forecasting extent, resources, and timeframe. The Leadership

Quarterly, 21, 439–456. Strange, J. M., & Mumford, M. D. (2005). The origins of vision: Effects of reflection, models, and analysis. The Leadership Quarterly, 16, 121–148. Swayne, L., & Ginter, P. (1993). Cases in strategic marketing. Englewood Cliffs, NJ: Prentice Hall. Thomas, J. B., & McDaniel, R. R. (1990). Interpreting strategic issues: Linkages among scanning, interpretation, action, and outcomes. Academy of Management

Journal, 34, 239–270. Vincent, A. H., Decker, B. D., & Mumford, M. D. (2002). Divergent thinking, intelligence, and expertise: A test of alternative models. Creativity Research Journal, 14,

163–178. Vroom, V., & Jago, A. (2007). The role of the situation in leadership. The American Psychologist, 62, 17–24. Ward, T., Patterson, M., & Sifonis, C. (2004). The role of specificity and abstraction in creative idea generation. Creativity Research Journal, 16, 1–9. Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scale. Journal of Personality

and Social Psychology, 54, 1063–1070. Weick, K. E. (1995). Sensemaking in organizations. Thousand Oaks, CA: Sage. Xiao, Y., Milgram, P., & Doyle, D. (1997). Capturing and modeling planning expertise in anesthesiology: Results of a field study. In C. Zsambok & G. Klein (Eds.),

Naturalistic decision making (pp. 197–205). Mawah, NJ: Lawrence Earlbaum. Yammarino, F. J., Mumford, M. D., Connelly, M. S., & Dionne, S. D. (2010). Leadership and team dynamics for dangerous military contexts. Military Psychology, 22,

15–41. Yukl, G. (2009). Leadership in organizations. Saddle River, NJ: Prentice Hall.

728 W.B. Vessey et al. / The Leadership Quarterly 22 (2011) 710–728