Synthesis Paper

Getting leaders to think: Effects of training, threat, and pressure on performance

Jamie D. Barrett, William B. Vessey, Michael D. Mumford ⁎ The University of Oklahoma, United States

a r t i c l e i n f o a b s t r a c t

It has been argued that leaders rely on case-based, or experiential, knowledge when attempting to solve organizational problems. To test this proposition, undergraduates were provided with instruction in strategies for working with cognitive (e.g. causes) or social (e.g. actors) knowledge in solving leadership problems. It was found that both forms of instruction contribute to better vision formation, better planning, and generation of more creative solutions to leadership problems. The effects of training were not moderated by time pressure. However, it was found that calling leaders' attention to threats could compensate for training. The implications of these findings for understanding leader cognition are discussed.

© 2011 Published by Elsevier Inc.

Keywords: Leadership Cognition Training Pressure Threat

1. Introduction

Few scholars would dispute the point that leaders must do many things (Yukl, 2010). Leaders must motivate followers (Harris, Wheeler & Kacmar, 2009). Leaders must structure the activities of group members (Marta, Leritz & Mumford, 2005), and leaders must represent the group (Briscoe et al., 2010). Implicit in the notion that leaders play many varied roles is an important, albeit often overlooked, proposition (Mumford, Connelly & Gaddis, 2003). More specifically, the demands made on those occupying leadership roles, and the multiple courses of action a leader might pursue in executing any given role (Jacobs & Jaques, 1990), implies that effective performance in leadership roles will depend on cognition (Mumford, Zaccaro, Harding, Jacobs & Fleishman, 2000) — thinking about the decision that must be made to influence others with respect to the attainment of certain goals (Lord & Hall, 2005).

Recognition of the fundamental importance of cognition to leader performance has led scholars to propose a number of models that might be used to account for leader cognition (Mumford, Friedrich, Caughron, & Byrne, 2007). For example, some scholars have sought to account for leader cognition in terms of basic abilities such as intelligence (Judge, Colbert & Ilies, 2004). Other scholars have sought to identify cognitive skills such as causal analysis (Marcy & Mumford, 2010), forecasting ((Shipman, Byrne, & Mumford, 2010), and wisdom (McKenna, Rooney & Boal, 2009) that might be particularly important to leader performance. Still other scholars have sought to identify the specific kinds of knowledge that contribute to leader performance (Hedlund et al., 2003), and still other scholars have sought to identify the conditions that make cognition either more or less important to performance in organizational leadership roles (Mumford, Campion & Morgeson, 2007b). Given the previous work in the area of leader cognition and the various models attempting to explain the processes involved, the present effort seeks to extend the current literature by investigating viable ways of influencing the effectiveness of leader cognition. Overall, the intended contributions of our work are to 1) determine which training strategies for working with case-based knowledge are more effective for leader problem-solving, 2) examine how conditions of stress affect leader problem-solving, and 3) determine if training strategies for working with case-based knowledge will impact leader problem solving under stressful conditions. In the sections that follow, we first discuss the processing activities and knowledge structures involved in leader cognition, then we discuss the impact that stress is held to have on cognitive resources and leader performance.

The Leadership Quarterly 22 (2011) 729–750

⁎ Corresponding author at: Department of Psychology, The University of Oklahoma, Norman, OK 73014, United States. E-mail address: [email protected] (M.D. Mumford).

1048-9843/$ – see front matter © 2011 Published by Elsevier Inc. doi:10.1016/j.leaqua.2011.05.012

Contents lists available at ScienceDirect

The Leadership Quarterly

journal homepage: www.elsevier.com/locate/leaqua

1.1. Cognition

Within Mumford et al.'s (2007a) model, three aspects of cognition are considered critical to leader problem-solving. More specifically, this model holds that leader problem solving depends on 1) processing activities, 2) knowledge, and 2) the strategies employed in working with knowledge during process execution (Mumford, Friedrich, Caughron & Antes, 2009). In fact, the nature of this model suggests that the processing activities engaged in by leaders are highly complex.

Leader problem-solving is held to begin with scanning of the internal and external environments vis-à-vis monitoring models (Koberg, Uhlenbruck & Sarason, 1996) to identify significant change events with respect to either threats or opportunities (Ford & Gioia, 2000). With identification of a potentially significant change event, information gathering will occur to define the nature and significance of the event (Rodan, 2002). The information gathering will activate, or, alternatively, lead to the construction of, mental models for understanding, or making sense of, the change event (Gioia & Thomas, 1996; Weick, 1995). Activation, or construction, of these mental models is important because it allows for identification of critical causes and salient goals (Marcy & Mumford, 2010; Thomas & McDaniel, 1990). Identification of critical causes and salient goals, in turn, provides a basis for searching extant knowledge structures to identify potential actions and actors.

With identification of relevant actions and actors, it becomes possible for leaders to formulate a prescriptive mental model to guide actions to be taken in resolving the problem, or change event, at hand. In fact, Strange & Mumford (2005) have provided evidence indicating that formation of prescriptive mental models is critical to leaders' subsequent articulation of a viable vision. Prescriptive mental models are important because they provide a basis for forecasting the effects of potential actions that might be taken to resolve the problem or problems at hand. And, in keeping with this argument, Shipman et al. (2010) have shown that the extent of leaders' forecasting is critical to their performance in both vision formation and problem-solving. These forecasts, in turn, with self-reflection and systems — reflection on forecasted outcomes, make it possible for leaders to construct plans and backup plans for the exercise of influence (Mumford, Schultz, & Osborn, 2002) with leaders executing these plans in an adaptive, opportunistic, fashion (Patalano & Siefert, 1997).

Although this model of the processing activities underlying leader cognition seems plausible given this findings obtained in studies by Hunter (2011), Marcy & Mumford (2010), Shipman et al. (2010) and Strange & Mumford (2005), a critical question has not yet been addressed. More specifically, on what forms of knowledge are these processing activities based? This question is of some importance because multiple forms of knowledge, schematic, associational, case-based, exist (Hunter, Bedell-Avers, Hunsicker, Mumford & Ligon, 2008) and the knowledge structures that provide a basis for process execution are critical determinants of the outcomes of leaders' problem-solving efforts.

Mumford et al. (2007a) argued that the knowledge employed by leaders in formulating influence attempts is case-based, or experiential, knowledge (Kolodner, 1997; Naidoo, Kohari, Lord, & Dubois, 2010). Thus Strange & Mumford (2005) in their study of vision formation found that the visions articulated by people in leadership roles were contingent on the cases being considered. Similarly, Isenberg (1986) and Nutt (1989) found that leaders' problem solutions were typically founded in case-based, or experiential, knowledge. Still other work, by Berger & Jordan (1992), indicated that case-based knowledge is commonly employed in formulating influence attempts.

Case-based, or experiential, knowledge is knowledge abstracted from past performance events (Kolodner, 1997). Case-based knowledge stems from declarative memory, which has two parts, semantic and episodic memory (Tulving, 1972). Case-based knowledge is essentially a form of episodic memory in that it uses episodic information and structures it around a specific episode, or case. Additionally case-based knowledge is usually complex, including information bearing on causes, resources, restrictions, contingencies, goals, actors, affect, and social systems (Hammond, 1990; Mumford et al., 2007a In fact, the complexity of case-based knowledge makes it difficult for people to work with multiple cases in problem-solving (Scott et al., 2005a). Cases are held to be stored in memory in a library system where cases are referenced against, and activated by, situational cues (Irby & Wilkinson, 2003). The cases recalled with activation are typical cases and major, or common exceptions, to these prototypic cases (Habermas & Daha, 2001). The activated cases, through analysis of relevant case elements, provide a basis for process execution and problem-solving (Scott, Lonergan & Mumford, 2005b). Mumford et al. (2007a) argued that two types of information stored in cases will prove particularly critical for leaders' problem-solving efforts, 1) information bearing on objective features of requisite actions — information pertaining to causes, resources, restrictions, and contingencies, and 2) information pertaining to the social context — information pertaining to actors, affect, goals, and the social system.

In working with the information and social aspects of case-based knowledge, people apply certain strategies during process execution where strategies reflect the procedures applied in working with knowledge (Mumford & Norris, 1999). In fact, effective execution of any given process during problem-solving appears to depend on selection and effective execution of an appropriate set of strategies for working with the knowledge at hand. Thus Baughman, Mumford & Sager (1997) have shown that application of feature search and feature mapping strategies contribute to effective execution of the conceptual combination process during creative problem solving. Similarly, Lonergan, Scott & Mumford (2004) have shown that application of a compensatory strategy during idea evaluation, for example seeking to improve the quality of original ideas, contributes to effective execution of the idea in the evaluation process.

These findings are noteworthy because they suggest that providing people with strategies for applying case-based knowledge in solving leadership problems would result in improved performance on the criteria commonly considered in studies of leader problem-solving — specifically solution creativity (Connelly et al., 2000), planning (Yukl, 2010), and vision formation (Strange & Mumford, 2005). Moreover, based on a study by Marcy & Mumford (2010), who found that training in causal analysis contributes to performance in solving leadership problems, it seems plausible to argue that training strategies contributing to effective

730 J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

utilization of informational attributes of case-based knowledge will contribute to leader problem-solving, planning, and vision formation. Along similar lines, given the importance of information bearing on the social context to leaders' exercise of influence (Bass, 1990), it also seems reasonable to expect that training in strategies for working with information bearing on the social context would also contribute to leader problem-solving, planning, and vision formation. Hence, our first two hypotheses:

Hypothesis one. Training in strategies held to contribute to effective use of informational attributes of case-based knowledge will result in improved performance with respect to creative problem-solving, planning, and vision formation on the part of the leaders.

Hypothesis two. Training in strategies held to contribute to effective use of social contextual attributes of case based knowledge will result in improved performance with respect to creative problem-solving, planning, and vision formation on the part of leaders.

1.2. Cognitive resources

Although it seems plausible to argue that by providing people with strategies, either informational or social strategies, for working with case-based knowledge leader cognitive performance will improve, this argument might be questioned based on the findings from studies of Cognitive Resource Theory (Fiedler & Garcia, 1987). Cognitive Resource Theory holds that cognitive capacities, specifically intelligence, will be related to leader performance only under certain conditions. More specifically, when conditions, such as stress, draw away cognitive resources, the relationship between cognition and leader performance is attenuated. Furthermore, encoding of experiences into episodic memory, through which case-based knowledge is organized, tends to be enhanced with emotion, specifically under conditions of stress. In fact, Fiedler & Garcia (1987) provided some initial support for this proposition in their research examining the effects of technical training. These studies involved military personnel who participated in a training program and then performed tasks under similar conditions. This study provided some evidence of the relationship between training and leadership performance.

More recent work supporting this proposition has been provided in a meta-analysis conducted by Judge et al. (2004). They examined 151 studies considered in previous meta-analyses concerned with the relationship between intelligence and leader performance. Leader performance was assessed with respect to three measures: 1) perceived emergence, 2) perceived performance, and 3) objective performance. Two moderators, leader stress level and leader directiveness, were coded. It was found that the relationships of intelligence to leader performance was moderated by stress level and directiveness with the relationships of intelligence to leader performance approaching zero under conditions of high stress and low directiveness.

Stress is, of course, a complexconstruct(Muchinsky,1993). Stress may beinvoked bytimepressure,perceptionsof changeand risk, or threat, among other variables (Yukl, 2010). This observation, in turn, broaches a new question. What aspects of stress might moderate the effects of strategy training on leader performance as it is reflected in problem-solving, planning, and vision formation?

One key attribute of stressful situations is time pressure (Parker & DeCotiis, 1983). Cognition, especially the complex cognitive activities called for from leaders, is time intensive and resource intensive or cognitively demanding (Ericsson & Charness, 1994; Reeves & Weisberg, 1994). Due to the demands made by complex cognitive activities, it is reasonable to expect that time pressure will result in diminished leader performance on cognitive tasks — specifically, creative problem-solving, planning, and vision formation tasks. By the same token, however, under conditions of high demand, such as those induced by time pressure, people often rely on familiar strategies in problem-solving (Kaizer & Shore, 1995). Because of this, introducing new problem-solving strategies, even though these strategies are for working with case-based knowledge, would not benefit individuals experiencing high demand, as they would disregard the new information and resort to what they already know. As such, a null hypothesis is proposed to test the relationship between strategies for working with case-based knowledge and time pressure. Although, it must also be noted that testing a null hypothesis has its problems, specifically Kirk (1996) contends that a null hypothesis is always false, and thus rejecting it is simply a matter of power. However, in this situation, it is appropriate because we are essentially testing a hypothesis of no difference. More centrally, peoples' tendency to rely on familiar strategies under conditions of stress, specifically conditions of time pressure, suggests that training in strategies for working with case-based knowledge will not interact with time pressure. Furthermore, in addition to p values, effects sizes will also be reported. Accordingly, the following two hypotheses seem indicated:

Hypothesis three. Time pressure will result in poorer performance in leader creative problem-solving, planning, and vision formation.

Hypothesis four. Time pressure will not interact with training in strategies for working with case-based knowledge with respect to leader creative problem-solving, planning, and vision formation.

Another aspect of stressful situations is that they involve perceptions of threat, or risk (Yukl, 2010). One key attribute of threatening, high risk, situation is that negative potential outcomes are apparent. What should be recognized, however, is that threat may lead people to invest resources in cognition. Thus Ford & Gioia (2000), in a study of managerial problem-solving, found that leaders often initiated problem-solving in response to change events implying a loss with respect to the effectiveness of current organizational operations. Thus stress, at least when stress is induced through perception of negative outcomes, may induce more intensive cognition in leaders as they seek to minimize the threat. Due to this greater investment of cognitive

731J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

resources, one would expect to see improvements in creative problem-solving, planning, and vision formation under conditions where potential negative outcomes are salient. Furthermore, based on Higgins Regulatory Focus Theory, Forster, Higgins & Bianco (2003) suggest that high threat induces a prevention focus, which was found to lead to higher accuracy on a drawing task. This provides support that threat can lead to better performance by engaging information processing activities.

Hence, hypothesis five:

Hypothesis five. Awareness of potential negative outcomes will result in improved creative problem-solving, planning, and vision formation by people working in leadership roles.

Although awareness of potential negative outcomes may induce greater investment of cognitive resources, and thus better cognitive performance, on the part of leaders, the effects of negative outcome perceptions may interact with the type of strategy training being provided. Informational attributes of case-based knowledge, for example causes or restrictions, are abstract material drawn from prior experience (Marcy & Mumford, 2007). Under conditions of threat, abstract features of case-based knowledge will tend to be discounted because they lack clear, immediate, relevance to addressing the threat at hand (Weick, 1995). As a result, one would expect that training in strategies for working with informational attributes of case-based knowledge would prove less effective under conditions of threat. In fact, the expectation is consistent with the findings of Judge et al. (2004) regarding the weaker effects of intelligence on leader performance under conditions of stress. In contrast, when leaders are presented with threat situations, the threat can be addressed by people acting in leadership roles only through the effective exercise of influence on others. As a result, in threatening situations, it can be expected that leaders will tend to focus on and think about others — followers, peers, and superiors — as a basis for responding to and resolving the threat (Hunt, Boal & Dodge, 1999; Mumford, 2006). The tendency of leaders to focus on others, under conditions of threat, implies that training in strategies for working with social context information embedded in case-based knowledge may exert greater effects on leader creative problem-solving, planning, and vision formation. Hence our sixth, and final, hypothesis:

Hypothesis six. Under conditions of threat, training in strategies for working with informational features of case-based knowledge will not result in gains in leader creative problem-solving, planning, and vision formation. However, under conditions of threat, training in strategies for working with social contextual features of case-based knowledge will prove beneficial with respect to leader creative problem-solving, planning, and vision formation.

2. Method

2.1. Sample

The sample used to test these hypotheses consisted of 193 undergraduates attending a large southwestern university. The 75 men, 111 women, and 7 individuals who did report their sex who agreed to participate in the study were recruited from undergraduate psychology classes providing extra-credit for participation in research efforts. Participants reviewed a website where study descriptions were presented and selected the study(s) in which they wished to participate. The average age of those who agreed to participate in this study was 18.7 years old with most participants being in their freshman year. Their academic ability lay roughly a quarter of a standard deviation above freshmen matriculating at doctoral level research institutions. Virtually all study participants had some real world work experience.

2.2. General procedures

Participants were recruited to take part in a study of leader problem-solving. During the first half our of this three-hour study, participants were asked to provide informed consent, complete a background information form and measures of two cognitive controls that were timed. Specifically these timed measures examined intelligence and divergent thinking.

After completing this initial set of control measures, participants were randomly assigned to one of three training conditions. In the informational training condition, participants were asked to work through a series of self-paced instructional exercises intended to provide people with strategies for working with informational attributes (causes, resources, restrictions, and contingencies) of case-based knowledge in solving real world, day-to-day life, problems. In the social context training condition, participants were asked to work through a series of self-paced instructional exercises intended to provide people with strategies for working with social attributes (goals, affect, actors, and systems) of case-based knowledge in solving real-world, day-to-day, problems. In the no training condition, instructional material was not presented and participants were instructed to proceed directly to the experimental task being used to assess leader cognition.

The experimental task used to assess leader cognition was drawn from Strange & Mumford (2005). On this task, participants were asked to assume the role of principal, the leader, of a new experimental school. After reading through background material bearing on this school, participants were informed that three problems had emerged in school operations: 1) lack of motivation, 2) poor attendance, and 3) low test scores. Participants were asked to propose a solution to each of those problems. These problem solutions were scored for three aspects of leader cognition: 1) attributes of creative problem solving, 2) attributes of plans for addressing the problem, and 3) attributes of a leader's vision. Thus measures could be obtained of problem-solving, planning, and vision formation within this leadership role.

732 J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

Manipulations to induce stress were made in the context of this experimental task prior to participants beginning work on their problem solutions. Time pressure was induced by setting a time limit 30% below the time it typically took participants to complete this task. This time limit was stated in the instructions. Threat was manipulated by adding, or not adding, two paragraphs to the school description noting potential negative outcomes for the school, teachers, students, and the community if those problems were not resolved. After participants provided their solutions to these three problems, they were asked to complete 1) a set of untimed covariate control measures and 2) a survey to permit manipulation checks.

2.3. Covariate controls

Because the task presented called for a cognitive performance, participants were asked to complete measures that have been shown to influence cognitive performance, specifically measures of intelligence and divergent thinking (Vincent, Decker & Mumford, 2002). These measures were administered as timed covariates. The measures of intelligence used to provide an assessment of general cognitive ability was the Wonderlic Personnel Test. This test is composed of 50 verbal and mathematical problems scored for the number of correct responses. This test yields split-half reliabilities above .80 (McKelvie, 1989). Evidence for the validity of this measure of intelligence has been provided by Frisch & Jessop (1989) and Hawkins, Faraone, Pepple, Seidman & Tsuang (1990).

The second timed covariate was a measure of divergent thinking. This control measure was employed because divergent thinking has been shown to predict performance on novel, complex, ill-defined problems (Mumford, 2006). Because the experimental task involved similar problems, it was useful to include the measure as a control. The specific measure of divergent thinking employed was Merrifield, Guilford, Christensen & Frick (1962) consequences measure. On this test, people are presented with 5 unlikely events (e.g. “what would happen if everyone lost the ability to read and write”). They are given 10 minutes to list as many consequences as they can think of to these unusual events. When scored for fluency, or the number of responses produced, this measures yields internal consistency coefficients above .70. Evidence for the validity of this measure of divergent thinking may be obtained by consulting Vincent et al. (2002).

Because participants were asked to perform a leadership task, untimed control measures were included that examined leadership styles. The first style measure administered was Fleishman & Harris (1962) measure of consideration and initiating structure. The 20 items included in this measure present behavioral statement typical of each style (e.g. assign people under you to particular tasks (initiating structure) or let others do their work the way they think best (consideration)). People are asked to rate, on a 5 point scale, how typical those behaviors are of them when working in leadership roles. The consideration and initiating structure scales yield internal consistency coefficients above .70 (Fleishman & Harris, 1962). Evidence for the validity of these scales as a measure of leadership styles has been provided by Bass (1990), Judge et al. (2004), and Skinner (1969).

The second measure of leadership style was intended to assess style in thinking about leadership problems. More specifically, participants were asked to complete Bedell-Avers, Hunter & Mumford (2008) measure of Mumford's (2006) charismatic, ideological, and pragmatic leadership styles. On this 12 item measure, participants are presented with three, one paragraph, abstracts of leadership behavior evidenced by charismatic, ideological, and pragmatic leaders. People are asked to select the leader whose style is most similar to their own. The split-half reliabilities obtained for these measures of charismatic, ideological, and pragmatic style preferences lie in the .70s to .80s. Bedell-Avers et al. (2008) have provided evidence for the validity of this measure in allocating for the problem-solving performance of people working in leadership roles.

The next untimed covariate was intended to take into account the nature of the task — e.g. educational task. Accordingly, participants were asked to complete an abridged version of Pintrich, Smith, García & McKeachie (1993) Motivated Strategies for Learning Questionnaire (MSLQ). The 8 items included in this scale ask participants to indicate their cognitive investment in educational activities. For example, questions ask “I prefer courses that assess my curiosity, even if they are difficult” or “It is important for me to understand the content of a course.” People are asked to rate, on a 5 point scale, the extent to which they agree with these statements. This scale typically yields internal consistency coefficients about .70. Evidence bearing on the validity of this scale as a measure of learning goals has been provided by McClendon (1996) and Davenport (2003).

The final untimed covariate measure was Goldberg (1990) measure of introversion, openness, conscientiousness, agreeableness, and extroversion. On this inventory, people are presented with 100 descriptions (e.g. active, agreeable, or energetic). They are asked to rate, on a 9 point scale, the extent to which each of the stimuli words accurately describes them. The resulting scales measuring neuroticism, openness, conscientiousness, and agreeableness produce internal consistency coefficients above .80. Becker, Billings, Eveleth & Gilbert (1997), Conway & Peneno (1999), Reysen (2005), and Saucier (2002) have provided evidence for the construct validity of the resulting scale scores.

Additionally, a post task questionnaire was included. This questionnaire contained questions to serve as manipulation checks in addition to measuring participants interest and engagement in the scenario. Participants were presented with 10 items and asked to rate each item on a 5 point Likert scale, with a 1 representing a low response and 5 representing a high response. An example question from this survey involving engagement is “How engaged were you in this scenario?” During the study administration, this questionnaire was the last measure given to participants.

2.4. Experimental task

The experimental task participants were asked to work on was a modified version of a secondary school leadership task drawn from Scott et al. (2005a). This task was selected as the basis for the present investigation based on two considerations. First, prior

733J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

studies (Scott et al., 2005b) have shown the college students have sufficient experience in secondary schools to produce viable solutions to this problem. Second, performance on this task has been shown to elicit problem-solving (Scott et al., 2005a), planning (Shipman et al., 2010), and vision formation (Strange & Mumford, 2005).

On this task participants are asked to assume the role of principal of a new experimental secondary school established as part of a federal initiative to improve students' academic performance. The first two pages of this three page scenario describe the objectives of this experimental program and the procedures that will be used by the federal government to assess the effectiveness of this experimental school. More specifically, it is noted that scores on academic achievement tests as compared to scores on these tests obtained by students in traditional secondary schools will provide the basis for evaluating school success. If the school was successful in improving student academic performance additional funding would be provided by the Department of Education.

In the next section of this problem scenario, the current academic performance of schools in the state is described. Subsequently, a detailed description of the secondary school in which participants were asked to assume the role of principal was provided. This description material noted that the school was composed of 400 students from various cultural backgrounds. As an experimental school, the state Department of Education has provided adequate support. The current student-faculty ratio is 20 to 1. The teachers recruited for this experimental school were described as being of high caliber professionally and as being motivated to insure the success of this experimental school.

After providing this contextual information, the three problems participants were asked to solve were presented. Prior to introduction of these problems, participants were informed that the Oklahoma Excel School, the school where participants were assigned the role of principle, had encountered some obstacles in meeting expectations for success. The first problem, a lack of motivation problem, stated “There has been a drop in the motivation of students. It appears that they are not excited about learning and they have stopped applying themselves as they once were.” The second problem, a low attendance problem, stated “Absences have drastically increased as students are no longer involved in their educational experience and appear to have lost sight of the importance of their education.” The third problem, a low test scores problem, stated as “Test scores have decreased dramatically and remained consistently below average”.

Participants, in their role as principal of this school, were asked to prepare a solution to each of these three problems. Solutions were provided in a one to two page summary of how, as principal, they would attempt to solve each of these three problems. These one to two page written problem solutions provided the basis for appraising creativity, planning, and vision formation.

2.5. Manipulations

2.5.1. Threat In the no threat condition, participants were presented with the educational scenario described above. In the high threat

condition, however, two paragraphs were added to this scenario following description of the school and just prior to the statement of the problems they were to address in their role as principle. In the high threat condition, these two additional paragraphs stated “However, after state and national review, if Oklahoma Excel does not show an increase in academic performance, several negative consequences may result. To begin, failure of Oklahoma Excel will hurt the community which has invested much in the program. In addition to community effects, many students may be forced to move to another school, losing friends and academic progress that has been made.

Additionally, when the school closes, jobs will be lost (i.e. custodial, food, administrative), damaging the financial situation. As for the high-caliber teachers who left jobs to work at Excel, they will also be suddenly left out of work, finding it difficult to secure a job for the rest of the school year. In addition, they will have trouble finding another position because their once impressive resumes now include Excel, a school that failed because its students did not succeed academically. Overall, the result of Oklahoma Excel's failure will cause families in the community to lose faith in any new schools that may be proposed because the consequences of another failure would be too difficult to endure again.” Thus threat was induced by anticipating potential negative outcomes for the school, students, teachers, and staff. Because threat is not directly experienced by participants, this manipulation treats threat as a perceived condition.

2.5.2. Time pressure In the no time pressure condition, participants were allowed to work at their own pace in the reading material and generating

their three problem solutions. In the high time pressure condition, participants were instructed to complete work on the educational material within a time limit. This time limit was set, based on the results obtained in a pilot study of 7 undergraduates, to be 30% below the amount of time typically needed by people to complete this exercise. These time limitations were noted in the instructions given to participants working in the high time pressure condition. Examination of a post study question item asking “Did you feel pressured for time when completing the problem-solving task” yielded a significant (p≤.01) positive correlation (r=.29) with exposure to this manipulation. Thus it appears that the time pressure manipulation did induce perceptions of pressure.

2.5.3. Instructional manipulations The instructional manipulations included three conditions. In the no training condition, participants were simply instructed to

begin work on the educational problem-solving task. In the informational strategies training condition, participants were asked to work through instructional modules intended to provide strategies for working with causes, resources, restrictions, and contingencies in solving problems through case-based knowledge. In the social strategies condition, participants were asked to

734 J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

work through instructional modules intended to provide strategies for working with goals, affect, actors, and systems in solving problems through case-based knowledge. It is of note this distinction between informational and social elements of case-based knowledge is consistent with prior theoretical work by Mumford et al. (2007a).

The same general instructional approach was applied in training strategies for working with informational and social material embedded in case-based knowledge. This general training protocol was based on earlier work by Marcy & Mumford (2007) examining instructional procedures for training strategy application. For any given strategy, in the first instructional module at hand, the strategy was defined in concrete operational terms and an illustration of how this strategy could be applied in problem-solving was provided. In the second component of a module, participants were presented with a brief, one paragraph problem, where strategy application was called for, and they were then asked to answer a series of 3 multiple choice questions about how this strategy could be applied in solving the problem at hand. In the third component of a module, they were asked to review correct answers to the questions presented in the second component where the reasons a given answer concerning strategy application was correct were provided. In the fourth component of each module, participants were presented with two, one paragraph, problem scenarios and were asked to provide a short, four to seven line written answer as to how the strategy under consideration might be used to solve each problem.

With regard to the structure of these instructional interventions one further point should be mentioned. All instructional material questions and problems were developed to reflect problem solving in general life domains — for example, inviting friends to dinner and finding the stove is broken. Use of general life problems as a basis for developing training content is useful for two reasons. First, the real life relevance of these materials increased participant motivation and learning during training. Second, use of general life content allowed transfer of training to the leadership domain to be assessed without undue confounding (Goldstein, 1986). Participants were allowed to work through each instructional module applying in their training condition at their own pace. After completing the relevant instructional modules, participants proceeded to working on the educational leadership problems.

In the informational training condition, participants were asked to complete 8 instructional modules with two strategies being trained for each relevant form of case content. Thus a) for causes — 1) work with causes that effect multiple outcomes, 2) work with causes that can be manipulated, b) for resources — 3) identify critical resources, 4) insure the availability of critical resources, c) restrictions — 5) identify restrictions that can be controlled or manipulated, 6) make adjustments for restrictions that are uncontrollable, d) contingencies — 7) insure necessary contingencies have been addressed, and 8) insure critical contingencies are being managed.

In the social training condition, participants were also asked to complete 8 instructional modules with two strategies being trained for each relevant form of case content. Thus a) for goals — 1) what goals can be obtained, 2) can goals, or outcomes, from different sources be managed, b) affect — 1) in what ways can positive affect be assured, 4) can the situation be changed to manage negative affect, c) actors — 5) what steps can be taken to insure critical actors are involved, 6) what can be done to manage negative actors, d) systems — 7) identify key components of system operation, and 8) identify how adjustments might be made to system operations.

2.6. Dependent variables

Solutions to the three problems presented in the educational scenario were evaluated by three judges. All judges of the solutions to each problem were students. Students were used as judges because prior studies (Strange & Mumford, 2005) have shown that evaluations of attributes of solutions to problems derived from this experimental task converge with the evaluations of other stakeholder groups. All ratings were to be made on 5 point benchmark ratings scales based on the findings of Redmond, Mumford & Teach (1993) concerning the judgmental evaluation of complex constructs such as solution quality and originality. Figs. 1–3 illustrate the nature of these rating scales.

Prior to evaluating solutions to these three problems, judges were provided with a 20-hour training program. In this training program, judges were familiarized with the operational definitions of each construct to be used in appraising each solution and the nature of the rating scales to be applied in appraising these constructs.

Subsequently, judges practiced making these ratings on a set of sample products. They then met and discussed any observed discrepancies. Following training, these judges were asked to rate aspects of solution creativity, planning, and vision formation with respect to the solutions provided to each of the three problems.

The three dimensions of solution creativity judges were asked to evaluate were drawn from earlier work by Mumford & Gustafson (2007). More specifically, judges were asked to appraise the quality, originality, and elegance of solutions proposed to each problem. As expected, based on prior work (Scott et al., 2005a), although evidencing adequate construct validity, based on scale correlations, these ratings of solution quality, originality and elegance produced the expected strong positive correlations. When aggregated to provide an index of overall creative problem — the resulting inter-rater agreement coefficient was .75. Accordingly, this index of solution creativity provided the first dependent variable examined in the present study.

The second set of ratings judges were asked to make focused on attributes of the quality for the plans apparent in participants' solutions to each of the three problems. Based on the observations of Marta et al. (2005), judges were asked to appraise each solution for the effectiveness of the plan provided, formulation of backup plans, forecasting outcomes of plan execution, and identification of monitoring events. Again, these planning ratings, while evidencing good construct validity, yielded the expected positive correlations. Hence they were aggregated into an overall planning dimension. The inter-rater agreement coefficient obtained for overall plan ratings was .86.

735J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

The third set of ratings judges were asked to make focused on attributes of leaders' vision as apparent in their problem solutions. The three dimensions of vision judges were asked to rate were drawn from Strange & Mumford (2005). More specifically, judges were asked to rate vision impact, inspirational motivation, and potential for adaptive implementation. Because these rating scales evidenced a substantively meaningful pattern of inter-correlations, they were aggregated to provide an overall dimension of leader vision quality. The inter-rater agreement coefficient obtained for this vision quality dimension was .76.

In order to examine the types of knowledge employed by participants on the problem-solving task, a fourth set of ratings for knowledge structures was completed. Figs. 4–6 provide an illustration of the nature of these ratings. Judges provided ratings for case-based (e.g., Scott et al., 2005b), schematic (e.g., Ward, Patterson & Sifonis, 2004), and associational knowledge (e.g., Gruszka & Necka, 2002). Two types of ratings were obtained for each type of knowledge, the quantity, or the amount of information given for each type of knowledge, and the quality, or how well the type of knowledge was employed by the participant. The inter-rater agreement coefficient obtained for these knowledge structure dimensions was .79.

2.7. Analyses

The analyses were conducted using a similar procedure recommended by Marta et al. (2005) to analyze multivariate studies that are sequential in nature. There were three sets of analyses used. First, training, threat, and pressure were treated as independent variables while the six knowledge scales (quantity and quality for case-based, schematic, and associational knowledge structures) served as dependent variables. Second, following a median split of the six new scales, the knowledge structures were treated as independent variables while creativity, solution planning, and leader vision were treated as dependent variables. Third, for the primary analyses, the overall scores for solution creativity, planning, and vision quality served as the

Quality

Definition: the overall quality of the leader’s plan

Things to look for: - Completeness: Did the participant understand the critical issues? Did he/she address all of the most relevant

information at hand? - Coherence: Wa s the response coherent? Wa s it well thought out and logical? - Usefulness: Is the response actually feasible and appropriate for addressing the problems?

Rating Scale

1 – Poor quality. The solutions are haphazard and fragmented and do not address any of the key issues; they do not provide key information in a logical manner.

Ex: “Lack of motivation has to do with the school and the stuff they are doing and how they are doing that.”

2 – Poor to average quality. A few key issues may be addressed; however, clear solutions are still not presented.

3 – Average quality. The solutions are presented in a logical form; a number of key issues may still be missing or vague, but overall the solutions address some of the major issues of the problems and are presented clearly and coherently.

Ex: “For low attendance, the school should offer attractive incentives for attendance. This may cost money, but if it encouraged students to attend, then it is worth it. Also, teachers need to ensure a proper and happy environment to make class interesting.”

4 – Average to excellent quality. Many of the key issues are addressed in the solutions are the solutions are feasible.

5 – Excellent quality. The solution is presented so that is exceptionally coherent and clear and addresses the key issues in a manner that is feasible.

Ex: “Too much of education these days is dry and boring. Allowing the students to actually see and experience the curriculum will motivate them to learn more, and (surprise!) this will have an impact on the attendance problem. Then, if you add something extra to attendance (say, daily random drawings for concert tickets, movie passes, food cards, etc) students would be motivated to attend to see if they get whatever is being offered. Cheap? No, and not exactly the reason we want them there, BUT if it gets them into the seat so we can then get them to become interested in the learning and want to go.”

Fig. 1. Example of creative problem-solving rating scales.

736 J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

dependent variables examined in a series of analyses of covariance tests. The independent variables examined in analyses consisted of the time pressure, threat, and training manipulations.

The control variables were treated as potential covariates along with responses to the post-study survey questions. A covariate was retained in the final set of analyses only if it produced a relationship significant at the .05 level in initial analyses. Covariate selection was made prior to final analyses using a combination of forward insertion and backward deletion procedures. As such, covariates were only retained if they were significant from covariate selection methods. This procedure was suggested by Tabachnick & Fidell (2001) because it maximizes degrees of freedom, and thus the utility of the ANCOVA, by ensuring that unnecessary covariates were not included in final analyses. Because of this, only a select few covariates were retained in the final analyses, and the covariates differ among analyses due to significance levels.

3. Results

3.1. Correlations

The means, standard deviations, and intercorrelations for creativity, solution planning, and leader vision are presented in Tables 1 and 2. Creativity yielded significant (p≤ .01) positive correlations with solution planning (r=.87) and leader vision

Vision

Definition: The extent to which the leader constructs an idealized prescriptive mental model of the predicted situation.

Things to look for: - How well does the participant construct a model of the solutions? - Does the leader present positive expected outcomes? - Is the solution guided by the most beneficial outcomes predicted by the leader? - It is important to note that the prescriptive mental model does not need to be attainable but simply provide

guidelines for actions.

Rating Scale

1 – Low vision. The leader does not construct a model of the best possible outcomes. The response focuses on negative outcomes or suggests that their actions are sure to fail. No mention of actions being guided by a model of predicted

Ex: “Do surveys and gather information on their lack of motivation. Talk with the board of education for a more credit policy on them.”

2 – Low to moderate vision. The leader somewhat constructs a model of the best possible outcomes.

3 – Moderate vision. The leader constructs an incomplete model of the best possible outcomes. The response may mention few negative outcomes or outcomes that may fail. There will be mention of actions being somewhat guided by a model of predicted effects, but is missing key issues.

Ex: “Student success rate seems to be fueled by their motivation. Though it is only one of three problems, if this single problem is solved, the others should be along with it. A new curriculum will be proposed for next hear brining about a more exciting hands-on experience for the students. Such changes include more unique labs in the science department and exclusive field trips to educational, yet fascinating locations. The studies will be strict, yet enjoyable, allowing for both knowledge and pleasure to be obtained throughout the year. “

4 – Moderate to excellent vision. The leader constructs a nearly complete model of the best possible outcomes.

5 – Excellent vision. The leader constructs a complete model of the best possible outcomes. The response will not mention any negative outcomes or failure. All actions will be guided by a model of predicted effects; however the model or actions may still be flawed.

Ex: “For students’ lack of motivation, we need teachers who will make class, lectures, and learning interesting and exciting. Students should see that to be successful in this world, one must be educated properly. We need students to realize that status, money, respect, and success come from education. Whatever they wish to do, the starting place is here. “

Fig. 2. Example of leader vision rating scales.

737J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

(r=.85). Solution planning also yielded a significant (p≤ .01) positive correlation with leader vision (r=.86). These correlations suggest that all three dimensions are closely related when employed by leaders in their problem-solving efforts.

3.2. Mediation

The firststep in the mediation analysis wasto test theeffectsof trainingon knowledge structures.Table 3 presents the results of this analysis. There were no significant findings for quality ratings of case-based, schematic, and associational knowledge. For ratings of the quantity of case-based knowledge, divergent thinking was found to be the only significant covariate (F(1, 180)=8.94, p≤.01). A significant main effect (F(2, 180)=3.07, p≤.05) was found for the quantity of case-based knowledge used. Consistent with our predictions, case-based knowledge exhibited higher means in conditions of informational training (M=2.87, SD=.88) and interpersonal training (M=2.87, SD=.63) than in the no training condition (M=2.49, SD=.53). No other significant main effects for training were obtained in this analysis.

For the next step in the mediation analyses, a median split of the six new scales was conducted and entered as the independent variables in two ANCOVAs, one for quantity ratings and one for quality ratings. Tables 4–9 present the results of these analyses. When testing quantity ratings on creativity, Engaged in Scenario was the only significant covariate (F(1, 180)=6.92, p≤ .01). Significant effects were found for the quantity of case-based knowledge (F(1, 180)=33.60, p≤.001), suggesting that individuals exhibiting higher levels of case-based knowledge (M=2.96, SE=.067) also had higher levels of creativity than those that employed less case-based knowledge (M=2.44, SE=.059) in their problem solutions. Similar findings were obtained for schematic knowledge (F(1, 180)=24.28, p≤ .001), such that those employing more schematic knowledge (M=2.92, SE=.061) had higher creative performance than those employing less schematic knowledge (M=2.48, SE=.065). Findings for associational

Plans

Definition: The extent to which the leader focuses his/her solution on plans.

Things to look for: - How well does the participant plan for actions to address the problems? - Does the leader structure actions of the school, employees, and students? - Is the solution geared toward future activities? - Does the solution exploit emergent opportunities?

Rating Scale

1 –Low plans. The leader does not focus on plans. The response does not contain coherent plans to address the problems. The leader does not adequately structure actions of the followers and system.

Ex: “Make students interested and motivated. Make students responsible for not showing up. Make them get a high score on tests or they are kicked out.”

2 –Low to moderate plans. The leader minimally focuses on plans.

3 – Moderate plans. The leader somewhat focuses on plans. The response contains coherent although incomplete plans to address the problems. The leader provides some structure for actions of the followers and system. There is some mention of future opportunities.

Ex: “To increase test grades I would create an incentives program that rewards students that do well on their tests with privileges, such as off-campus lunches and priority parking spots. Students would be more inclined to do well on these tests with rewards attached.”

4 – Moderate to excellent plans. The leader focuses on plans.

5 – Excellent plans. The leader strongly focuses on plans. The response contains very coherent plans to address the problems. The leader provides in depth structure for actions of the followers and system. There is considerable discussion of future opportunities.

Ex: “First off all the students must be surrounded by peers they can enjoy. The teachers must show a passion for their work that motivates the children to work hard to succeed. The new school cannot be all work and no play. Addition of extracurricular activities will really help the students to enjoy school. Teaching methods need to be innovating and fun. They kids cannot sit in a classroom all day reading a book or taking notes. Lastly the teachers must understand and communicate well with the students.”

Fig. 3. Example of planning rating scales.

738 J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

1. Case-Based Knowledge

Definition: Case-based knowledge is episodic knowledge that involves the formation of a mental model describing critical aspects of past performance. (For example: planning a shopping trip) This is a form of contextual knowledge that provides a model for action when people encounter similar situations to those they have faced in the past. They draw on these cases, generating solutions based on the results of previous experiences.

Includes information about: Goals, key actions, outcomes, contingencies, restrictions, and potential opportunities Typical cases and deviation from typical cases Identification of a similar case in past experience

Rating Scale: 1 - Poor rating: Quantity: Case-based knowledge was not employed by the participant. There is no mention of typical or atypical cases drawn from past experience. Quality: Of the case-based knowledge employed by participants, it does not appear to be of high quality. The case is not relevant to the problem at hand.

Quantity Example: Show them the status-quo and tell them that the shame on the whole state is the same on themselves, but the success they can make makes them the pride of the whole state.

Quality Example: I would begin a program called “Excel dollars.” What this is, is a way for students to get motivated to do well. Every time a student scores well on a test, then they get a set number of “Excel dollars.”

2 - Poor to average rating: Quantity: Case-based knowledge is alluded to by the participant, although it is unclear whether a model of past experiences was drawn upon. Quality: The case provided appears to be relatively low quality. The case is likely unrelated to the problem at hand, or the connection is unclear.

3 - Average rating: Quantity: The participant employs case-based knowledge somewhat. There is some use of previous cases regarding typical situations and deviations from the typical situation. Quality: Of the case-based knowledge employed by participants, it appears to be of moderate quality. The case is related in some way to the problem at hand.

Example (Quantity and Quality): In order to increase motivation, I would allow out of classroom studies to be conducted. Teachers would be allowed to take the students on education focused field-trips. This may be sort of childish, but it could increase the motivation by getting the students out of the normal, everyday classroom, and into a more interesting learning environment.

4 - Average to excellent rating: Quantity: It is clear that a previous case model is being drawn up on in the problem solution. There is some analysis of typical and atypical cases, and appropriate adaptation to the current situation is engaged. Quality: The schematic knowledge mentioned is of somewhat good quality, however the knowledge provided is not complete. The case is clearly related to the problem at hand.

5) Excellent rating: Quantity: The participant has successfully drawn from case-based knowledge. It is clear that previous cases have been employed, analyzed, and adapted to the current situation. Similar cases from past experience have been used along with an analysis of typical cases and deviations from the typical cases. Quality: Of the case-based knowledge employed, it is of good quality and could be employed with successful results. The case is directly related to the problem at hand.

Example (Quantity and Quality): To improve the motivation of the student, I would encourage the teachers to provide more hands-on activities to keep up the interest. Also, I would bring in professionals from different careers to give speeches to the kids to also encourage them to get excited about learning and getting a full education. These are things that worked at my high school.

Fig. 4. Example of case-based knowledge rating scales.

739J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

knowledge also indicated significant effects for creativity (F(1, 180)=24.28, p≤ .01), such that those using more associational knowledge (M=2.85, SE=.061) had higher scores than those using less (M=2.55, SE=.065).

A similar pattern of findings was also found for quality ratings of the knowledge structures on creativity. Again, Engaged in Scenario was the only significant covariate (F(1, 180)=6.29, p≤.05). Significant effects were again found for case-based knowledge (F(1, 180)=10.86, p≤.001), such that higher levels of case-based knowledge (M=2.88, SE=.063) led to higher scores on creativity

2. Schematic Knowledge

Definition: Schematic knowledge involves concepts and principles abstracted from past experience. (Ex: birds fly and have feathers) The concepts may be viewed as a set of categories to organize objects or exemplars.

Includes information about: Principles for organizing and establishing relationships between concepts Construction of relationships linking different categories or concepts Feature search and mapping Principles abstracted from past experience

Scale and Benchmarks: 1 - Poor rating: Quantity: Schematic knowledge is not employed by the participant in the solution. Abstracted principles are not engaged. Quality: Of the schematic knowledge employed by participants, it does not appear to be of high quality. The principles used have little to do with the problem.

Quantity Example: I would begin a program called “Excel dollars.” What this is, is a way for students to get motivated to do well. Every time a student scores well on a test, then they get a set number of “Excel dollars.”

Quality Example: I would work for funding for more extra-curricular activities and more strict rules on absences.

2 - Poor to average rating: Quantity: The participant may be slightly drawing upon principles, but it is unclear. Quality: Some allusion to concept principles/relationships may occur, but they are not of very high quality. These principles or abstractions seem to be unrelated or only slightly related to the problem.

3 - Average rating: Quantity: Schematic knowledge is somewhat drawn upon. There is some use of principles abstracted from past experience or linkages between different concepts. Quality: Of the schematic knowledge employed by participants, it is of moderate quality. The principles used are related to the problem in some way, though the connection may be somewhat unclear.

Example (Quantity and Quality): It doesn’t matter is the students are motivated if there is only half the population there. I would enforce a strict attendance policy. The policy would set a maximum number of absences before the student would be suspended.

4 - Average to excellent rating: Quantity: It is fairly clear that schematic knowledge is being used. There are principles drawn from abstracted principles and relationships between concepts or categories. Features appear to be searched and mapped for the problem at hand. Quality: The processes are of somewhat good quality, though they are not complete. The principles used are clearly related to the problem but they may be used in a less than optimal manner.

5 - Excellent rating: Quantity: The participant engages in the use of schematic knowledge by drawing from abstracted principles, relationships between concepts and categories, and feature search and mapping. Quality: The schematic knowledge used by participants is of quite good quality and could be successfully applied to a problem. The principles used are directly related to the problem.

Example (Quantity and Quality): Do surveys and gather information on their lack of motivation. Negotiate with the Board of Education for a more credit policy on them. Making quizzes and tests slightly harder than before. Do surveys and teaching evaluations. Tell them that once successful in improving the educational ranking, 1/3 of the extra funding would be used on giving out prizes.

Fig. 5. Example of schematic knowledge rating scales.

740 J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

3. Associational Knowledge

Definition: Associational knowledge reflects regularities in experience based on linkages among stimuli and responses. (Ex: attending a party is associated with having fun) These stimuli/response relationships are organized in a network structure, such that activation of one relationship serves to activate other related relationships.

Includes information about: Usually information that is social in nature Links between people or ideas and their behavior Predictive assumptions based on linkages between stimuli and response

Scale and Benchmarks: 1 - Poor rating: Quantity: Associational knowledge is not employed by the participant in the solution. Linkages between people or ideas and their behaviors are not considered. Quality: Of the associational knowledge employed by participants, it does not appear to be of high quality. The stimuli and responses considered are unrelated to the problem.

Quantity Example: I would work for funding for more extra-curricular activities and more strict rules on absences.

Quality Example: It doesn’t matter is the students are motivated if there is only half the population there. I would enforce a strict attendance policy. The policy would set a maximum number of absences before the student would be suspended.

2 - Poor to average rating: Quantity: The participant may be slightly drawing upon relationships between stimuli and responses, but it is unclear. Quality: Some allusion to relationships between ideas/people or predictive assumptions may be present, but they are not of very high quality. These relationships may only be tangentially related to the problem.

3 - Average rating: Quantity: Associational knowledge is somewhat drawn upon. There is some use of predictive assumptions based on relationships between a stimulus and a response or consideration about these relationships. Quality: Of the associational knowledge used by participants, it is of moderate quality. The knowledge used indicates an understanding of the relationships between stimuli and possible reactions within the problem. The knowledge used is at least somewhat related to the problem.

Example (Quantity and Quality): I think it is important to develop traditions and high school spirit. This way, students carry pride and respect, and want to do well. This could be accomplished by requiring each student to participate in an extracurricular activity. Create more school challenges and drives to work as a whole.

4 - Average to excellent rating: Quantity: It is clear that the relationships between stimuli and responses are being considered. Quality: Although there is some analysis of potential responses to specific stimuli, and predictions are made based on this analysis, it is only of somewhat high quality. The knowledge used is related to the problem, though some stimuli may be ignored or not fully considered.

5 - Excellent rating: Quantity: The participant engages in the use of associational knowledge by drawing from knowledge of how stimuli and responses interact, the potential consequences of these interactions, and the use of this knowledge to make assumptions about future behavior. Quality: Of the associational knowledge used by participants, it is of quite good quality and could be successfully applied to a problem. The knowledge used is clearly related to the problem and the participant is considering most, if not all, of the stimuli within the problem.

Example (Quantity and Quality): I will provide hands-on, interactive tutoring after school before exams. My teachers are all about the students and will do anything to help them succeed. After school tutoring and advisory tutoring will improve the test scores.

Fig. 6. Example of associational knowledge rating scales.

741J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

than lower levels of case-based knowledge (M=2.55, SE=.079). Significant findings for schematic knowledge (F(1, 180)=12.81, p≤.001) also indicated that higher levels of creativity were found when more (M=2.90, SE=.070) as opposed to less (M=2.53, SE=.074) schematic knowledge was used. Again, a similar pattern was observed for associational knowledge (F(1, 180)=6.75, p≤.001) in that higher scores (M=2.85, SE=.083) led to more creativity than lower scores (M=2.58, SE=.059).

Quantity ratings were also tested on solution planning. A significant main effect was found for case-based knowledge (F(1, 181)=33.66, p≤ .001), indicating that more use of case-based knowledge (M=1.99, SE=.039) led to higher scores on planning than using it less (M=1.69, SE=.034). Significant effects were also found for schematic knowledge (F(1, 181)=21.66, p≤ .001). These findings also suggests that more schematic knowledge (M=1.96, SE=.035) leads to more planning than less schematic knowledge (M=1.72, SE=.038). Associational knowledge was also significant (F(1, 181)=14.39, p≤.01), and it was found that more use of associational knowledge (M=1.94, SE=.035) led to more planning than less use (M=1.74, SE=.038).

For solution planning, the results for quality ratings exhibited, again, a similar pattern. Case-based knowledge was found to be significant (F(1, 181)=8.01, p≤ .01), indicating that using more case-based knowledge (M=1.93, SE=.038) led to more planning than using less (M=1.76, SE=.048). Again, for schematic knowledge, the results show significant main effects (F(1, 181)=8.86, p≤ .01), such that more schematic knowledge (M=1.94, SE=.042) leads to more planning than less schematic knowledge (M=1.75, SE=.044). Associational knowledge was also marginally significant (F(1, 181)=3.21, p≤ .10), with higher scores (M=1.90, SE=.050) leading to more planning than lower scores (M=1.79, SE=.036).

When looking at quantity ratings of case-based, schematic, and associational knowledge on leader vision, Engaged in Scenario was again found to be significant (F(1, 180)=6.14, p≤ .05). As expected, case-based knowledge resulted in significant main effects (F(1, 180)=29.24, p≤ .001), such that more case-based knowledge (M=2.27, SE=.044) led to higher levels of leader vision than less case-based knowledge. Schematic knowledge was also significant (F(1, 180)=19.34, p≤.001), with higher levels (M=2.24, SE=.042) alsobeing associated withmore leader visionthanlowerlevels(M=1.96,SE=.045).Significant maineffects forassociationalknowledge (F(1, 180)=8.58, p≤.01) also indicated that more use (M=2.19, SE=.042) led to more leader vision than less use (M=2.01, SE=.045).

Table 1 Means and standard deviations by condition.

Informational training

Interpersonal training

No training Threat No threat Time pressure No time pressure

M SD M SD M SD M SD M SD M SD M SD

Creativity 2.62 0.74 2.72 0.67 2.72 0.77 2.68 0.78 2.70 0.67 2.86 0.79 2.54 0.64 Solution planning 1.81 0.43 1.88 0.37 1.81 0.43 1.81 0.44 1.81 0.39 1.90 0.44 1.77 0.38 Leader vision 2.08 0.53 2.12 0.45 2.09 0.51 2.08 0.51 2.10 0.47 2.20 0.54 2.00 0.44

Table 2 Means, standard deviations, and intercorrelations.

Mean SD 1 2 3

1 Creativity 2.69 .73 1.00 2 Solution planning 1.83 .49 .85 1.00 3 Leader vision 2.09 .41 .87 .86 1.00

Note. N=187. All correlations significant at p≤ .01 level.

Table 3 Analysis of covariance results for training on quantity ratings of knowledge structures.

F df p η2

Case-based Covariates

Divergent thinking 8.94 1, 180 .003 .049 Main effects

Training 3.07 2, 180 .049 .034

Schematic Covariates

Intelligence 10.45 1, 180 .001 .058 Main effects

Training .94 2, 180 .39 .011

Associational Main effects

Training .45 2, 180 .81 .000

Note: F = F ratio; df = degrees of freedom; p = significance level using Roy's largest root; η2 = effect size (eta squared).

742 J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

The results of the analysis looking at quality ratings of the knowledge structures on leader vision also fell into the same pattern. Engaged in Scenario was found to be a significant covariate. (F(1, 180)=5.30, p≤ .001). Case-based knowledge resulted in significant main effects (F(1, 180)=10.83, p≤.001), with more (M=2.22, SE=.044) leading to higher leader vision scores than less (M=1.99, SE=.055). Significant findings for schematic knowledge (F(1, 180)=8.66, p≤ .01) also indicated that more use of this knowledge (M=2.21, SE=.049) led to more leader vision than using it less (M=2.00, SE=.051). Associational knowledge also showed significant effects (F(1, 180)=7.21, p≤ .01) along these lines. It was found that higher use (M=2.20, SE=.057) of associational knowledge led to higher leader vision scores than lower use (M=2.01, SE=.041).

3.3. Creative problem-solving

Table 10 presents the results obtained in the analysis of covariance conducted to account for creativity of solutions provided the three educational leadership problems. The only significant covariate (F(1, 180)=5.44, p≤ .05) was perceived difficulty of the problem-solving tasks as indicated in the post experimental questionnaire. As might be expected, perceived difficulty was negatively related to the creativity of solutions produced to the three leadership problems as evidenced in their quality, originality, and elegance. A significant main effect (F(1, 180)=11.17, p≤ .001) was also obtained for time pressure (See Fig. 7). Consistent with hypothesis three, creative problem-solving performance was better when leaders were not (M=2.86, SE=.072) working creative time pressure as opposed to when they were (M=2.52, SE=.070) working under time pressure.

In testinghypothesis three, although a significantmain effectwasfound for time pressure, no significantmain effects were found for training. A significant interaction (F(1, 180)=3.45, p≤.05) was obtained between perceived threat and training condition (See Fig. 8). As with hypothesis six, inspection of the cell means indicated that training in strategies for working with objective information embedded in case-based knowledge proved more beneficial under conditions where no threat was present (M=2.74, SE=.12) as opposed to conditions where threat was present (M=2.47, SE=.12). Training in strategies for working with social information embedded in case-based knowledge, however, proved beneficial under both high (M=2.73, SE=.12) and low (M=2.77, SE=.12) threat conditions with regard to leaders creative problem solving. Notably, when no threat was induced and no training provided, consistent with hypotheses 1 and 2, the worst leader creative problem-solving performance was observed (M=2.46, SE=.12). However, consistent with hypothesis five that threat might induce creative problem-solving on the part of leaders under conditions of high threat, but no training, the best creative problem-solving performance was observed (M=2.92, SE=.12).

Table 4 Analysis of covariance results for quantity of knowledge structures on creativity.

F df p η2

Covariates Engaged in Scenario (post questionnaire) 6.92 1, 180 .009 .038

Main effects Case-based 33.60 1, 180 .000 .160 Schematic 24.28 1, 180 .000 .121 Associational 11.65 1, 180 .001 .062

Note: F = F ratio; df = degrees of freedom; p = significance level using Roy's largest root; η2 = effect size (eta squared).

Table 5 Analysis of covariance results for quality of knowledge structures on creativity.

F df p η2

Covariates Engaged in Scenario (post questionnaire) 6.29 1, 180 .013 .034

Main effects Case-based 10.86 1, 180 .000 .058 Schematic 12.81 1, 180 .000 .067 Associational 6.75 1, 180 .001 .037

Note: F = F ratio; df = degrees of freedom; p = significance level using Roy's largest root; η2 = effect size (eta squared).

Table 6 Analysis of covariance results for quantity of knowledge structures on solution planning.

F df p η2

Main effects Case-based 33.66 1, 181 .000 .016 Schematic 21.66 1, 181 .000 .107 Associational 14.39 1, 181 .000 .076

Note: F = F ratio; df = degrees of freedom; p = significance level using Roy's largest root; η2 = effect size (eta squared).

743J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

3.4. Solution plans

Table 11 presents the results obtained when performance in planning solutions to these leadership problems was assessed. In this case it was found that the post experimental survey question examining engagement in the problem–solution task proved to be a significant (F(1, 177)=6.76, p≤ .01) covariate, with greater engagement inducing stronger planning (Mumford, Schultz & Van Doorn, 2001). It was also found that divergent thinking proved to be a marginally significant control (F(1, 177)=2.40, p≤ .15) with the findings indicating that better plans arose when people engaged in divergent thought perhaps as a result of more extensive forecasting (Byrne, Shipman, & Mumford, 2010).

Again, in testing hypothesis three, only time pressure produced a significant (F(1, 177)=10.68, p≤.001) main effect (See Fig. 9), training did not. It was found that people in leadership roles produced better plans when they were not (M=2.21, SE=.049) as opposed to when they were (M=1.99, SE=.048) placed under time pressure. For hypothesis four, a marginally significant interaction (F(2, 177)=2.79, p≤.10) was also observed between the training and threat manipulations (See Fig. 10). In keeping with the findings obtained for hypothesis six for creative problem-solving, it was found that training in strategies for using information imbedded in cases was beneficial under neutral (M=2.15, SE=.083) asopposed to high threat (M=2.01, SE=.083) conditions. For hypothesis six, training interventions, at least informational strategy training under neutral conditions and social strategy training under both threat and neutral conditions, proved more effective than no training under neutral conditions (M=1.94, SE=.085). This finding is also somewhat consistent with hypotheses 1 and 2. However, threat with no training, as predicted by hypothesis five, produced comparable performance to that observed under the conditions where strategy training proved beneficial (M=2.19, SE=.084).

3.5. Leader vision formation

Table 12 presents the results obtained in the analysis of covariance when attributes of leader vision formation were treated at the dependent variable of interest. As may be seen, perceived problem difficulty (F(1, 178)=5.83, p≤ .05) and engagement in the problem solving task (F(1, 178)=5.73, p≤ .05), both post study survey questions, proved to be significant covariates. As might be expected, engagement in the task was positively related to formation of viable visions. Moreover, perhaps due to greater investment of cognitive resources, perceived task difficulty was also positively related to vision.

Time pressure, again for hypothesis three, produced a significant (F(1, 178)=9.31, p≤ .01) main effect (See Fig. 11). Main effects for training were, again, not significant. In keeping with our earlier observations for hypothesis three, it was found that

Table 7 Analysis of covariance results for quality of knowledge structures on solution planning.

F df p η2

Main effects Case-based 8.01 1, 181 .005 .043 Schematic 8.86 1, 181 .003 .048 Associational 3.21 1, 181 .075 .048

Note: F = F ratio; df = degrees of freedom; p = significance level using Roy's largest root; η2 = effect size (eta squared).

Table 8 Analysis of covariance results for quantity knowledge structures on leader vision.

F df p η2

Covariates Engaged in scenario (post questionnaire) 6.14 1, 180 .014 .034

Main effects Case-based 29.24 1, 180 .000 .142 Schematic 19.34 1, 180 .000 .098 Associational 8.58 1, 180 .004 .046

Note: F = F ratio; df = degrees of freedom; p = significance level using Roy's largest root; η2 = effect size (eta squared).

Table 9 Analysis of covariance results for quality knowledge structures on leader vision.

F df p η2

Covariates Engaged in scenario (post questionnaire) 5.30 1, 180 .023 .029

Main effects Case-based 10.83 1, 180 .001 .058 Schematic 8.66 1, 180 .004 .047 Associational 7.21 1, 180 .008 .039

Note: F = F ratio; df = degrees of freedom; p = significance level using Roy's largest root; η2 = effect size (eta squared).

744 J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

stronger visions emerged under conditions of no time pressure (M=2.20, SE=.049) as opposed to conditions of time pressure (M=2.00, SE=.047). For hypothesis six, a significant interaction (F(2, 178)=3.27, p≤ .05) was also obtained, again, between the training and threat manipulations (See Fig. 12). With regard to vision formation, it was found that training in informational strategies proved beneficial under neutral conditions (M=2.15, SE=.082) as opposed to conditions of threat (M=1.98, SE=.086) while training in social strategies for using case-based knowledge proved beneficial in both neutral (M=2.15, SE=.082) and negative (M=2.15, SE=.083) threat conditions. In testing hypotheses one and two, it was found that, when training proved of value, informational training under neutral conditions or social training under both neutral and threat conditions, better vision formation was observed than when no training was provided under neutral conditions (M=1.97, SE=.083). Again, however, with the predictions of hypothesis five, when no training was provided and leaders were asked to work under threatening conditions stronger visions were obtained (M=2.21, SE=.083).

Table 10 Analysis of covariance results for solution creativity.

F df p η2

Covariates Perceived problem difficulty (post questionnaire) 5.44 1, 180 .021 .03

Main effects Time pressure 11.17 1, 180 .001 .06 Threat .03 1, 180 .862 .00 Training .83 2, 180 .436 .01

Interactions Time pressure by threat .02 1, 180 .877 .00 Time pressure by training .94 2, 180 .390 .01 Threat by training 3.45 2, 180 .034 .04 Time pressure by threat by training .21 2, 180 .810 .00

Note: F = F ratio; df = degrees of freedom; p = significance level using Roy's largest root; η2 = effect size (eta squared).

2.3

2.4

2.5

2.6

2.7

2.8

2.9

No Pressure Pressure

Fig. 7. Effects of time pressure on creativity.

2.2

2.3

2.4

2.5

2.6

2.7

2.8

2.9

3

No Threat Threat

None

Informational

Social

Fig. 8. Effects of training and threat on creativity.

745J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

4. Discussion

Before turning to the broader conclusions flowing from the present study, certain limitations should be noted. To begin, the present study was based on an experimental paradigm. Although the leadership task which provided the basis for this study is engaging and can be meaningfully performed by undergraduates (Strange & Mumford, 2005), the question remains as to whether these findings can be generalized to working adult populations of experienced leaders.

It should also be recognized that in the present study only objective attributes of stress were examined. More specifically, time pressure was induced by reducing the time available to complete the task, and threat was induced through the content of the material presented in the performance exercise. Although objective manipulation of aspects of stress is, in the context of the present study, desirable, it should be recognized that the present study has little to say about the more subjective aspects of stress (Lazarus, 1976). Along related lines, it should also be recognized that only two objective aspects of stress, time pressure and threat, were examined. Thus the question remains as to how other objective stressors might operate. Furthermore, threat was only induced in terms of perceived threat, and no manipulation check for perceived threat was included in the study. Therefore, the actual degree of threat felt by participants is uncertain.

With regard to the instructional manipulations another, rather different, set of limitations apply. In the present study the informational and social strategies provided in training were presented as a “packaged” manipulation. As a result, the present investigation has little to say about the specific informational strategies or the specific informational strategies that had the largest effects on leader performance.

With regard to the training intervention, another limitation should be noted. In the present study, no attempt was made to train all the informational or social strategies that might prove useful in working with case-based knowledge in leader problem-solving. This point is of some importance because it implies that somewhat different effects might be observed if different strategies had been examined in the training manipulation. By the same token, it is also true that two strategies for working with each aspect of case-based knowledge identified by Mumford et al. (2007a) were trained. As a result, there is reason to suspect that the results obtained in the present study have some generality.

Finally, it should be recognized that the present study did not directly examine case-based knowledge, but instead the apparent use of case-based, schematic, and associational knowledge. In the training manipulations, strategies for working with select

Table 11 Analysis of covariance results for solution planning.

F df p η2

Covariates Engaged in scenario (post questionnaire) 6.76 1, 177 .010 .04 Divergent thinking 2.40 1, 177 .122 .01

Main effects Time pressure 10.68 1, 177 .001 .06 Threat .29 1, 177 .585 .00 Training .67 2, 177 .511 .01

Interactions Time pressure by threat .06 1, 177 .939 .00 Time pressure by training .53 2, 177 .586 .01 Threat by training 2.79 2, 177 .064 .03 Time pressure by threat by training .32 2, 177 .725 .00

Note: F = F ratio; df = degrees of freedom; p = significance level using Roy's largest root; η2 = effect size (eta squared).

1.85

1.9

1.95

2

2.05

2.1

2.15

2.2

2.25

No Pressure Pressure

Fig. 9. Effects of time pressure on solution planning.

746 J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

aspects of case-based knowledge held to be relevant to leaders' problem-solving efforts (Mumford et al., 2007a) were trained. Although prior studies by Strange & Mumford (2005) have provided evidence that case-based, or experiential, knowledge is used by leaders in problem-solving, because these knowledge structures were not manipulated directly, it is unclear as to whether these strategies were actually applied to case-based knowledge. In addressing this issue, however, we have provided evidence that participants that were trained on both sets of strategies did engage in the use of case-based knowledge in solving leadership problems, while not engaging in the use of schematic and associational knowledge. This finding suggests that the two types of training for working with case-based knowledge used in the present effort did lead to the use of case-based knowledge. Further, it was found that the use of case-based, schematic, and associational knowledge lead to higher performance on creativity, solution planning, and leader vision. Thus, although the training was found to only engage the use of case-based knowledge, when an individual employs any of the three types of knowledge it may still lead to subsequent increases in performance.

Even bearing these limitations in mind, we believe that the present study has some noteworthy theoretical and practical implications. Perhaps the first and most direct implication of the present study bears on the model of leader cognition proposed by Mumford et al. (2007a). This model of leader cognition draws a distinction between abstract informational aspects of case-based knowledge (e.g. causes, resources) and social contextual aspects of case-based knowledge (e.g. actors, affect). The finding that training in strategies for working with informational and social aspects of case-based knowledge exerted different effects on leader cognitive performance under certain conditions provides some support for a critical tenet of this model.

More centrally, our first two hypotheses held that training in strategies for working with either informational or social aspects of case-based knowledge would, given the model proposed by Mumford et al. (2007a), prove beneficial in enhancing leader cognitive performance. In fact, the findings obtained in the present study indicate that in comparison to a no training conditions where no threat was apparent, training in social strategies resulted in better performance across three key manifestations of leader cognition — creative problem-solving (Mumford et al., 2003), planning (Marta et al., 2005), and vision formation (Strange & Mumford, 2005). Moreover, under neutral, low threat, conditions cognitive training also proved beneficial with respect to creative problem-solving, planning, and vision formation.

This pattern of findings is noteworthy because it suggests that a viable way to improve leader cognition is to provide leaders with strategies for working with, or thinking about, the knowledge they have acquired with experience. In fact, strategy based

No Threat Threat

1.8

1.85

1.9

1.95

2

2.05

2.1

2.15

2.2

2.25

None

Informational

Social

Fig. 10. Effects of training and threat on solution planning.

Table 12 Analysis of covariance results for leader vision.

F df p η2

Covariates Perceived problem difficulty (post questionnaire) 5.83 1, 178 .017 .03 Engaged in scenario (post questionnaire) 5.73 1, 178 .018 .03

Main effects Time pressure 9.31 1, 178 .003 .05 Threat .14 1, 178 .707 .00 Training .59 2, 178 .552 .01

Interactions Time pressure by threat .02 1, 178 .894 .00 Time pressure by training .55 2, 178 .575 .01 Threat by training 3.27 2, 178 .040 .04 Time pressure by threat by training .28 2, 178 .753 .00

Note: F = F ratio; df = degrees of freedom; p = significance level using Roy's largest root; η2 = effect size (eta squared).

747J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

training has proven useful in a number of other areas of complex cognition, proving especially useful for enhancing performance when base knowledge has already been acquired (Scott, Leritz & Mumford, 2004). The results obtained in the present study point to the promise of this approach in enhancing leader cognition and multiple forms of leader cognition — creative problem-solving, planning, and vision formation.

In this regard, however, it is important to bear in mind our findings with regard to hypotheses six. Broadly speaking, this hypothesis held that cognitive strategies would prove less useful under conditions of threat due to their abstract nature while social strategies would continue to prove useful under conditions of threat as leaders searched for direct actions that might be taken with respect to others to minimize threat. The findings obtained in the present study supported this hypothesis. More centrally, it suggests that informational strategy training will prove more beneficial when leaders are not under threat while social strategy training will prove more beneficial when leaders must work under conditions of threat. Thus different training, or educational approaches, may be acquired when preparing leaders to work under crisis or high risk conditions (Yammarino, Mumford, Connelly & Dionne, 2010) — although training in social strategies proved beneficial with respect to leader cognition under conditions of both high and low threat.

Our foregoing observations about the effects of threat on leader cognition, however, brings to fore another set of implications of the present study. Here we refer to the implications of our findings with regard to Cognitive Resource Theory. Broadly speaking, Cognitive Resource Theory holds that stress disrupts leader cognition attenuating the relationship between intelligence and leader performance (Fiedler & Garcia, 1987; Judge et al., 2004). With regard to time pressure, and consistent with our initial hypotheses, time pressure was found to exert sizable negative effects on all three forms of leader cognition — creative problem-solving, planning, and vision formation. Thus, in this sense, our findings provide some support for this model. Moreover, these findings suggest that when organizations need leaders to think, they must be given time to think, and exceptional leader cognitive performance cannot be expected when time is short.

With this said, however, Cognitive Resource Theory is often read to say that cognitive instruction will prove of little value because leaders are often pressured for time (Mintzberg, 1973). Leaving aside the point that leaders might not always be pressured for time, the findings obtained in the present study indicate that time pressure does not interact with cognitive strategy training. Thus, time pressure does not rule out the value of at least some types of cognitive training. In fact, because people tend to rely on familiar strategies when solving problems under time pressure (Glaser & Bassock, 1989; Kaizer & Shore, 1995), it was not expected

1.85

1.9

1.95

2

2.05

2.1

2.15

2.2

2.25

No Pressure Pressure

Fig. 11. Effects of time pressure on leader vision.

No Threat Threat

1.8

1.85

1.9

1.95

2

2.05

2.1

2.15

2.2

2.25

None

Informational

Social

Fig. 12. Effects of training and threat on leader vision.

748 J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

that an interaction would be obtained between time pressure and the strategies provided in training. Our confirmation of this hypothesis, in turn, would lead one to conclude that cognitive based training, at least if it is strategy based training, will continue to prove of value even when leaders are placed under time pressure.

Our findings, however, make another perhaps more fundamental point. Stress is a complex construct (Muchinsky, 1993). In addition to time pressure stress may also be induced by threat. However, based on the findings of Ford & Gioia (2000) concerning the tendency of managers to initiate problem-solving in response to perceived threats, we hypothesized that better leader cognition would be observed when threat was evident due to greater investment of cognitive resources. In fact, the findings obtained in the present study confirmed this hypothesis indicating that under conditions of threat with no training better leader performance was observed with regard to creative problem-solving, planning, and vision formation.

One implication of this finding is that leader cognition might be improved by calling leaders attention to potential threats. The other implication of these findings, however, is that not all stressors necessarily undermine leader cognition. In fact, some stressors might at times prove beneficial. Thus the present study points to the need for further research examining how different types of stressors either act to inhibit or to enhance leader cognition. Because identification of these stressor specific effects may make it possible to design work environments likely to promote leader cognition, further work along these lines might have some practical value. We hope that the present investigation provides an impetus for further research along these lines.

Acknowledgements

We would like to thank Tamara Friedrich, Jay Caughron, Rich Marcy, and Alison Antes for their contributions to the present effort.

References

Bass, B. M. (1990). Handbook of leadership: A survey of theory and research. New York, NY: Free Press. Baughman, W., Mumford, M., & Sager, C. (1997). Measuring complex skills. Applied measurement methods in industrial psychology (pp. 261–288). Palo Alto, CA US:

Davies-Black Publishing. Becker, T. E., Billings, R. S., Eveleth, D. M., & Gilbert, N. W. (1997). Validity of scores on three attachment style scales: Exploratory and confirmatory evidence.

Educational and Psychological Measurement, 57, 477–493. Bedell-Avers, K. E., Hunter, S. T., & Mumford, M. D. (2008). Conditions of problem-solving and the performance of charismatic, ideological, and pragmatic leaders: A

comparative experimental study. The Leadership Quarterly, 19, 89–106. Berger, C., & Jordan, J. (1992). Planning sources, planning difficulty and verbal fluency. Communication Monographs, 59, 130–149. Briscoe, J. P., Hoobler, S. M., & Byle, K. A. (2010). Do “protean” employees make better leaders? The answer is in the eye of the beholder. The Leadership Quarterly, 21,

783–795. Byrne, C. L., Shipman, A. S., & Mumford, M. D. (2010). The effects of forecasting on creative problem-solving: An experimental study. Creativity Research Journal, 22,

119–138. Connelly, M. S., Gilbert, J. A., Zaccaro, S. J., Threlfall, K. V., Marks, M. A., & Mumford, M. D. (2000). Predicting organizational leadership: The impact of problem-solving

skills, social judgment skills, and knowledge. The Leadership Quarterly, 11, 65–86. Conway, J. M., & Peneno, G. M. (1999). Comparing structured interview question types: Construct validity and applicant reactions. Journal of Business and

Psychology, 13, 485–506. Davenport, M. (2003). Modeling motivation and learning strategy use in the classroom: An assessment of the factorial, structural, and predictive validity of the

motivated strategies for learning questionnaire. Dissertation Abstracts International Section A, 64, 394. Ericsson, K., & Charness, N. (1994). Expert performance: Its structure and acquisition. The American Psychologist, 49, 725–747. Fiedler, F., & Garcia, J. (1987). New approaches to effective leadership: Cognitive resources and organizational performance. Oxford England: John Wiley & Sons. Fleishman, E., & Harris, E. (1962). Patterns of leadership behavior related to employee grievances and turnover. Personnel Psychology, 15, 43–56. Ford, C., & Gioia, D. (2000). Factors influencing creativity in the domain of managerial decision making. Journal of Management, 26, 705–732. Forster, J., Higgins, E. T., & Bianco, A. T. (2003). Speed/accuracy decisions in task performance: Built-in trade-off or separate strategic concerns. OBHDP, 90, 148–164. Frisch, M., & Jessop, N. (1989). Improving WAIS—R estimates with the Shipley–Hartford and Wonderlic Personnel Tests: Need to control for reading ability.

Psychological Reports, 65, 923–928. Gioia, D. A., & Thomas, J. B. (1996). Identity, image, and issue interpretation: Sensemaking during strategic change in academia. Administrative Science Quarterly, 41,

370–403. Glaser, R., & Bassock, M. (1989). Learning theory and the study of instruction. Annual Review of Psychology, 40, 631–666. Goldberg, L. R. (1990). An alternative “description of personality”: The Big-Five factor structure. Journal of Personality and Social Psychology, 59, 1216–1229. Goldstein, H. (1986). Multilevel mixed linear model analysis using iterative generalized least squares. Biometrika, 73, 43–56. Gruszka, A., & Necka, E. (2002). Priming and acceptance of close and remote associations by creative and less creative people. Creativity Research Journal, 14,

174–192. Harris, K., Wheeler, A., & Kacmar, K. (2009). Leader-member exchange and empowerment: Direct and interactive effects on job satisfaction, turnover intentions,

and performance. The Leadership Quarterly, 20, 371–382. Hawkins, K., Faraone, S., Pepple, J., Seidman, L., & Tsuang, M. (1990). WAIS—R validation of the Wonderlic Personnel Test as a brief intelligence measure in a

psychiatric sample. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 2, 198–201. Hedlund, J., Forsynthe, G. B., Horvath, J. A., Williams, W. M., Snoot, S., & Sternberg, R. J. (2003). Identifying and assessing tacit knowledge: Understanding the

practical intelligence of military leaders. The Leadership Quarterly, 14, 117–140. Hunt, J., Boal, K., & Dodge, G. (1999). The effects of visionary and crisis-responsive charisma on followers: An experimental examination of two kinds of charismatic

leadership. The Leadership Quarterly, 10, 423–448. Hunter, S. T. (2011). First and ten leadership: A historiometric investigation of the CIP leadership model. The Leadership Quarterly, 22, 70–91. Hunter, S. T., Bedell-Avers, K. E., Hunsicker, C. M., Mumford, M. D., & Ligon, G. S. (2008). Applying multiple knowledge structures in creative thought: Effects on idea

generation and problem-solving. Creativity Research Journal, 20, 137–154. Irby, D. M., & Wilkinson, L. A. (2003). Educational innovations in academic medicine and environmental trends. Journal of General Internal Medicine, 18, 370–376. Isenberg, D. (1986). Thinking and managing: A verbal protocol analysis of managerial problem solving. Academy of Management Journal, 29, 775–788. Jacobs, T. O., & Jaques, E. (1990). Military executive leadership. In K. E. Clark & M.B. Clark (Eds.), Measures of leadership. West Orange, NJ: Leadership Library of

America. Judge, T., Colbert, A., & Ilies, R. (2004). Intelligence and leadership: A quantitative review and test of theoretical propositions. The Journal of Applied Psychology, 89,

542–552.

749J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750

Kaizer, C., & Shore, B. M. (1995). Strategy flexibility in more and less competent students on mathematical word problems. Creativity Research Journal, 8, 77–82. Kirk, R. E. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56, 746–759. Koberg, C. S., Uhlenbruck, N., & Sarason, Y. (1996). Facilitators of organizational innovation: The role of life-cycle stage. Journal of Business Venturing, 11, 133–149. Kolodner, J. L. (1997). Educational implications of analogy: A view from case-based reasoning. The American Psychologist, 52, 57–66. Lazarus, A. (1976). Multimodal behavior therapy: I. Oxford England: Springer. Lonergan, D. C., Scott, G. M., & Mumford, M. D. (2004). Evaluative aspects of creative thought: Effects of idea appraisal and revision standards. Creativity Research

Journal, 16, 231–246. Lord, R. G., & Hall, R. J. (2005). Identity, deep structure, and the development of leadership skill. The Leadership Quarterly, 16, 591–615. Marcy, R., & Mumford, M. (2007). Social innovation: Enhancing creative performance through casual analysis. Creativity Research Journal, 19, 123–140. Marcy, R. A., & Mumford, M. D. (2010). Leader cognition: Improving leader performance through causal analysis. The Leadership Quarterly, 21, 1–19. Marta, S., Leritz, L. E., & Mumford, M. D. (2005). Leadership skills and group performance: Situational demands, behavioral requirements, and planning. The

Leadership Quarterly, 16, 97–120. McClendon, R. (1996). Motivation and cognition of preservice teachers: MSLQ. Journal of Instructional Psychology, 23, 216–220. McKelvie, S. (1989). The Wonderlic Personnel Test: Reliability and validity in an academic setting. Psychological Reports, 65, 161–162. McKenna, B., Rooney, D., & Boal, K. (2009). Wisdom principles as a meta-theoretical basis for evaluating leadership. The Leadership Quarterly, 20, 177–190. Merrifield, P. R., Guilford, J. P., Christensen, P. R., & Frick, J. M. (1962). The role of intellectual factors in problem solving. Psychological Monographs, 76, 1–21. Mintzberg, H. (1973). The nature of managerial work. New York: Harper & Row. Muchinsky, P. (1993). Psychology applied to work: An introduction to industrial and organizational psychology (4th ed.). Belmont, CA US: Thomson Brooks/Cole

Publishing Co. Mumford, M. D. (2006). Pathways to outstanding leadership: A comparative analysis of charismatic, ideological, and pragmatic leadership. Mahulah, NJ: Erlbaum. Mumford, M. D., & Gustafson, S. B. (2007). Creative thought: Cognition and problem solving in a dynamic system. In M. A. Runco (Ed.), Creativity research handbook:

Volume II. Cresskill, NJ: Hampton. Mumford, M. D., & Norris, D. G. (1999). Heuristics. In M. A. Runco & S. Pritzker (Eds.), Encyclopedia of creativity: Volume II (pp. 139–146). San Diego, CA: Academic. Mumford, M., Zaccaro, S., Harding, F., Jacobs, T., & Fleishman, E. (2000). Leadership skills for a changing world: Solving complex social problems. The Leadership

Quarterly, 11, 11–35. Mumford, M. D., Schultz, R. A., & Van Doorn, J. R. (2001). Performance in planning: Processes, requirements, and errors. Review of General Psychology, 5, 213–240. Mumford, M., Connelly, S., & Gaddis, B. (2003). How creative leaders think: Experimental findings and cases. The Leadership Quarterly, 14, 411–432. Mumford, M. D., Friedrich, T., Caughron, J., & Byrne, C. (2007a). Leader cognition in real-world settings: How do leaders think about crises? Leadership Quarterly, 18,

515–543. Mumford, T. V., Campion, M. A., & Morgeson, F. P. (2007b). The leadership skills stratplex: Leadership skill requirements across organizational levels. The Leadership

Quarterly, 18, 154–166. Mumford, M. D., Friedrich, T. L., Caughron, J. J., & Antes, A. (2009). Leadership research: Traditions, developments and current directions. In D. A. Buchanan & A.

Bryman (Eds.), Handbook of organizational research methods. Thousand Oaks, CA: Sage Publications Ltd. Naidoo, L. J., Kohari, N. E., Lork, R. G., & Dubois, D. A. (2010). ‘Seeing’ is retrieving: Recovering emotional content in leadership ratings through visualization. The

Leadership Quarterly, 21, 886–900. Nutt, P. C. (1989). Making tough decisions: Tactics for improving managerial decision making. San Francisco, CA: Jossey-Bass. Parker, D., & DeCotiis, T. (1983). Organizational determinants of job stress. Organizational Behavior and Human Performance, 32, 160–177. Patalano, A. L., & Siefert, C. M. (1997). Opportunistic planning: Being reminded of pending goals. Cognitive Psychology, 34, 1–36. Pintrich, P. R., Smith, D. A. F., García, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ).

Educational and Psychological Measurement, 53, 801–813. Redmond, M., Mumford, M., & Teach, R. (1993). Putting creativity to work: Effects of leader behavior on subordinate creativity. Organizational Behavior and Human

Decision Processes, 55, 120–151. Reeves, L., & Weisberg, R. (1994). The role of content and abstract information in analogical transfer. Psychological Bulletin, 115, 381–400. Reysen, S. (2005). Construction of a new scale: The Reysen likability scale. Social Behavior and Personality, 33, 201–208. Rodan, S. (2002). Innovation and heterogeneous knowledge in managerial contact networks. Journal of Knowledge Management, 6, 152–163. Saucier, G. (2002). Orthogonal markers for orthogonal factors: The case of the Big Five. Journal of Research in Personality, 36, 1–31. Scott, G., Leritz, L., & Mumford, M. (2004). The effectiveness of creativity training: A quantitative review. Creativity Research Journal, 16, 361–388. Scott, G. M., Lonergan, D. C., & Mumford, M. D. (2005a). Contractual combination: Alternative knowledge structures, alternative heuristics. Creativity Research

Journal, 17, 21–36. Scott, G., Lonergan, D., & Mumford, M. (2005b). Conceptual combination: Alternative knowledge structures, alternative heuristics. Creativity Research Journal, 17,

79–98. Shipman, A., Byrne, C. L., & Mumford, M. D. (2010). Leader vision and forecasting: The effects of forecasting extent, resources, and timeframe. The Leadership

Quarterly, 21, 439–456. Skinner, E. (1969). Relationships between leadership behavior patterns and organizational-situational variables. Personnel Psychology, 22, 489–494. Strange, J. M., & Mumford, M. D. (2005). The origins of vision: Effects of reflection, models, and analysis. The Leadership Quarterly, 16, 121–148. Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics. Needham Heights, MA: Allyn & Bacon. Thomas, J. B., & McDaniel, R. R. (1990). Interpreting strategic issues: Linkages among scanning, interpretation, action, and outcomes. Academy of Management

Journal, 34, 239–270. Tulving, E. (1972). Episodic and semantic memory. In E. Tulving & W. Donaldson (Eds.), Organization of memory. New York: Academic. Vincent, A. H., Decker, B. D., & Mumford, M. D. (2002). Divergent thinking, intelligence, and expertise: A test of alternative models. Creativity Research Journal, 14,

163–178. Ward, T. B., Patterson, M. J., & Sifonis, C. M. (2004). The role of specificity and abstraction in creative idea generation. Creativity Research Journal, 16, 1–9. Weick, K. E. (1995). Sensemaking in organizations. Thousand Oaks, CA: Sage. Yammarino, F. J., Mumford, M. D., Connelly, M. S., & Dionne, S. D. (2010). Leadership and team dynamics for dangerous. Journal of Military Psychology, 22, 15–41. Yukl, G. (2010). Leadership in organizations. Englewood Cliffs, NJ: Prentice Hall.

750 J.D. Barrett et al. / The Leadership Quarterly 22 (2011) 729–750