Article Component Papers (PERSPECTIVES ON EFFECTIVE LEADER BEHAVIOR)

Schriesheimetal.1995.LQ.6.111-145.pdf

Home >Business & Finance homework help >Management homework help >Article Component Papers (PERSPECTIVES ON EFFECTIVE LEADER BEHAVIOR)

IS IT “TRUSTWORTHY”? A MULTIPLE-LEVELS-OF-ANALYSIS

REEXAMINATION OF AN OHIO STATE LEADERSHIP STUDY, WITH IMPLICATIONS

FOR FUTURE RESEARCH

Chester A. Schriesheim* Claudia C. Cogliser

Linda L. Neider University of Miami

A moderator subgroup study, originally published by Schriesheim and Murphy (1976), is reexamined using the within-group agreement coefficient (rr+~; James, Demaree, & Wolf, 1984), moderated raw-

score multiple linear regression, and moderated within- and betweenentity analysis (WABA,

Dansereau, Ahttto, & Yammarino, 1984; Schriesheim, forthcoming). The results show serious

discrepancies in results between the subgroup and regression approaches, the need to exercise care in

interpreting the r~c and raw-score regression results, and the necessity of conducting WABA and

supplemental withinsntity WABA moderation analyses to gain meaningful insight into the phenomena

being investigated. Implications and future directions for leadership theory and research are considered.

INTRODUCTION

Twenty years ago, leadership research had emerged from its “behavioral phase” and was moving toward increased sophistication in theory and measurement (Yukl & Van Fleet, 1992). Situational leadership approaches, such as House’s (1971) path-goal theory, were well established and being subjected to critical analysis and refinement

* Direct all correspondence to: Chester A. Schriesheim, Department of Management, School of Business Administration, University of Miami, 414 Jenkins Building, Coral Gables, FL 33124-6548.

Leadership Quarterly, 6(2), 1 I l-145.

All rights of reproduction in any form reserved. ISSN: 1048-9843

112 LEADERSHIP QUARTERLY Vol. 6 No. 2 1995

(e.g., House & Dessler, 1974; House & Mitchell, 1974), and major research measures, such as the Ohio State leadership scales (e.g., Stogdill, 1963), were also being subjected to serious psychometric examination (e.g., Schriesheim & Stogdill, 1975) and improvement (e.g., Schriesheim, 1978).

In the midst of all this activity, the first author and a colleague (Schriesheim & Murphy, 1976) published a study testing four hypothesized situational moderators (unit size, job anxiety, role clarity, and supervisory consideration) of relationships between perceived initiating structure (task-oriented and directive supervisory behavior) and consideration (friendly and interpersonally supportive supervisory behavior; Stogdill, 1963) and subordinate satisfaction and performance. The hypotheses that were examined had been derived from an extensive review of the literature (Kerr, Schriesheim, Murphy, & Stogdill, 1974), and the leadership measures used were taken from the most psychometrically sound version of the Ohio State leadership scales (Schriesheim & Bird, 1979; Schriesheim & Kerr, 1977)-Form XII of the Leader Behavior Description Questionnaire (the LBDQ-XII asks respondents to describe the behavior of their supervisor toward the work group or unit of which they are a member; Stogdill, 1963). Following common convention and the recommendations of the editor and reviewers, moderator subgroup analysis was employed to test for situational moderation, which was found for all but the role clarity variable. (As commonly practiced, moderator subgroup analysis involves first determining a “cutting score”- typically the median value in the sample-then partitioning respondents into “low” and “high” moderator subgroups and correlating the independent and dependent variables within the subgroups separately; Zedeck, 1971).

While neither particularly well nor poorly done, the Schriesheim and Murphy (1976) investigation has been cited in various reviews of the literature over the years (e.g., Bass, 1990; Bryman, 1986; Miner, 1980) and, given the substantial progress in the field (House, 1988), the current authors thought it would be both interesting and informative to examine whether conclusions similar to those reached 20 years ago would be reached today. Thus, this study uses a contemporary and multi-method approach to explore the situational moderating effects of role clarity on relationships between supervisory initiating structure and consideration and subordinate satisfaction and performance. Role clarity was selected for examination in the current investigation for three reasons. First, it is potentially both an individual- and a group-level variable. Second, role clarity was the only situational moderator that was not generally supported in the Schriesheim and Murphy study. (Of the four hypothesized role clarity moderator relationships that were tested, only one effect was uncovered, and this was directionally opposite to what was hypothesized.)’ Finally, the moderator analysis approach currently used in most research (moderated multiple regression) should be both more powerful in detecting situational moderation and less prone to error than the subgrouping approach employed by Schriesheim and Murphy. (The greater power of moderated multiple regression arises partly from the fact that it treats moderators as continuous variables, thereby retaining information that is lost in subgroup analysis when the moderator is transformed into a binary variable representing “1ow”versus “high” subgroup membership [cf. Cohen & Cohen, 1983; Zedeck, 19711. Additionally, the subgrouping technique has other shortcomings, including being prone to making Type I errors if the ratio of independent to dependent variable variances differs across subgroups [Arnold, 1982; Peters & Champoux, 1979; Peters, O’Connor, & Wise, 19841.)

Multi-Method Examination 113

BACKGROUND

In addition to adopting moderated multiple regression as the dominant method of testing situational moderator effects in nonexperimental research, many advances have occurred in the field since publication of the Schriesheim and Murphy article. Most directly pertinent to the current study are the recent increased interest in formulating hypotheses, models, and theories that involve multiple or cross levels of analyses (e.g., House, 1991) and the increased realization that it is important to first specify and then appropriately examine and test a phenomenon at its hypothesized level(s) of analysis (Dansereau, Alutto, & Yammarino, 1984; Glick & Roberts, 1984; Klein, Dansereau, & Hall, 1994; Mowday & Sutton, 1993; Roberts, Hulin, 8z Rousseau, 1978; Rousseau, 1985; Yammarino, forthcoming a, forthcoming b).

In the early and mid-1970s, levels-of-analysis issues were usually not explicitly treated in leadership research. Thus, it is not surprising that Schriesheim and Murphy did not discuss or justify their level of analysis. Instead, they simply analyzed their data using the individual (unaggregated) raw scores of their respondents. Although much research of the era was conducted in a similar manner (for reviews and discussion about these analytic practices, see Dansereau, Alutto, Markham, & Dumas, 1982; Dansereau 8z Dumas, 1977; Schriesheim, 1979), two serious problems are now apparent. First, as suggested by the work of Dansereau and Dumas (1977), Schriesheim (1979), and others (e.g., Dansereau et al., 1982; Dansereau, Alutto, & Yammarino, 1984; Yammarino, 1990), using the LBDQ-XII and then ignoring that scale’s group measurement referent resulted in a serious misalignment of theory and data. While Schriesheim and Murphy’s underlying theorization concerned individual-directed leader behavior, the behavior that was actually measured concerned “group-0riented”leadership. Thus, although high correlations frequently exist between respondents’ descriptions of their supervisor’s behavior toward them (as individuals) and toward their work group (as a unit), empirical similarity is not the same as theoretical isomorphism (Schriesheim, 1979; Yammarino, 1990). Second, and as recently emphasized by Klein et al. (1994), Schriesheim and Murphy’s treatment of their data involves the implicit levels-of-analysis assumption that perceived leader behavior-subordinate satisfaction and performance relationships are independent of group effects (i.e., that obtained relationships hold both within and between groups), so work unit membership is not relevant in explaining those relationships that were examined. Schriesheim and Murphy did not perform any tests to support their implicit level-of-analysis assumption, and this is particularly troublesome both because of the LBDQ-XII’s group-centered measurement referent and because Schriesheim and Murphy’s data were collected from geographically dispersed (and relatively decentralized) subunits of a national black social services organization. As Klein et al. (1994) and others (e.g., Rousseau, 1985) point out, this sampling practice is quite appropriate for research that seeks to explore differences between groups, since it might be expected to enhance such effects (possibly at the expense of within-group differences). However, by not testing for the presence of within- and between-group effects in their data, Schriesheim and Murphy produced a study that must be viewed today as “uninterpretable’‘-we simply do not know whether the relationships they report held regardless of the respondents’ group/unit membership or whether their “findings” are actually better viewed as occurring at some other level

114 LEADERSHIP QUARTERLY Vol. 6 No. 2 1995

of analysis (e.g., within groups or between groups; Klein et al., 1994). Since the same is likewise true for much of the published empirical leadership literature (cf. Dansereau

et al., 1982; Dansereau, Alutto, & Yammarino, 1984; Dansereau & Dumas, 1977), the

current investigation should also facilitate an enhanced understanding of the likely

interpretability or “trustworthiness” of this accumulated body of evidence (for such uses

as future hypothesis generation and theory construction).

TWO APPROACHES TO TESTING FOR GROUP EFFECTS

With the work of Dansereau, Alutto, and Yammarino (1984), Klein et al. (1994), and

others (e.g., Yammarino, forthcoming a, forthcoming b), it now appears to be

increasingly accepted that researchers should test their data for alignment with the

level(s) of analysis that they hypothesize or theorize. However, different approaches

have been proposed (and used) to conduct such tests, and recently a heated exchange

has ensued between the proponents of the rWG within-group similarity or agreement

index (James, Demaree, & Wolf, 1984) and within- and between-entity analysis

(WABA; Dansereau et al., 1984) over the appropriateness of each for assessing whether

data collected from individual respondents can be aggregated to the level of work groups

or units (e.g., George & James, 1993; Yammarino & Markham, 1992). Since the

Schriesheim and Murphy study may be reconceptualized by specifying that their

hypothesized relationships could occur at (1) only the work group level (i.e., between

groups), (2) only the level of within work groups, or possibly (3) both within and between

groups (i.e., as independent of work groups), both the rWG and WABA approaches are

applied to reanalyzing the Schriesheim and Murphy data. However, we should mention

that because the purpose of this article is not to fuel the rwG-WABA debate, advantages

and disadvantages of both approaches will not be discussed in depth, and interested

readers are instead referred to George (1990), George and James (1993), Kozlowski

and Hattrup (1992) Schriesheim and Charnes (1995) Schriesheim (forthcoming),

Yammarino (forthcoming a, forthcoming b), and Yammarino and Markham (1992)

for more detailed treatments of this controversy.

The fwG Within-Group Agreement Coefficient

The rwo within-group similarity or agreement index was developed by James et al.

(1984) for assessing within-group agreement when two or more individuals (“judges”)

complete ratings on one or more items (“targets”) to describe a common referent (e.g., their work group). The Appendix presents computational specifics for this coefficient;

the logic underlying it is that before scores that are obtained from individuals can be

meaningfully aggregated and assigned to a higher level of analysis (e.g., as indicative of a work group or unit), agreement must be shown among those individuals in terms

of the attitude or behavior that they have described. As noted elsewhere, this appears quite reasonable, particularly since “virtually all small group theorists emphasize that, to reasonably be called a ‘group,’ any collection of people must share certain attributes, one of which is common perception of key elements in their environment”(Schriesheim,

forthcoming).

Multi-Method Examination 115

Criticisms of f wG The rWG coefficient may be particularly useful when traditional analysis-of-variance

assumptions that underlie WABA (such as homogeneity of variance) are violated. However, one serious problem associated with using this index has been that of determining the statistical significance of obtained values (since the sampling distribution of TWG for various null distributions is not known; James et al., 1984). Research using rwG has typically employed a .7 rule of thumb to judge whether obtained values are sufficient to justify the aggregation of data (e.g., George, 1990) and, more recently, a Monte Carlo simulation has been developed to estimate quantiles of the rWG sampling distribution (for testing whether obtained values may be assumed to be significantly greater than zero; Charnes & Schriesheim, forthcoming). This still leaves other issues about rWG unresolved, including whether it is a “1enient”statistic (i.e., values significantly greater than zero are easily obtained) and how rwG values should be summarized (an rwG coefficient is computed to assess agreement within each group; thus, to draw an overall conclusion about aggregation across multiple groups, some summary analysis or rule of thumb is needed) (Schriesheim & Charnes, 1995). Finally, to use rWG, several requirements specified by James et al. (1984) must be met. These include: the measures that are used must “have acceptable psychometric properties;” they should obtain approximately equal-interval measurement (James et al., 1984, p. 85); and there should be empirical evidence supporting the null distribution that is employed (James et al., 1984, pp. 93-94). These latter three requirements are often not addressed in research that uses the rWG coefficient, but each has been dealt with in the current research and is briefly discussed further in the “Measures” and “Methods of Analysis” sections below.

Within- and Between-Entity Analysis (WABA)

WABA is based on well-known analysis of variance statistical principles, and it includes a carefully developed set of guides to aid interpretation and integration back to theory (Dansereau et al., 1984). Probably its most radical departure from traditional data-analytic techniques (such as correlation and regression) is that these traditional methods use only one score (the raw score) for each respondent and that they do not test a phenomenon’s level of analysis (to determine where the phenomenon is more likely to be occurring). On the other hand, WABA was developed by Dansereau and his colleagues to overcome these shortcomings and to help assess whether empirical findings should be seen as occurring within entities, between entities, both, or neither (null or nonoperative).

Detailed computational specifics on WABA are presented in the Appendix, but WABA can basically be conceptualized as involving a series of analytic steps. First, in WABA I, a study’s variables are assessed to determine their relative amounts of betweenentities variation (e.g., between work groups or units) (indicative of between- entity heterogeneity) and within-entities variation (indicative of within-entity heterogeneity). Second, WABA II examines the relationships among the variables to determine the amounts of between- and within-entities covariance. Finally, overall conclusions are reached by integrating the WABA I and II results and by decomposing the raw-score correlations into within- and betweenentities components (Dansereau

116 LEADERSHIP QUARTERLY Vol. 6 No. 2 1995

et al., 1984; Yammarino & Markham, 1992, pp. 171-172). (As part of the third step, supplemental tests may be conducted, such as those for within-entities moderation; cf. Schriesheim, forthcoming).

Criticisms of WABA Despite being readily accessible for over a decade (Dansereau et al., 1984), WABA

appears to be just now gaining increased attention and use from leadership and other scholars (although a “core” of researchers have been actively using WABA since its inception; cf. Yammarino, forthcoming a, forthcoming b). As part of this new wave of interest in WABA, refinements in its conceptual distinctions (e.g., Klein et al., 1994), critiques of its assumptions (e.g., George &James, 1993), and extensions of its statistical underpinnings (e.g., Schriesheim, forthcoming) are beginning to appear.

Although a number of criticisms may be leveled at WABA (cf. George, 1990; George & James, 1993), there are two that we view as most problematic. First, as George and James (1993) correctly point out, range restriction across group scores may lead to erroneous WABA I conclusions (since this leads to the conclusion that groups are not present), and this may be a frequent occurrence since groups are often nested within a larger entity (e.g., an organization)-which would tend to restrict variance in between- group scores. In the present study we do not believe that this is a major concern since, as mentioned earlier, Schriesheim and Murphy’s sample of respondents in decentralized and geographically diverse subunits of a national organization would most probably not yield variance restriction as severe as that which typically occurs in most single- sample studies.

The second major criticism of the WABA approach is that having significantly more between- than within-groups variance does not necessarily support a group level of analysis, since a shared perspective among group members (i.e., within-group agreement) is usually seen by small-group theorists (e.g., Hare, 1976; Shaw, 1981) as necessary to empirically support the existence of a group. We basically agree with this position and, since there is no reason why the James et al. (1984) within-group agreement coefficient cannot be used along with WABA (Dansereau et al., 1984), the current reanalysis of Schriesheim and Murphy uses both approaches and then compares the two. However, before turning to considering theory and research that supports the levels of analysis which the current study examines, it is perhaps useful to mention briefly that should a conflict exist between rWG and WABA results, we believe that WABA findings should be given greater weight in drawing conclusions. This is simply because if between-groups variance is small (WABA I), between-groups covariance is also going to be small (WABA II), and between-groups relationships cannot be empirically manifested. Thus, WABA as an overall data-analytic system speaks directly to the likely practical relevance of potential group-level attributes (i.e., their empirical usefulness), something that the assessment of within-group agreement alone does not do (and something that, of course, cannot occur if between-groups variance is restricted).

LEVELS OF ANALYSIS AND RESEARCH HYPOTHESES

Although the proponents of the opposing rWG and WABA approaches for testing the appropriateness of group-level aggregation differ with respect to their preferred analytic

Multi-Method Examination 117

strategies, both agree that support is needed justifying how variables are treated with respect to their hypothesized levels of analysis (e.g., George, 1990; Yammarino & Markham, 1992). It should be noted that supporting both the within- and between- groups perspectives would buttress Schriesheim and Murphy’s implicit treatment of variables (as independent of group effects) and that there is both theory and research that supports conceptualizing perceived leader behavior and subordinate performance, satisfaction, and role clarity at both the within- and between-groups levels of analysis. This literature is reviewed below. However, the leadership literature is reviewed in a bit more depth, simply because a number of theorists have very forcefully argued for the primacy (or “correctnessn) of either a within-groups (e.g., Graen & Cashman, 1975, p. 150) or a between-groups (e.g., Seeman, 1957, p. 95) perspective in examining leadership phenomena.

levels of Analysis in the Leadership Variables Examined

Although it might be tempting to argue for treating leadership as either a within- or between-groups phenomenon, we believe that existing theory and research actually support leadership as a phenomenon that is potentially operative at either or both levels of analysis. For example, in the small-groups area, it has long been recognized that groups influence their members by providing both ambient (constant to all group members) and discretionary (variable and individually directed) stimuli on a whole range of cognitive, affective, and motivational variables (such as task performance strategies, interpersonal support, etc.; Hackman, 1992). One could clearly argue the same for supervisory leadership.

Support for a Between-Groups Conceptualization of leadership Seeing leadership as a between-groups or work units phenomenon basically amounts

to treating leader behavior as a “style,” something that is seriously out of vogue with most current conceptualizations of the leadership process. It must be noted, however, that the notion of styles or consistencies in behavioral patterns (across situations, other persons, etc.) is pervasive in many areas of psychology. In fact, the fields of personality and clinical psychology are largely predicated upon the existence of such general behavioral patterns or styles (cf. Hogan, 1991). Additionally, generalized patterns of leader behaviors or styles would also be predicted from other approaches. For example, equity theory (e.g., Adams, 1963) would predict at least some general behavioral consistencies from supervisors, arising from such causes as the need to maintain equity among work unit members and limitations in the resources (e.g., time, energy, etc.) available to supervisors for them to tailor different behaviors toward different subordinates (Schriesheim, Mowday, & Stogdill, 1979). Also, as suggested by Cummings (1975), even if individuals are treated very differently by a supervisor, they may still perceive such behavior in a stylistic manner, for reasons including general or habituated responses (i.e., styles of subordination).

Empirical support for the viability of group-level treatments of leadership phenomena is quite substantial. It was shown in the early Ohio State Leadership Studies that reasonable agreement often exists among subordinates in describing their supervisors’ leadership behavior. In these studies, interrater correlations were typically between .50

118 LEADERSHIP QUARTERLY Vol. 6 No. 2 1995

and .70, with the median falling around .60 (e.g., Fleishman, Harris, & Burtt, 1955; Halpin, 1957a, 1957b). Fleishman (1957), for example, reported eight coefficients for four different samples and two leadership styles; the range was from .47 to .72, with a median of 64. Furthermore, examining within- versus between-groups effects by analysis of variance (essentially, “WABA I”), the Stogdill and Coons (1957) monograph alone presents five different studies (some of which are multi-sample) which obtained significantly more between- than within-groups variance in the leader behavior descriptions provided by subordinates (Fleishman, 1957; Halpin, 1957a, 1957b; Halpin & Winer, 1957; Hemphill & Coons, 1957). Other, non-Ohio State research has yielded similar results (e.g., Evans, 1972; Jago & Vroom, 1975), and substantial evidence also exists that leaders do affect group processes (cf. Bass, 1960, 1990; Schriesheim et al., 1979). For example, early research found leadership styles to affect group cohesiveness and arousal toward task-goal accomplishment (e.g., Lewin, Lippitt, & White, 1939) and these findings have also been supported by more recent studies (e.g., Greene & Schriesheim, 1980).

Support for a Within-Groups Conceptualization of Leadership As with the extant theory and research about leadership as a group-level

phenomenon, substantial support exists for viewing leadership as a dyadic or within- groups phenomenon. Dyadic theoretical analyses have been shown to be useful in explaining a whole range of interpersonal behaviors (cf. Hogan, 1991; Kanfer, 1990). In addition, many specific theories are congruent with the tenets of a dyadic approach. For example, expectancy theory (e.g., Vroom, 1964) would predict that leaders will behave differentially toward different subordinates in their work unit, depending upon how instrumental each is to furthering his or her desired goals or outcomes. Many theories in social psychology would also predict within-unit variations in leader behavior toward different subordinates, based upon such things as personality, congruence in held attitudes, and differences in person-specific attributions (Hogan, 1991; Kanfer, 1990).

As for the theoretical support of the within-groups approach, the empirical evidence also seems impressive. We know from experimental studies that the behavior of leaders can be flexible (e.g., Hill, 1973) and that certain situational contingencies (e.g., differences in subordinate competence) can cause leaders to act very differently toward different subordinates within their work units (e.g., Lowin & Craig, 1968). Evidence from field studies is also abundant and supportive of a dyadic approach. For example, in an early Ohio State study, Seeman (1957) found that initiating structure was significantly and negatively correlated with years of job experience and years under the supervisor (but not with “status”). Seeman concluded that leaders adjust their structuring behavior to fit a subordinate’s experience (relative to other subordinates). Similar findings are reported in other studies in the same volume (Stogdill & Coons, 1957). Hemphill and Coons (1957) for example, found perceptions of four different types of leader behavior to be significantly and negatively related to time spent under the leader. Finally, research on the “Vertical Dyad Linkage”(VDL) or “Leader-Member Exchange” (LMX) model has yielded replicated findings supporting the usefulness of viewing leadership from a within-groups or dyadic perspective (Graen & Scandura, 1987).

Multi-Method Examination 119

Summary on leadership The theoretical and empirical evidence summarized briefly above provides subst~ti~

support for the viability of viewing leadership as both a within-groups and a between- groups phenomenon. Intuitively, it seems unrealistic to suppose that leaders will tailor all of their behavior to each individual subordinate or that they interact with all work group members on the basis of some generalized style. This view, of course, is not new and Cummings (1975, p. 184) perhaps stated it best by remarking that the assumption of heterogeneity (all individu~ly oriented leade~hip behavior) “is equally as unrealistic as homogeneity.” In other words, leaders’ behaviors will usually contain variance attributable to style (between-groups effects) and variance attributable to individual tailoring (within-groups effects). This perspective is shared by Campbell (1977, p. 227), who suggested that dyadic approaches to leadership “should be taken very seriously while at the same time remembering there is undoubtedly a main effect due to groups,” and it is directly supported by empirical evidence showing that LBDQ-XII initiating structure items have both meaningful within-and between-groups variance (Markham & Scott, 1983).

Levels of Analysis for Other Study Variables

While the pe~o~ance of indi~du~ subordinates may be conceptu~~ed as a within- groups variable or as a variable that is independent of group membership (as implicitly done by Schriesheim and Murphy), it is operationalized in this study (as in many) through supervisory performance ratings. Thus, factors such as the politics of appraisal and pay systems, as well as various rater biases (e.g., leniency or harshness), may produce a substantial between-groups effect, since different supervisors supervise different work groups and probably utilize different frames of reference when completing performance ratings for their subordinates. Evidence in the performance appraisal domain is highly supportive of this possibility (cf. Borman, 1991; Guion, 1991), as is prior research that has used WABA to uncover such effects in supervisory performance appraisals (e.g., Markham, 1988; Yammarino, Dubinsky, & Hartley, 1987). Thus, rather than assuming that the performance variable of the Schriesheim and Murphy study is best treated as a within- and/or a between-groups variable, the current reanalysis explores both possibilities.

With respect to general or overall job satisfaction, current treatments do not normally deal with it as a between-groups variable (cf. Locke, 1976). However, older leadership research often did (Kerr & Schriesheim, 1974), frequently correlating aggregated (to the work group level) leadership and job satisfaction (or “morale”) measures and finding significant results (e.g., Campbell, 1956; Katz, Maccoby, & Morse, 1950). This, coupled with recent research on group-level affect (e.g., George, 1990), suggests the prudence of examining both within- and between-groups effects for job satisfaction and not implicitly assuming that either level must be nonoperative.

Finally, while role clarity has usually been conceptualized and treated in leadership research as being independent of the respondent’s work group (e.g., House, 1971), a more macroscopic view (e.g., Etzioni, 1964) would probably argue that individuals in the same work units (who are performing similar jobs) are likely to share a common view of job-relevant attributes (such as role clarity). Thus, from a more sociological

120 LEADERSHIP QUARTERLY Vol. 6 No. 2 1995

perspective (e.g., Perrow, 1970), treating role clarity as a potential between-groups construct is not unfounded. Additionally, while there is very limited direct empirical support for such treatment, Yammarino (1990) did obtain significant WABA I results (F = 4.18, df = 12,41; p 5 .Ol) indicating more between- than within-group variance in the same role clarity measure as used by Schriesheim and Murphy.

Research Hypotheses

Schriesheim and Murphy hypothesized:

The more subordinates perceived their work roles to be clearly defined, the less positive

would be the relationship between leader structure and subordinate satisfaction and performance, and the more positive would be the relationship between leader consideration and subordinate satisfaction and performance (p. 636).

We have no reason to expect any different pattern of moderation (the Schriesheim and Murphy article discusses the logic underlying these predicted relationships). However, based upon our using a more powerful data-analytic technique (moderated multiple regression) and the recognition that leader behavior-subordinate satisfaction and performance relationships can probably vary both within and between groups, we would expect to find support for both hypotheses at both levels of analysis (that is, of course, what Schriesheim and Murphy’s implicit level-of-analysis assumption also mandates).

METHOD

Additional specifics concerning the sample and measures used in this study may be found in Schriesheim and Murphy. However, the following brief descriptions should help the reader follow the analyses and results that are subsequently reported.

Sample and Procedure

Sample As mentioned briefly above, the sample consisted of first-line supervisors randomly

selected from 19 decentralized and geographically dispersed units of a national black social services organization (each respondent reported directly to his/ her unit’s head). Due to the necessity of having to match respondents to organizational units, six of Schriesheim and Murphy’s original sample of 54 had to be excluded, yielding a final sample of 48 from 18 different units (6 of the units were each represented by 2 respondents, while the remaining 12 had 3 each).

These 48 supervisors were demographically quite similar to the 54 examined by Schriesheim and Murphy. Most had completed four or more years of college training (M = 15.8; SD = .8) and had a substantial amount of experience in their current job (M = 4.4; SD = 2.6) and their profession (M = 7.3; SD = 4.2). The average age was 37.8 (SD = 4.8); 67% were male; 76.6% were currently married; they had an average number of 2.6 children (SD = 1.9); and the average size of the 18 units from which they came was 34.8 total employees (SD = 10.2). Most of these demographic statistics

Multi-Method Examination 121

are virtually identical to those reported by Schriesheim and Murphy; in fact, all absolute mean differences are no greater than 0.1.

Under the assurance of confidentiality and university research sponsorship, the respondents completed a survey questionnaire that contained all of the measures discussed below-with the exception of the performance measure. That instrument was completed by the head of each unit.

Measures

Most of the measures used in this research meet the James et al. (1984) rwG requirements for psychometric quality, equal-interval response categories, and empirically supported null distributions. As mentioned in the “Methods of Analysis” section below, archival data were available for all but the performance measure, enabling the development of empirically based null distributions. Additionally, and as mentioned immediately below, all but the performance measure are well-known and reasonably well-supported scales that also use response categories that are known to approximate equal psychological intervals.

Perceived leader Behavior As already mentioned, Schriesheim and Murphy used the IO-item (each) initiating

structure and consideration subscales of the LBDQ-XII (Stogdill, 1963). Detailed reviews of these measures are provided by Schriesheim and Kerr (1974, 1977); however, it seems worthwhile to note briefly that the LBDQ-XII scales are the soundest available measures of initiating structure and consideration (Schriesheim & Bird, 1979) and that they use a five-point roughly equal-interval frequency response scale (always, often, occasionally, seldom, and never; cf. Schriesheim & Schriesheim, 1974).

Performance Performance was measured by a 17-item evaluation form that was derived from a

job analysis and that was completed by the 18 supervisors of the 48 respondents. The items assessed quantity, quality, and speed of response to task demands, and responses were recorded on a seven-point scale (excellent, very good, good, average, fair, poor, and inadequate). A principal axis factor analysis (with squared multiple correlations as communality estimates) of the 17 items yielded eigenvalues of 10.39, 1.37, and 0.87 for first three factors and, using an oblimin rotation, the correlation between factors 1 and 2 was .68. Thus, as Schriesheim and Murphy had done, a single summated (across the 17 items) performance measure was employed for analytic purposes.

Job Satisfaction The 20-item short form of the Minnesota Satisfaction Questionnaire (MSQ; Weiss,

Dawis, England, & Lofquist, 1967) was used to measure overall or global job satisfaction. Like the long MSQ, the short version uses a five-point Likert-type response scale (very satisfied, satisfied, undecided or neutral, dissatisfied, and very dissatisfied); it also was carefully developed and refined, and substantial evidence exists on its validity and reliability (e.g., Wanous, 1974; Weiss et al., 1967).

122 LEADERSHIP QUARTERLY Vol. 6 No. 2 1995

Role Chrify Rizzo, House, and Lirtzman’s (1970) 6-item measure of role ambiguity (reverse-scored

for role clarity) was employed; it is now one of the most extensively used instruments in organizational research (cf. Van Sell, Brief, & Schuler, 1981). The Rizzo et al. items are scored on a seven-point Likert scale (from strongly agree to strongly disagree), and a number of investigations have shown the measure to have good psychometric adequacy (e.g., House, Schuler, & Levanoni, 1983; Schuler, Aldag, & Brief, 1977).

Methods of Analysis

Raw-Score Analyses Standard raw-score analyses were conducted using the respondents’ untransformed

(raw) data and computing variable means, standard deviations, coefficient alpha internal consistency reliabilities, and Pearson product-moment intercorrelations in the usual manner. Then, moderated linear multiple-regression analysis (Cohen & Cohen, 1983; Zedeck, 1971) was employed to assess the presence of significant interaction effects between role clarity and supervisory initiating structure and consideration as correlates of subordinate performance and satisfaction. In the moderated regression procedure, the perceived leader behavior independent variable (initiating structure or consideration) was entered into the equation first. Then, the role clarity moderator was entered second. Finally, a cross-product term was added to each regression to assess the unique variance contributed by the interaction of role clarity and the perceived leader behavior variable.2 When significant interactions were found, the distributions of the independent and moderator variables were dichotomized into low and high subgroups (based upon median splits) and the dependent variable cell means for the resulting 2 X 2 matrix were calculated and graphically plotted to show the form of the interaction effect (Arnold, 1982, 1984).

lames et al. (1984) Within-Croup Agreement Coefficient The James et al. (1984) within-group agreement statistic (TWG) was computed and

tested for statistical significance (Chames & Schriesheim, forthcoming) for the sample and measures employed in this study under two different assumptions about the null distribution of responses. First, a uniform or rectangular null distribution was employed. According to James et al. (1984), this constitutes a proper null distribution under the assumption that various response biases are not operative and that lack of agreement is therefore reflected in random responses (where each response alternative has an equal probability of being selected). This is the distribution used in virtually all applications of the James et al. (1984) coefficient to date (e.g., George, 1990), and it results in item response probabilities and expected item means and variances of .20, 3.00, and 2.00, respectively, for a five-point response scale and .143, 4.00, and 4.00 for a seven-point scale.

The second null distribution used was the “slight skew” distribution suggested by James et al. (1984, p. 9 1) for scales with live-point response alternatives. This distribution was selected because the authors’experience (gained from prior research on the LBDQ- XII) suggests that the consideration and initiating structure scales are probably subject to some negative skew but that these effects are most likely to be relatively modest

Multi-Method Examination 123

(Schriesheim & Kerr, 1974, 1977; Schriesheim, Kinicki, & Schriesheim, 1979). In addition, the authors went through their data archives and examined the item means, standard deviations, and response distributions from over 3,000 LBDQ-XII respondents; this suggested that James et al’s (1984) characterization of the slight skew distribution appeared to be a reasonably good approximation of obtained LBDQ- XII responses (probabilities of .05, .15, .20, .35, and .25 for the first through fifth response categories, respectively, and an expected item mean and variance of 3.60 and 1.34). Examination of data reported in the MSQ manual (Weiss et al., 1967, pp. 24, 113-119) and of almost 2,000 MSQ respondents in the authors’ data archives also supported the James et al. (1984) five-point slight skew distribution as a reasonable portrayal of likely response biases on the MSQ.

A slight skew was also selected as the second null distribution for the two seven- point response scales used in the current research (the role clarity and performance measures). Interpolating James et al.‘s (1984, p. 91) live-point slight skew distribution to a seven-point scale yielded probabilities of .02, .05, .08, .14, .20, .28, and .23 for the first through seventh response categories, respectively, along with an expected item mean and variance of 5.21 and 2.39. These values appeared quite consonant with data in the authors’ archives for almost 2,000 respondents who had completed the Rizzo et al. (1970) role clarity measure over the past two decades.

WABA The bivariate WABA I and II analyses proceeded as outlined in the Appendix and

as described in greater detail by Dansereau et al. (1984). The multivariate and moderated WABA analyses followed the procedures also summarized in the Appendix and elaborated upon in Schriesheim (forthcoming). It should be noted that the statistical significance of the within- and between-entity multiple correlations, and the statistical significance of increases in these correlations (as variables were added to the regressions), were tested by the hierarchical multivariate F-test procedures outlined in Cohen and Cohen (1983, Ch. 3). Additionally, treating the multiple correlations as bivariate composite r’s (which are based upon linear combinations of variables), the Fisher r-to- Z transformation was employed to test for significant differences between the multiple correlations obtained for the within- and betweenentity regressions. Also, following the same logic, the standard WABA A and R tests of practical significance were used, treating the multiple correlations as bivariate correlations for analytical purposes. Finally, since some withinentities effects were uncovered, analyses were undertaken to test for the presence of within-entities moderation (i.e., interactions between parts and entities). These analyses used the design outlined in the Appendix (Equations 9 and 10) and, again, the statistical significance of increases in explained variance were tested by the hierarchical multivariate F-test procedures outlined in Cohen and Cohen (1983, Ch. 3).

RESULTS

Raw-Score Analyses

Table 1 presents the raw-score variable means, standard deviations, coefficient alpha internal consistency reliabilities, and Pearson product-moment intercorrelations. All of

124 LEADERSHIP QUARTERLY Vol. 6 No. 2 1995

Table 1 Raw-Score Variable Means, Standard Deviations, Coefficient Alpha Reliabilities,

and Intercorrelations

Intercorrelarions

Variable M SD (2 1. 2. 3. 4. 5. 6.

1. Structure (IS) 37.83 5.70 .85 -

2. Consideration (C) 35.62 4.58 .73 .74 -

3. Role Clarity (R) 33.40 3.84 .76 .27 .12 -

4. Satisfaction 83.26 6.66 .77 -.05 -.13 .37 -

5. Performance 103.19 10.68 .96 -.21 -.13 -.02 -.08 -

6. IS X R 1269.51 279.47 - .86 .58 .73 .14 -.13 -

7.CXR 1191.74 220.76 - .70 .77 .72 .12 -.08 .50

Note: The correlations shown attain significance at .29 @ < .05) and .38 @ < .Ol).

the scale reliabilities exceed .70 and appear acceptable (Nunnally, 1978), and all of the Table 1 entries are virtually identical to those reported in Schriesheim and Murphy. In absolute terms, the largest difference in mean scores is 0.46 (for performance) while the largest difference in standard deviations is 0.17 (for initiating structure), and the largest difference in correlations is .09 (for consideration and performance). (Because basically the same sample is involved, differences cannot be tested for statistical significance; cf. McNemar, 1969.) As in Schriesheim and Murphy, neither perceived initiating structure nor consideration obtain significant correlations with the satisfaction and performance dependent variables (suggesting the desirability of examining moderated relationships).

Table 2 Raw-Score Moderated Regression Results

Step and Independent

Variable Added

Unstandardized Partial

Regression Coefficient

Step 1 Step 2 Step 3

AR’

When

Added

Total

R’

Dependent Variable: Performance 1. Structure (1s)

2. Role Clarity (R) 3. ISX R

Dependent Variable: Satisfaction 1. Structure (Z&J

2. Role Clarity (R) 3. ISX R

Dependent Variable: Performance 1. Consideration (C)

2. Role Clarity (R) 3.CXR

Dependent Variable: Satisfaction 1. Consideration (C)

2. Role Clarity (R) 3.CXR

-0.39 -0.41 0.11

4.05 -0.18 0.71**

-0.29 -0.29 -6.40 -0.01 -6.78

- 0.18

-0.18 -0.25

0.67**

-7.79** .04

-8.09** .oo 0.21** .15**

3.69* 00 .oo 5.01** .16** .16*

-0.11* .10* .26**

4.27* .02 .02 5.68** .14** .16*

-0.14* .13* .27**

.02

.OO

.07

.04

.19*

.02

.09

Notes: * p < .05. ** p < .Ol.

Multi-Method Examination 125

Role Clarity Level 1

Independent Variable (Initiating Structure or Consideration)

Figure 1. Ihstrative Interaction Plot

Table 2 presents the raw-score moderated regression results. As shown in Table 2, significant role clarity moderator effects are apparent for three of the four tested relationships: initiating structure and performance, initiating structure and satisfaction, and consideration and satisfaction. In these three instances, the increases occurring in I? with the addition of the moderator interaction terms (at the third step in the regressions} are, in fact, reasonably large (. 15, . IO, and .13, respectively).

Figure 1 presents an illustrative graphical plot that can be used to explain the three significant interactions. For the first interaction, initiating structure is associated with increased performance under high role clarity (shown as the role clarity level 2 relationship in Figure l), whereas under low role clarity, it is associated with decreased performance (shown as the role clarity level 1 relations~p~; this is opposite what was hypothesized, but it was also the only role clarity moderator effect that was obtained by Schriesheim and Murphy. On the other hand, for the second interaction, initiating structure is positively associated with satisfaction under low role clarity (shown as level 2 in Figure 1) and negatively associated with satisfaction under high role clarity (shown as level 1). This pattern is as hypothesized (but it was not found in the Schriesheim and Murphy study). Finally, the third interaction is the same as the second but it involves perceived consideration. However, this is opposite what was predicted (and Schriesheim and Murphy also did not obtain this result).

Within-Group Agreement

Table 3 presents the distribution of obtained James et al. (1984) within-group agreement coefficients (TWG). As can be seen in Table 3, only 8 of the 180 r&W coefficients (4.4%) are less than .7 and only 9 are not statistically significant (5.0%). Furthermore, all of the mean and median ~wG coefficients are in excess of .82 and all but one is

T ab

le 3

is tr

ib u

ti on

o f

W it

h in

-G ro

u p

A gr

ee m

en t

C oe

ff ic

ie n

S tr

u ct

u re

o n

si d

er a

ti o

R o

C la

ri ty

a ti

sf a

ct io

P er

fo rm

a n

G ro

u p

s u

0. 88

1* *

0. 92

9* *

0. 97

1* *

0. 93

0. 76

0. 97

8* *

0. 98

3* *

0. 93

1* *

0. 96

8* *

0. 97

8* *

0. 96

8* *

0. 96

2* *

0. 92

9* *

0. 95

0. 87

8* *

0. 89

1* *

0. 95

0. 98

3* *

M ea

n 0.

93 6*

M dn

. 0.

95 4*

0. 62

0. 84

0. 95

0* *

0. 87

0. 00

0. 96

4* *

0. 97

2* *

0. 85

0. 94

0. 96

4* *

0. 94

4* *

0. 93

2* *

0. 84

0. 91

4”

0. 60

0. 68

0. 91

0. 97

2* *

0. 82

0. 91

22 %

0. 84

3* *

0. 92

9* *

0. 95

3* *

0. 96

3* *

0. 94

0. 95

3* *

0. 95

0. 90

8* *

0. 97

2* *

0. 92

5* *

0. 95

9* *

0. 86

2* *

0. 93

4* *

0. 91

0. 96

5* *

0. 89

8* *

0. 96

3* *

0. 96

8* *

0. 93

0. 94

0. 23

0. 84

0. 91

0. 93

0. 88

0. 91

0. 76

0. 95

0. 83

0. 92

0. 46

0. 85

0. 80

0. 93

8* *

0. 72

0. 93

0. 94

4* *

0. 82

0. 88

11 %

0. 94

2* *

0. 79

0. 85

0. 97

0. 92

0. 98

2* *

0. 98

5* *

0. 98

3* *

0. 93

0. 97

4* *

0. 98

8* *

0. 92

3* *

0. 96

5* *

0. 97

7* *

0. 90

0. 97

1* *

0. 98

1* *

0. 95

5* *

0. 94

5* *

0. 96

0. 87

0. 00

0. 49

0. 94

0. 82

0. 96

9* *

0. 97

0* *

0. 86

0. 95

2* *

0. 97

9* *

0. 82

0. 93

0. 95

0. 78

0. 94

6* *

0. 96

0. 91

0. 84

0. 94

11 %

0. 99

1* *

0. 94

6* *

0. 98

0* *

0. 97

2* *

0. 99

4* *

0. 88

4* *

0. 98

3* *

0. 97

4* *

0. 98

6* *

0. 99

0* *

0. 97

3* *

0. 97

6* *

0. 97

5* *

0. 96

8* *

0. 97

8* *

0. 99

2* *

0. 98

0* *

0. 98

1* *

0. 97

9* *

0. 98

0* *

0. 98

6* *

0. 84

0* *

0. 96

3* *

0. 94

4* *

0. 99

1* *

0. 97

3* *

0. 96

9* *

0. 94

8* *

0. 97

5* *

0. 98

4* *

0. 94

6* *

0. 95

3* *

0. 95

1* *

0. 93

1* *

0. 95

9* *

0. 98

7* *

0. 96

4* *

0. 96

5* *

0. 95

7* *

0. 96

3* *

0. 99

6* *

0. 99

6* *

0. 99

9* *

0. 99

4* *

0. 99

5* *

0. 99

4* *

0. 99

8* *

0. 98

8* *

0. 98

1* *

0. 99

5* *

0. 99

9* *

0. 99

5* *

1. 00

0* *

1. 00

0* *

0. 99

7* *

0. 99

5* *

0. 99

4* *

0. 99

0* *

0. 99

5* *

0. 99

5* *

0. 99

3* *

0. 99

4* *

0. 99

9* *

0. 98

9* *

0. 99

2* *

0. 98

9* *

0. 99

6* *

0. 97

8* *

0. 95

9* *

0. 99

1* *

0. 99

9* *

0. 99

1* *

1. 00

0* *

1. 00

0* *

0. 99

5* *

0. 99

1* *

0. 98

9* *

0. 98

2* *

0. 99

0* *

0. 99

2* *

N ot

es :

G ro

up s

4 ,

5 ,

7 ,

9 ,

1 4

, an

17 h

av e

tw o

m em

be rs

; al

l o

th er

s ha

th re

A s

su gg

es te

d by

m es

a l.

(1 98

4) ,

va lu

es o

f 0

.0 0

ha ve

ee n

bs ti

tu te

d fo

r co

ef fi

ci en

g re

at er

th an

.0 o

r le

ss t

ha n

0 .0

. “U

” h

ea d

s co

lu m

w hi

th e

co ef

fi ci

en ts

er e

co m

pu te

d u

n d

as su

m ed

if or

di st

ri bu

ti on

. w

h er

ea s

“S ”

h ea

d s

co lu

m ns

ti m

at ed

in g

th e

sl ig

sk ew

d is

tr ib

ut io

n su

gg es

te d

by J

am es

a l.

( 1

9 8

4 )

(s ee

t h

e te

fo r

co m

pu ta

ti on

de ta

il s)

* p

.0 5

** p

.O l.

Multi-Method Examination 127

statistically significant. Overall, these results thus appear clearly to support aggregation to the between-groups level of analysis. However, two observations should be noted about the distribution of rWG coefficients shown in Table 3. First, seven nonsignificant coefficients were obtained for the role clarity measure under the slightly skewed null distribution. Given that this distribution is the more plausible null, concern must be raised about the appropriateness of aggregating this variable. Second, Table 3 is a good illustration of several possibly unrecognized facts about the ~wG coefficient. As can be seen in Table 3, TwG coefficients can be well above .7 and still not be statistically significant. Additionally, James et al. (1984) note that the number of “judges” (group members) may exert a substantial effect on obtained TWG coefficients. However, Table 3 shows that the number of “targets” (items in a scale) also seems to substantially affect the size of the r&VG coefficient (and its statistical significance). For example, all of the coefficients for the 20-item satisfaction and 17-item performance measures are statistically significant, and the lowest is .84. This appears reasonable, since longer scales tend to have less error (Nunnally, 1978). Thus, the poor showing for the 6-item role clarity measure may, in some part, be an artifact of its relatively small number of items (at least as compared with the four other measures). This suggests that aggregation of the role clarity measure should not be precluded and that conducting additional aggregation analyses (e.g., WABA) is also highly desirable.

WABA

Table 4 presents the results from the WABA I analyses, while Table 5 presents the WABA II findings. Table 6 summarizes the inferences drawn from the WABA I and II results; it also presents the within- and between-groups correlation components and integrates all the findings into an overall conclusion. (For ease in following the discussion below, each relationship shown in Tables 5 and 6 is consistently numbered in Tables 4 through 6; the decision rules that were employed for the assessments are briefly outlined in Dansereau et al. 11984, pp. 169-185, especially Tables 7.8 and 7.91.)

WABA I As can be seen in Table 4, the WABA I E-tests support five of the variables tested

as having meaningfully more dyadic or within- than between-groups variance; additionally, one variable (performance) has meaningfully more between- than within- groups variance. However, for only two of these six are statistically significant F-tests obtained (for performance and for the multivariate composite variable of relationship 5--that involving initiating structure and role clarity); additionally, significant F-test results are obtained for three variables whose E-tests are nonsupportive-initiating structure and the composite variables for relationships 2 and 3.

Combining the E and F-test results shows good support for viewing relationships that involve performance as being largely between groups in nature and for seeing relationship 5 (which involves the composite initiating structure and role clarity variable) as being largely dyadic, or within groups. Also, based upon significant F-test results, weak inferences of between-groups variation seem reasonable for initiating structure and for the composite variables of relationships 2 and 3. Finally, those variables without a significant F-test are shown in Table 4 as supporting both the within- and between-

128 LEADERSHIP QUARTERLY Vol. 6 No. 2 1995

Table 4 Within- and Between-Groups Analysis I for Variables

Eta

Relationship(s) and Variable(s) Within Bef ween E F Inference

1,4 Structure (IS) .68 .I3 1.07 2.03* Weak Group 7,lO Consideration (C) .74 .68 0.92 0.67 Both

Role Clarity (R) .89 .46 0.52++ 2.12 Both 4-6,10-12 Satisfaction .75 .66 0.88 0.73 Both

l-3,7-9 Performance .29 .96 3.31++ 19.34** Group 2 IS, R .69 .72 1.04 2.04* Weak Group 3 IS, R, IS X R .67 .75 1.12 2.51* Weak Group 5 IS, R .90 .43 0.48++ 2.34* Dyad 6 IS, R, IS X R .79 .61 0.77+ 0.84 Both 8 C. R .74 .68 0.92 0.63 Both 9 C, R, C X R .75 .61 0.89 0.63 Both 11 C. R .87 .49 0.56++ 1.68 Both 12 C, R, C X R .81 .59 0.73+ 0.94 Both

Nore: The variables for relationships 2, 3, 5, 6, 8, 9, 1 I, and 12 are linear composites, developed using the multivariate WABA procedures outlined in Schriesheim (forthcoming). Italicized values are shown for F-tests that are corrected

for testing the significance of within-groups effects; those with one independent variable have df= 30,17, whereas those with two and three (composite variables) have 30.16 and 30.15, respectively; regular F-tests (nonitalicized

values) with one independent variable have df = 17,30, while those with two and three have 16,30 and l&30, respectively.

’ Significant by the 15” test.

” Significant by the 30” test.

* p < .05.

** p < .Ol.

groups perspectives-because there is no sound basis for inferring that more variation exists in one perspective than the other (cf. Dansereau et al., 1984).

WABA II Table 5 summarizes the WABA II findings. It should be noted that none of the

differences in correlations (Z-tests) is statistically significant, although meaningful practical differences (A-tests) are shown for relationships I, 2, and 12. Thus, using the interpretive criteria of Dansereau et al. (1984), those relationships where either (or both) the within- or between-groups correlations obtain significant R-test results are shown in Table 5 as supporting an inference of “both.“This leads to inferences of both within- and between-groups covariation for relationships 1, 2, 3, 5, 6, 9, 11, and 12, and to inferences of no within- or between-groups covariation for relationships 4, 7, 8, and 10. Finally, Table 5 shows a statistically significant increase in explained within-groups satisfaction variance for initiating structure and role clarity (relationship 5) over just initiating structure alone (relationship 4), and for consideration and role clarity (relationship 11) over just consideration alone (relationship 10). Additionally, a significant increase in explained between-groups satisfaction variance is obtained for relationship 12, involving consideration, role clarity, and their interaction (over relationship 11, which does not include the interaction term). However, neither of the two other moderator relationships that were uncovered in the raw-score analysis obtain

D ep

en d

en t

V a

ri a

bl e

3 S

$ 5 S L

ab le

W it

h in

- an

d B

et w

ee n

-G ro

u p

s A

n al

ys is

I I

fo r

R el

at io

n sh

ip s

2 In

d ep

en d

en t

W it

h in

-g ro

u p

s B

et w

ee n

-g ro

u p

s g

V a

ri a

bl e(

C o

rr el

a ti

o n

’ C

o rr

el a

ti o

n b

z In

fe re

n ce

1. P

er fo

rm an

S tr

uc tu

(I s)

2. P

er fo

rm an

S tr

uc tu

(I s)

, R

ol e

C la

ri ty

3. P

er fo

rm an

IS ,

R ,

IS X

4. S

at is

fa ct

io n

S tr

uc tu

(I s)

S at

is fa

ct io

n S

tr uc

tu re

s) ,

R ol

e C

la ri

(R )

6. S

at is

fa ct

io n

IS ,

R ,

IS X

7. P

er fo

rm an

C on

si de

ra ti

(c )

8. P

er fo

rm an

C on

si de

ra ti

(C ),

ol e

C la

ri ty

) 9.

P er

fo rm

an ce

,R ,C

X R

. S

at is

fa ct

io n

C on

si de

ra ti

(C )

11 .

S at

is fa

ct io

n C

on si

de ra

ti on

R ol

e C

la ri

(R )

12 .

S at

is fa

ct io

n C

, R

, C

. 0 5

-. 02

.3 2+

-. I5

.4 1+

.4 7+

-. 05

.0 5

.2 1

-. 07

.3 8+

.4 3+

-. 31

.2 6+

.3 1+

.5 3+

.0 7

-. 08

.4 4+

.6 1+

.1 6

-. 18

.I 8

.1 3

.4 1+

-. 18

.5 0+

.7 0*

+ +

* .3

0. 81

ot h

0. 91

ot h

0. 76

ot h

-0 .2

5 N

on e

0. 09

ot h

0. 59

ot h

0. 41

on e

0. 39

on e

0. 68

ot h

0. 34

on e

0. 45

ot h

1. 21

ot h

N ot

es :

’ F

or k

I in

de pe

nd en

t va

ri ab

le s,

flt )

= 2

9; f

k =

2 ,

df If

l =

27 ;

an d

r k

= 3

, dA

3 ,2

6 .

b F

or k

1 in

de pe

nd en

t va

ri ab

le s,

fl t)

; fo

= 2

, d

fl F

) =

,1 .5

; a n

fo rk

3 ,

df lF

) =

,1 4

’ F

or k

1 in

de pe

nd en

t va

ri ab

le s,

lz )

= 15

,2 8;

f or

k =

2 ,

df f.Z

) =

14 ,2

7; a

n d

r k

= 3

, df

lz )

1 3

,2 6

* S

ig ni

fi ca

@ <

5 )

R ’

in cr

ea se

pr ev

io us

la ti

on sh

(w it

ho ut

e la

in de

pe nd

en t

va ri

ab le

’ S

ig ni

ti ca

by t

h e

1 5

’ te

st .

” S

ig ni

fi ca

by t

h e

3 0

” te

st .

* p

.0 5

T ab

le 6

u m

m ar

y of

W A

B A

I a

n d

I I

R es

u lt

s, W

it h

in -

an d

B et

w ee

n -E

n ti

ty C

or re

la ti

on C

om p

on en

ts ,

an d

O ve

ra ll

I n

fe re

n ce

D ep

en d

en t

In d

ep en

d en

t W

A B

I W

A B

W it

h in

er w

ee n

v er

a ll

a ri

a bl

e V

a ri

a bl

e( s)

fe re

n ce

fe re

n ce

o m

p o

n en

t C

o m

p o

n en

t In

fe re

n ce

1 . P

er fo

rm an

2 .

P er

fo rm

an ce

. P

er fo

rm an

4 .

S at

is fa

ct io

n 5

. S at

is fa

ct io

n 6

. S

at is

fa ct

io n

7 .

P er

fo rm

an ce

. P

er fo

rm an

9 .

P er

fo rm

an ce

0 . S

at is

fa ct

io n

1 1

. S at

is fa

ct io

n 1

2 . S

at is

fa ct

io n

S tr

uc tu

(I S

) S

tr uc

tu re

S ),

ol e

C la

ri ty

( I?

) IS

, R

. IS

X R

tr uc

tu re

S )

S tr

uc tu

re (

IS ),

ol e

C la

ri ty

( R

) IS

, R

, IS

X R

on si

de ra

ti on

’)

C on

si de

ra ti

(c ),

ol e

C la

ri ty

( R

) C

, R

, C

X R

on si

de ra

ti on

) C

on si

de ra

ti on

), R

ol e

C la

ri ty

( R

) C

, R

, C

X R

W ea

k G

ro up

ea k

G ro

W ea

k G

ro up

ot h

W ea

k D

ya d

B ot

h W

ea k

G ro

W ea

k G

ro up

ea k

G ro

B ot

h B

ot h

B ot

h B

ot h

B ot

h N

on e

B ot

h B

ot h

N on

e B

ot h

N on

e B

ot h

B ot

. O l

-. 22

.2 1

.3 8

-. 08

.2 8

.2 7

-. O

l -.

.O l

.1 2

.2 6

-. 08

.I 6

.2 7

B ot

h B

ot h

W ea

k G

ro up

on e

B ot

h B

ot h

N on

e N

on e

B ot

h N

on e

B ot

h B

ot h

N o

te s:

fe re

n ce

s of

“ N

o n

e” d

en o

te s

th at

i nd

uc ti

o r

co nc

lu si

sh ou

b e

d ra

w n

’ S

ig ni

fi ca

nt ly

re at

th an

e b

et w

ee n

-g ro

u p

s co

m po

ne nt

t h

e 1

5 ”

A -t

es t.

Multi-Method Examination 131

a statistically significant increase in explained within- or between-groups variance (relationships 3 and 6).

Combined Results Table 6 summarizes the WABA I and II inferences which were presented above; Table

6 also gives the decomposed raw-score correlations (into within- and between-groups correlation components) and presents overall or summary inferences. It should be emphasized that the one overall inference which is labeled in Table 6 as “Weak Group” should probably be considered “very weak” (i.e., as more indicative of both within- and between-groups effects than of between-groups only)-based upon the evaluative criteria of Dansereau et al. (1984). Particularly noteworthy are the findings with respect to the moderator relationships which were supported in the raw-score analyses (relationships 3,6, and 12). Here, it can be seen that those moderator effects are probably best viewed as occurring both within- and between-groups, although a slightly stronger between-groups effect is evident for relationship 3, involving role clarity as a moderator of the initiating structure-performance relationship.

As a follow-up to the Table 6 results, two sets of plots were made of the interactions which were obtained for relationships 3, 6, and 12. One set used only within-groups scores and the other used only between-groups scores; both followed the same procedures outlined earlier for Figure 1. These plots are not shown here because all six showed interaction patterns similar to those of the raw-score interactions. Thus, the form of the interactions does not seem to differ within and between groups, so that the earlier discussion of raw score interactions portrays these satisfactorily.

Within-Groups Moderation Effects As a final planned analytic step, the composite variables involved in the three

moderator effects (those for relationships 3,6, and 12) were tested to determine whether significant interactions exist between individual employees (“parts’? and work groups (“entities”). Significant moderation was found for relationship 3 (Ap = .04; F= 11.67; df = 1,12; p < .Ol), involving initiating structure, role clarity, their interaction, and performance, and for relationship 6 (A* = .16; F= 6.67; df = 1,12;p < .05), concerned with initiating structure, role clarity, their interaction, and satisfaction. However, the results for relationship 12 (involving consideration, role clarity, their interaction, and satisfaction) were not significant (AR* = .l 1; F = 3.51; df = 1,12; ns). Thus, these analyses help further interpret the WABA results reported above-by highlighting that the obtained relationship 3 and 6 within-groups effects are best viewed as due to parts (employees) withinentities (work units or supervisors) and that they are not due to parts alone.

DISCUSSION

Substantive Findings

The purpose of this article was not to develop new substantive knowledge about leadership processes, so our discussion of substantive implications is intentionally kept brief. However, it seems worthwhile to highlight that support was found for the

132 LEADERSHIP QUARTERLY Vol. 6 No. 2 1995

hypothesized moderating effect of role clarity on the relationship between initiating

structure and subordinate satisfaction but that evidence contradicting the hypotheses

was obtained for the initiating structure and performance relationship and for the

relationship between consideration and satisfaction.

In conformity with the implicit levels-of-analysis assumption embedded in the

Schriesheim and Murphy study, these moderator effects are probably best viewed as

occurring relatively independently of the employees’ work groups (i.e., as occurring

largely both within and between groups), although some greater weight should probably

be placed on the between-groups explanation for the initiating structure and

performance relationship. Looking at possible substantive interpretations for the

obtained findings, and drawing on path-goal leadership theory (House, 1971) and the

contingency model (Fiedler, 1967), the results for the first significant interaction may

be explained by considering one set of probable role clarity effects.

Jobs with high degrees of respondent-perceived role clarity are likely to be seen by

their incumbents as highly predictable (Rizzo et al., 1970) and are not likely to be

“intrinsically motivating”(cf. House & Dessler, 1974). Additionally, they also are likely

to afford the leader the opportunity to be more directive, by better knowing the “ins”

and “outs” of the task and by being able to insist that subordinates follow standard

task execution procedures (Fiedler, 1967). Thus, the imposition of leader initiating

structure may reduce “soldiering” among subordinates and thereby be associated with

increases in job performance. Conversely, jobs low in perceived role clarity are more

likely to be intrinsically motivating, so that the imposition of leader initiating structure

may reduce such things as felt autonomy, and thereby relate to decreased motivation

and job performance.

The results for the second and third significant interactions may be likewise explained

using basic path-goal theory tenets (House, 1971; House 8z Dessler, 1974) and the facts

that supervisory consideration apparently contains a substantial element of “role

clarification” behavior (Schriesheim, 1978) and that, as commonly reported in the

literature (Schriesheim & Kerr, 1977) it was seen by this sample as sharing substantial

task-oriented and directive elements with perceived initiating structure (the initiating

structure-consideration correlation was .74; see Table 1). Thus, supervisory initiating

structure and consideration may have helped clarify task-related contingencies and

helped subordinates with low perceived role clarity perform at higher levels, thereby

deriving more intrinsic and extrinsic rewards from their jobs and their employer.

Conversely, for subordinates with high role clarity, such supervisory behavior may have been seen as “an imposition of control that is redundant” and “as excessively directive

and restrictive” (House & Dessler, 1974, p. 41) and thereby related to decreased

subordinate job satisfaction.

These explanations of our obtained results are, of course, post hoc and quite

speculative. However, they do fit the data and existing theoretical frameworks and,

therefore, probably warrant additional exploration and testing. However, since the main

focus of this article has been primarily on methodological issues, we now turn to a

fuller treatment of some of the research method implications that we believe our results

suggest.

Multi-Method Examination 133

Methodological Findings

Differences in Inferences About Aggregation Between rwG and WABA A quick comparison of the TWG and WABA I results (Tables 3 and 4, respectively)

shows that both approaches would raise concerns and cause the researcher to be at least somewhat tentative or cautious about aggregating the role clarity variable to the between-groups level of analysis. Additionally, both approaches are clearly supportive of aggregating performance, while the WABA I results are only somewhat less encouraging than the TWG findings with respect to aggregating perceived supervisory initiating structure. Major differences, however, can be seen with respect to perceived supervisory consideration and subordinate job satisfaction. Here, the within-groups agreement coefficients clearly show strong support for aggregation to the between- groups level of analysis, while the WABA I results are ambivalent-suggesting both within- and between-groups levels as most appropriate.

These general findings, coupled with the apparent sensitivity of the rWG coefficient to the number of “targets” (items) and/or “judges” (group members), suggests that it may be most judicious to not rely on a rWG analysis alone for making aggregation decisions but to also employ WABA I. Additionally, since the usual follow-up to a rWG analysis would typically either use raw scores or group-averaged (i.e., between- groups) scores (depending upon the rWG results), insight into the separate contributions of the phenomena as occurring within and between groups would usually be foregone. Thus, since the use of WABA I and II together allows the partitioning of variance and covariance and provides more complete information about the locus of a particular empirical relationship, it seems to make the most sense to rely principally on WABA and to supplement it with a rWG analysis to increase confidence in the obtained results. This, of course, is a conclusion with which some might disagree. However, until additional research is conducted on characteristics of the rWG coefficient, on WABA, and on how these two approaches interrelate, the most cautious and conservative approach would seem to involve using both.

Necessity of Examining Within-Entities Moderation One fallout of the current study is the obtaining of support for within-groups

moderation effects for two of the relationships that were tested (relationships 3 and 6). However, since at least some between-groups effects were also supported for these relationships (see Table 6), the interpretation of results still remains that these relationships are best viewed from both a within- and between-groups perspective. Thus, while this analysis did not affect the inferences that were drawn in the current investigation, it does support the George and James (1993) argument that the presence of significant within-entities effects should mandate the testing of potential within- entities moderation effects.

Differences in Raw-Score Moderator Findings Obtaining differences in our results between the raw-score regression and Schriesheim

and Murphy’s subgroup analyses was not unexpected, and such differences could be caused by shortcomings in the subgroup moderator design, artifacts introduced by Schriesheim and Murphy’s use of partial correlations, minor differences in the samples

134 LEADERSHIP QUARTERLY Vol. 6 No. 2 1995

Table 7 Zero-Order and First-Order Partial Correlation Moderator Subgroup Results

Role Clarity

Relationship Low (N = 23) High (N = 25) Z

Zero-Order Correlations

Initiating Structure-Performance -.60** .08 2.50** Initiating Structure-Satisfaction .I3 -.33 1.50 Consideration-Performance -.40* .21 2.0s Consideration-Satisfaction .Ol -.41** 1.66* Initiating Structure-Consideration .80** .72**

Fit-Order Partial Correlations”

Initiating Structure-Performance -.51* -.I0 I.46 Initiating Structure-Satisfaction .22 .Ol 0.68 Consideration-Performance .I7 .22 0.16 Consideration-Satisfaction -.15 -.34 0.64

Nom: ’ The partiak for relationships involving initiating structure have consideration statistically controlled, and vice versa.

* p < .05.

** p < .Ol.

employed (as noted, six respondents used in Schriesheim and Murphy could not be matched to groups and were therefore deleted from our analyses), or some combination of these factors (cf. Arnold, 1982; McNemar, 1969; Peters & Champoux, 1979; Peters

et al., 1984; Zedeck, 1971). Thus, to better understand the source of the obtained differences we reanalyzed our data, duplicating the design of Schriesheim and Murphy. Splitting the 48 respondents at their median score (433 and >33) for role clarity resulted

in a low subgroup (it4 = 30.23; SD = 2.69; N = 23) and a high subgroup (M = 36.20; SD = 2.14; N = 25) and, following Schriesheim and Murphy, we tested the two subgroups for the difference in their mean role clarity scores and obtained a significant result (t = 8.46; p < .Ol). We next computed zero-order and first-order partial correlations between the perceived leader behaviors and subordinate satisfaction and

performance for the two subgroups separately (the partials involved statistically controlling for the perceived leader behavior that was not being treated as an

independent variable). Differences in the paired correlations (for low and high subgroups) were then tested for significance by the Fisher Z-transformation (McNemar, 1969). The results obtained are presented in Table 7, and it should be mentioned that the average absolute difference between the first-order partial correlations shown in Table 7 and those reported in Schriesheim and Murphy is only .03 (one difference is .05, two each are .04., .03, and .02, and one is .Ol).

As shown in Table 7, even using the same exact sample, different raw-score moderator results are obtained for both the zero-order and first-order subgroup analyses as compared to the moderated multiple regression results (see Table 3). Looking at the zero-order findings first, it can be seen that the initiating structure-performance and consideration-satisfaction results parallel those of the moderated regressions. However,

Multi-Method Examination 135

opposite the regression results, the moderated initiating structure-satisfaction relationship is not significant, whereas the moderated consideration-satisfaction relationship is significant. For the first-order partial subgroups analysis, the discrepancy is even more pronounced. Here, none of the moderator effects attain statistical significance, although that for the initiating structure-performance relationship comes close to attaining significance (.05 <p < .lO). Thus, this reanalysis of the Schriesheim and Murphy subgroup results suggests that the differences from the current investigation’s raw-score-moderated multiple regression findings are a result of both the subgroup method and the use of partial correlations; sample differences appear to play a very minor role in the obtained discrepancies.

We believe that these findings have serious implications for integrating and using the extant leadership literature for hypothesis generation and new theory construction. This is particularly true because a large number of studies have used conceptual frameworks and operational procedures that are similar to those of Schriesheim and Murphy, and because these form the basis for much of what is “known”in the leadership domain (cf. Kerr et al., 1974; House, 1988). Simply put, because some studies have used moderator subgroup designs and some have used moderated regression, and sometimes a partialling procedure has been used and sometimes not, we do not believe that sound conclusions can be confidently drawn from much of the existing leadership literature-at least not without making heroic (and quite probably unfounded) assumptions about the equivalency of analytic procedures and, of course, the results they produce. Unfortunately, this pessimistic point of view does not take into account level-of-analysis concerns, to which we now turn our final attention.

The Necessity of Examining Within- and Between-Groups Effects Although the current research basically supported the implicit levels-of-analysis

assumption of Schriesheim and Murphy, there is no guarantee that such would be the case if we had tested another study. Thus, perhaps one of the most important implications that we see as arising from the results of the current study is the absolute necessity of having researchers explicitly state the levels of analysis at which they believe their hypothesized phenomena hold and of testing obtained data for alignment with those expectations. If this is not done, we simply cannot know how to interpret a particular study’s findings: Do they tell us of relationships that hold independently of unit membership? Or, are they indicative of within-unit or between-unit relationships? Furthermore, how can research that implicitly demands a particular level of analysis but does not test for it, lay any claim to being worthwhile or to advancing the field? Finally, we believe that the alignment of theory and data is also critical with respect to both sampling and measurement in leadership research. For example, we wonder whether the group-oriented referent of the LBDQ-XII and the sampling strategy used by Schriesheim and Murphy might have biased our results at least somewhat toward the between-groups level (although some LBDQ-XII items appear to have an individual-based referent and this might have lessened somewhat the overall bias toward inferences of between-groups effects). Also, as our last parenthetical comment, we note that because we were conducting a reanalysis of existing data, we did not raise issues related to sample size. However, it should be pointed out that some of the inferences which were drawn in our study were clearly affected by having a small number of units

136 LEADERSHIP QUARTERLY Vol. 6 No. 2 1995

(18) and a small total sample size (48) (for example, our Z-tests for statistically significant

differences between correlations may have been adversely affected by our small ns).

Thus, researchers who are interested in conducting sound levels-of-analysis research

might be well advised to keep sample-size issues clearly in mind. These and similar

critical concerns have been largely ignored in the past but we hope that this study, along

with the burgeoning literature on levels of analysis, will increase appreciation for and

interest in explicitly addressing level-of-analysis issues in future theory building and

testing.

CONCLUSION

The first author was once accused of beating his breast “over the imminent demise of

leadership theory” and of being a harbinger of “doom and despair” (Fiedler, 1977, pp.

4546). Perhaps this is true. Perhaps we have been overly pessimistic in some of our

statements about the leadership literature. However, we still believe that very serious

problems exist.

There are well-known measurement problems that plague the established literature

and make drawing conclusions across different studies-even if they supposedly concern

the same phenomena-quite problematic (cf. Schriesheim 8z Kerr, 1977). When we add

to this the likely interpretive and integrative problems that arise from differences in

data-analytic methods (e.g., moderator subgroups versus moderated regression),

misalignment of theory and level of analysis actually measured (e.g., using the LBDQ-

XII for individual-level measurement), and failure to explicitly specify and/or test a

phenomenon’s level(s) of analysis, we do not feel confident about being able to use

the extant literature to develop sound hypotheses and theory for future research. Thus,

we believe that much new research (that is attentive to the issues mentioned above)

will be necessary to build a solid base of knowledge for future advances in the field.

We also see a need for empirical investigations of the relative proportions of variance

attributable to individual-, group-, and organizational-level phenomena, and we expect

that such analyses will probably not yield a consistent answer concerning explained

variance. Our knowledge of the pervasiveness of contingency factors suggests that

situational attributes may cause our findings to be context-specific. The “bad news”

associated with all of this is that we have much work to do before we can confidently

speak about leadership at different levels of analysis. However, the “good news” is that

we have surely advanced in our knowledge of the concerns that future leadership theory and research must address in order to help us successfully develop an accumulated store

of sound leadership knowledge.

Acknowledgments: The authors would like to gratefully acknowledge the extensive advice we received from Larry James and Fran Yammarino on conducting rwG and WABA analyses and the very constructive comments of Fred Dansereau, Larry James, Steve Markham, and Fran Yammarino on earlier drafts of this article. Financial support

from the School of Business Administration, University of Miami, is also gratefully

acknowledged.

Multi-Method Examination 137

APPENDIX: Technical Information on Computations

The rwG Coefficient

The TWG index is based upon the ratio of the mean observed variances of judges’ scores (where the variances are averaged across items) to the expected variance ifjudges’ scores were drawn randomly from a null probability distribution that is specified by the investigator. Mathematically, rWG is computed as follows:

TWO(J) = {Jr1 - (2 / c7E2)l} / (Jr1 - (sxiz ’ m2)1 + G I cJE2N

where rWG(J) is the within-group agreement coefficient for judges’ mean scores based on J items, G is the mean of the observed variances on the J items, and ~~~ is the expected variance of a hypothesized null distribution (the expected variance of the judges’ responses when no agreement exists among them with respect to the substantive meanings of the ratings).

WABA

The WABA Equation In WABA, within- and between-entity indicators are computed and compared to

each other by tests of statistical and practical significance. Drawing upon the logic of analysis of variance, data are divided into within-cells (deviation from cell average) and between-cells (cell average) components, with the cells representing analytic entities such as work groups or other organizational units. The relationships that result from these calculations may be summarized in the basic WABA equation as:

‘Bx "By 'Bxy + ’ W, 'WY 'WV = 'Tw (2)

where “Bx and “By are the between-entity etas for variables x and y, ’ W, and ‘WY are the corresponding within-entity etas, ‘Bv and ’ W, are the corresponding between-entity and withinentity correlations, and ‘TV is the total (raw-score) correlation.

17 W, and rl WY are camp uted by correlating the raw-scores ([x,] or lj4) with the appropriate within-entity deviation scores (lx,, - %I or [y” - ykl) for n parts (e.g., the 1 to N respondents) within k entities (e.g., the 1 to K work units); ‘B, and “By are computed by correlating the raw-scores of the n parts (x, or yn) with their between- entity scores (i.e., the appropriate [&I or [ykl for the entity within which each part is situated). The within-entity correlation is calculated by correlating the within-entity deviation scores (i.e., lx, - %I and 1~~ - ykl) for the n parts, whereas the between- entity correlation is calculated by assigning each part its appropriate between-entity scores ([%I and &I) and then correlating these across the parts.

As shown in the fundamental WABA equation (Equation 2), any raw-score correlation may be partitioned into two separate components-a between-entity component (“Bx ‘By rBxy) and a withinentity component (’ W, ‘WY ‘WV); both are the products of multiplying their appropriate etas and component correlations. Thus, raw- score correlations (‘T,) cannot be interpreted unambiguously-at least not without

138 LEADERSHIP QUARTERLY Vol. 6 No. 2 1995

examining and considering all six of the terms in the fundamental WABA equation (Pedhazur, 1982; Robinson, 1950).

WABA I WABA I examines each variable’s variance by partitioning the original raw-score

(e.g., [x,1) into within- and between-entity component scores (e.g., lx, - %I and [%I); these component scores are then correlated with the original raw score to yield within- entity (7~) and between-entity (7,~) ems. Finally, the etas are tested (relative to each other) with F-tests of statistical significance and E-tests of practical significance.

The traditional F-tests of statistical significance have K - 1 and N - K degrees of freedom for the between- and withinentity etas, respectively, where K is the number of entities and N is the total number of parts within entities. When a between-entities eta exceeds its corresponding within-entities eta, a traditional F-test is used. However, when the withinentities eta is larger, a corrected F-test, which is simply the inverse of the traditional F-test, is employed (Dansereau et al., 1984; Haggard, 1958):

Cormted F= i(d) / (N - K)l / I(&) / (K- I)] = 11 F.

The E- (eta ratio) tests assess the magnitude of within- versus betweeneffects relative to each other; they are geometrically based, not dependent upon degrees of freedom, and calculated as:

E= VB/ VW. (4)

WABA II WABA II examines relationships among variables by first computing within- and

between-entities correlations (using all within- or all between-entity scores for the n parts). The magnitude of these correlations is then tested for statistical significance (using traditional t-tests) and for practical significance (using newly developed R-tests). The t-tests have K - 2 and N - K - 1 degrees of freedom for the between- and within- entity correlations, respectively. The geometrically based R- (correlation) tests of practical significance are not dependent upon degrees of freedom and are computed as:

2 l/2 Re=r~/ (l--B) ,

and

2 l/2 Rw=rw/ (l--w) .

Finally, differences between within- and betweenentity correlations which involve the same variables are tested using Fisher Z transformation tests of statistical significance (with K - 3 and N - K - 2 degrees of freedom for the between- and within-entity correlations, respectively), and A (angular) tests of practical significance (for the Z and A tests, only differences in absolute magnitudes are tested). The A-tests are geometrically based and not dependent upon degrees of freedom and are computed as:

Multi-Method Examination 139

where OW and 0~ are the angles associated with the within- and between-entity correlations, respectively.

Drawing Inferences Inferences are drawn using the .05 and .Ol levels of statistical significance for the

F, Z, and t tests and the 15” and 30” levels of practical significance for the E, R, and A tests. The 15’ and 30” angle criteria come from the fact that a 90” angle represents unrelated variables whereas a 0” angle represents a perfect relationship; smaller angles thus represent stronger relationships (in the R-tests), whereas angular ratios different from 1 .O or larger differences between angles (in the E and A tests, respectively) indicate more meaningful differences in the magnitudes of relationships (see Dansereau et al., 1984, for additional details). Decomposed raw-score correlation components are also computed by multiplying the product of the WABA I (within- or between-entities) ems with their WABA II (within- or between-entities) correlations; these results can then be examined by the A-test to determine whether one component is meaningfully greater than the other. Finally, the WABA I and II results and the decomposed correlations are examined and an overall conclusion drawn concerning the phenomenon under investigation. (Dansereau et al. 11984, pp. 183-1851, present guidelines for integrating WABA I and II results; for excellent yet brief descriptions of WABA I and II procedures, see Markham and McKee 119911, Yammarino and Dubinsky 119921, and Yammarino and Markham [ 19921.)

Multivaria te WABA Since leadership and other researchers are very often interested in conducting

multivariate and moderator analyses, the basic WABA procedures outlined above have recently been extended to the multivariate case by the use of hierarchical linear multiple regression (Schriesheim, forthcoming). The logic underlying this extension of WABA is relatively straightforward, since (1) one way of conceptualizing linear multiple regression is that it is simply bivariate correlation/ regression in which the independent variable is a weighted linear composite of several other variables (Cohen & Cohen, 1983; McNemar, 1969), and (2) the coefficient of multiple determination (R) can be viewed as a coefficient of simple determination (r*) between the observed data (I$) and the corresponding fitted values (Pi) (Neter, Wasserman, 8z Kutner, 1985; Cohen & Cohen, 1983). Based upon this logic, the following multivariate extension of the basic WABA equation may be developed for the case where two independent variables (xl and x2) and one dependent variable (_v) are employed:

The terms in Equation (8) have the same meanings as do those in Equation (2), with the only major differences being that the composite between (“BXrX2) and within (” W,,,2) independent variable etas replace the independent variable etas (“B, and ‘W,) of Equation (2) and the Equation (2) bivariate between and within correlations (‘B,, ’ W,, and ‘TX,,) are replaced by their multivariate analogs (‘BXiX2,+ rWXrX~y, and rTXrX~y) in

140 LEADERSHIP QUARTERLY Vol. 6 No. 2 1995

Equation (8). Computationally, the elements of Equation (8) can be easily calculated using procedures outlined in Schriesheim (forthcoming), and the ease of applying the above multivariate extension should be apparent, as well as the fact that it can be extrapolated further (e.g., so as to test for moderator effects; cf. Schriesheim, forthcoming).

Testing Within-Entity Effects for Moderation As recently noted by George and James (1993), ‘W, is a common or “pooled”

regression parameter which is computed across all parts; ’ W,, therefore, summarizes any within-entity differences in regressions (i.e., any interactions between parts and

entities). Thus, as suggested by Schriesheim (forthcoming), the presence of these moderating effects can be assessed by using procedures outlined by Cohen and Cohen (1983, ch. 8) to test whether significant incremental explained variance (AI?) is produced by a regression that includes appropriate parts-by-entity interaction terms (over one which does not). These two equations are as follows:

y = a + bid, + . . . by-,d~-1 + bra + e (9)

and

y=a+bldl+... bK-,dK-1 + bKx + bK+ldlX + . . . b2K-,dK-IX + f?. (10)

where x and y are the independent and dependent variables (raw scores), respectively, dr to dk-1 represent effects-coded dummy variables for the K entities, and e is error. Obtaining a significant increase in explained variance for Equation 10 over Equation 9 should lead to caution in interpreting ’ W, and to recognizing that any previously obtained within-entity effects are relational (i.e., effects for the parts must be interpreted as occurring within the entities).

NOTES

1. A transposition error was uncovered in Schriesheim and Murphy’s results while conducting the current analyses. The initiating structure-performance correlations that are reported in Schriesheim and Murphy (1976, Table 3) should be reversed, so that the significant negative correlation (r = -.53; p < .OS) occurs under low role clarity and the nonsignificant negative correlation (r = -. 10) is under high role clarity (the significant difference between the two remains significant; p < .05). This pattern is opposite that hypothesized by Schriesheim and Murphy, but correcting this error does not affect the overall conclusion that Schriesheim and Murphy drew (since their three other role clarity moderator relationships were nonsignificant and they concluded that role clarity was generally not a moderator in their study).

2. In conformity with prevailing contemporary practices (e.g., House & Dessler, 1974), Schriesheim and Murphy used first-order partial correlations in their moderator subgroup analyses-statistically controlling for initiating structure in analyses involving consideration, and vice versa. We have not done likewise for two reasons. First, we believe that such a procedure may introduce statistical artifacts into obtained findings and, in any event, is a theoretically questionable practice at best (cf. McNemar, 1969, p. 184). Second, using an analogous “partialling” procedure for the current multiple regression analyses yielded only one difference

Multi-Method Examination 141

in results-when the effect of initiating structure was controlled in examining the relationship between consideration and performance, role clarity showed a marginally significant moderator effect. However, since this effect was not strong, because initiating structure did not attain statistical significance as an independent variable in any step in these moderator regressions, and since we have serious reservations about statistically controlling for variables (without having a strong theoretical justification for doing so), we report only our “unpartialled” multiple regression results. (Parenthetically, we note that the “partialled” multiple regressions involved entering initiating structure first in equations which employed consideration as the independent variable, and vice versa; cf. Cohen & Cohen, 1983.)

REFERENCES

Adams, J.S. (1963). Toward an understanding of inequity. Journal of Abnormal and Social Psychology, 67,422436.

Arnold, H.J. (1982). Moderator variables: A clarification of conceptual, analytic, and psychometric issues. Organizational Behavior and Human Performance, 29, 143-174.

Arnold, H.J. (1984). Testing moderator variable hypotheses: A reply to Stone and Hollenbeck. Organizational Behavior and Human Performance, 34,214224.

Bass, B.M. (1960). Leadership, psychology, and organizational behavior. New York: Harper. Bass, B.M. (1990). Buss & Stogdill’s handbook of leadership. 3rd edition. New York: Free Press. Borman, W.C. (1991). Job behavior, performance, and effectiveness. In M.D. Dunnette & L.M.

Hough (Eds.), Handbook of industrial & organizationalpsychology, 2nd edition (Vol. 2, pp. 271-326). Palo Alto, CA: Consulting Psychologists Press.

Bryman, A. (1986). Leadership and organizations. London: Routledge t Kegan Paul. Campbell, D.T. (1956). Leadership and its effects upon the group. Columbus: Bureau of Business

Research, Ohio State University. Campbell, J.P. (1977). The cutting edge of leadership: An overview. In J.G. Hunt & L.L. Larson

(Eds.), Leadership: The cutting edge (pp. 221-235). Carbondale, IL: Southern Illinois University Press.

Chames, J.M., & Schriesheim, C.A. (Forthcoming). Estimation of quantiles for the sampling distribution of the rWG within-group agreement index. Educational and Psychological Measurement, 45.

Cohen, J., & Cohen, P. (1983). Appliedmultipleregression~correlation analysisfor the behavioral sciences. 2nd edition. Hillsdale, NJ: Erlbaum.

Cummings, L.L. (1975). Assessing the Graen/Cashman model and comparing it with other approaches. In J.G. Hunt & L.L. Larson (Eds.), Leadership frontiers (pp. 181-185). Carbondale, IL: Southern Illinois University Press.

Dansereau, F., Alutto, J.A., Markham, S.E., & Dumas, M. (1982). Multiplexed leadership and supervision: An application of within and between analysis. In J.G. Hunt, U. Sekaran, & C.A. Schriesheim (Eds.), Leadership: Beyond establishment views (pp. 81-103). Carbondale, IL: Southern Illinois University Press.

Dansereau, F., Alutto, J.A., & Yammarino, F.J. (1984). Theory testing in organizational behavior: The varient approach. Englewood Cliffs, NJ: Prentice-Hall.

Dansereau, F., & Dumas, M. (1977). Pratfalls and pitfalls in drawing inferences about leadership behavior in organizations. In J.G. Hunt & L.L. Larson (Eds.), Leadership: The cutting edge (pp. 68-83). Carbondale, IL: Southern Illinois University Press.

Etzioni, A. (1964). Modern organizations. Englewood Cliffs, NJ: Prentice-Hall. Evans, M.G. (1972). Leadership behavior: Demographic factors and agreement between

subordinate and self-descriptions. Personnel Psychology, 25, 649-653.

142 LEADERSHIP QUARTERLY Vol. 6 No. 2 1995

Fiedler, F.E. (1967). A theory ofleadership effectiveness. New York: McGraw-Hill. Fiedler, F.E. (1977). A rejoinder to Schriesheim and Kerr’s premature obituary of the contingency

model. In J.G. Hunt & L.L. Larson (Eds.), Leadership: The cutting edge (pp. 45-51). Carbondale, IL: Southern Illinois University Press.

Fleishman, E.A. (1957). A leader behavior description for industry. In R.M. Stogdill & A.E. Coons (Eds.), Leader behavior: Its description andmeasurement (pp. 103-I 19). Columbus, OH: Bureau of Business Research, Ohio State University.

Fleishman, E.A., Harris, E.F., 8z Burtt, H.E. (1955). Leadership and supervision in industry. Columbus, OH: Bureau of Educational Research, Ohio State University.

George, J.M. (1990). Personality, affect, and behavior in groups. Journal of Applied Psychology, 75, 107-l 16.

George, J.M., & James, L.R. (1993). Personality, affect, and behavior in groups revisited: Comment on aggregation, levels of analysis, and a recent application of within and between analysis. Journal of Applied Psychology, 78,798-804.

Glick, W.H., & Roberts, K.H. (1984). Hypothesized interdependence, assumed independence. Academy of Management Review, 9,722-735.

Graen, G.B., & Cashman, J.F. (1975). A role making model of leadership in formal organizations. In J.G. Hunt & L.L. Larson (Eds.), Leadership frontiers (pp. 143-165). Carbondale, IL: Southern Illinois University Press.

Graen, G.B., & Scandura, T.A. (1987). Toward a psychology of dyadic organizing. Research in Organizational Behavior, 9, 175-208.

Greene, C.N., & Schriesheim, C.A. (1980). Leader-group interactions: A longitudinal field investigation. Journal of Applied Psychology, 6.5, 50-59.

Guion, R.M. (1991). Personnel assessment, selection, and placement. In M.D. Dunnette & L.M. Hough (Eds.), Handbook of industrial & orgunizutionulpsychology, 2nd edition (Vol. 2, pp. 327-397). Palo Alto, CA: Consulting Psychologists Press.

Hackman, J.R. (1992). Group influences on individuals in organizations. In M.D. Dunnette & L.M. Hough (Eds.), Handbook of industrial & organizational psychology, 2nd edition (Vol. 3, pp. 199-267). Palo Alto, CA: Consulting Psychologists Press.

Haggard, E. (1958). Zntrucluss correlation and the analysis of variance. New York: Dryden Press. Halpin, A.W. (1957a). The leader behavior and effectiveness of aircraft commanders. In R.M.

Stogdill & A.E. Coons (Eds.), Leader behavior: Its description and measurement (pp. 52- 64). Columbus, OH: Bureau of Business Research, Ohio State University.

Halpin, A.W. (1957b). The observed leader behavior and ideal leader behavior of aircraft commanders and school superintendents. In R.M. Stogdill & A.E. Coons (Eds.), Leader behavior: Its description and measurement (pp. 65-68). Columbus, OH: Bureau of Business

Research, Ohio State University. Halpin, A.W., & Winer, B.J. (1957). A factorial study of the leader behavior descriptions. In

R.M. Stogdill & A.E. Coons (Eds.), Leader behavior: Its description and measurement (pp. 39-51). Columbus, OH: Bureau of Business Research, Ohio State University.

Hare, A.P. (1976). Handbook of small group research. 2nd edition. New York: Free Press. Hemphill, J.K., & Coons, A.E. (1957). Development of the leader behavior description

questionnaire. In R.M. Stogdill & A.E. Coons (Eds.), Leader behavior: Zts description and measurement (pp. 6-38). Columbus, OH: Bureau of Business Research, Ohio State University.

Hill, W.A. (1973). Leadership style: Rigid or flexible? Orgunizutionul Behavior and Human Performance, 9, 3547.

Hogan, R.T. (1991). Personality and personality measurement. In M.D. Dunnette & L.M. Hough (Eds.), Handbook of industrial & orgunizutionulpsychology, 2nd edition (Vol. 2, pp. 753- 832). Palo Alto, CA: Consulting Psychologists Press.

Multi-Method Examination 143

House, R.J. (1971). A path-goal theory of leader effectiveness. Administrative Science Quarterly, 16, 321-338.

House, R.J. (1988). Leadership research: Some forgotten, ignored, or overlooked findings. In J.G. Hunt, B.R. Baliga, H.P. Dachler, & CA. Schriesheim (Eds.), Emerging leadership vistas (pp. 245260). Lexington, MA: Lexington.

House, R.J. (1991). The distribution and exercise of power in complex organizations: A meso theory. Leadership Quarterly, 2,23-58.

House, R.J., & Dessler, G. (1974). The path-goal theory of leadership: Some post hoc and a priori tests. In J.G. Hunt & L.L. Larson (Eds.), Contingency approaches to leadership (pp. 29-59). Carbondale, IL: Southern Illinois University Press.

House, R.J., & Mitchell, T.R. (1974). Path-goal theory of leadership. Journal of Contemporary Business, 3, 81-98.

House, R.J., Schuler, R.S., & Levanoni, E. (1983). Role conflict and ambiguity scales: Reality or artifacts? Journal of Applied Psychology, 68,334-337.

Jago, A.G., & Vroom, V.H. (1975). Perceptions of leadership style: Superior and subordinate descriptions of decision-making behavior. In J.G. Hunt & L.L. Larson (Eds.), Leadership frontiers (pp. 103-120). Carbondale, IL: Southern Illinois University Press.

James, L.R., Demaree, R.G., & Wolf, G. (1984). Estimating within-group interrater reliability with and without response bias. Journal of Applied Psychology, 69,85-98.

Kanfer, R. (1990). Motivation theory in industrial and organizational psychology. In M.D. Dunnette & L.M. Hough (Eds.), Handbook of industrial 81 organizational psychology, 2nd edition (Vol. 1, pp. 75-170). Palo Alto, CA: Consulting Psychologists Press.

Katz, D., Maccoby, N., & Morse, N.C. (1950). Productivity, supervision. and morale in an office situation. Ann Arbor, MI: Institute for Social Research, University of Michigan.

Kerr, S., & Schriesheim, C.A. (1974). Consideration, initiating structure, and organizational criteria-An update of Korman’s 1966 review. Personnel Psychology, 27, 555-568.

Kerr, S., Schriesheim, C.A., Murphy, C.J., & Stogdill, R.M. (1974). Toward acontingency theory of leadership based upon the consideration and initiating structure literature. Organizational Behavior and Human Performance, I2,62-82.

Klein, K.J., Dansereau, F., & Hall, R.J. (1994). Levels issues in theory development, data collection, and analysis. Academy of Management Review, 19, 195-229.

Kozlowski, S.W.J., & Hattrup, K. (1992). A disagreement about within-group agreement: Disentangling issues of consistency versus consensus. Journal of Applied Psychology, 77, 161-167.

Lewin, K., Lippitt, R., & White, R.K. (1939). Patterns of aggressive behavior in experimentally created social climates. Journal of Social Psychology, 10, 271-301.

Locke, E.A. (1976). The nature and causes of job satisfaction. In M. Dunnette (Ed.), Handbook of industrial and organizationalpsychology (pp. 1297-1349). Chicago: Rand-McNally.

Lowin, A., & Craig, J.R. (1968). The influence of level of performance on managerial style: An experimental object lesson in the ambiguity of correlational data. Organizational Behavior and Human Performance, 3,440-458.

Markham, S.E. (1988). Pay-for-performance dilemma revisited: Empirical example of the importance of group effects. Journal of Applied Psychology, 73, 172-180.

Markham, S.E., & McKee, G.H. (1991). Declining organizational size and increasing unemployment rates: Predicting employee absenteeism from within- and between-plant perspectives. Academy of Management Journal, 34, 952-965.

Markham, S.E., & Scott, K.D. (1983). A component factor analysis of the initiating structure scale of the Leadership Behavior Description Questionnaire, form XII. PsychoZogical Reports, 52, 71-77.

McNemar, Q. (1969). Psychological statistics. 4th edition. New York: Wiley.

144 LEADERSHIP QUARTERLY Vol. 6 No. 2 1995

Miner, J.B. (1980). Theories of organizational behavior. Hinsdale, IL: Dryden Press. Mowday, R.T., & Sutton, R.I. (1993). Organizational behavior: Linking individuals and groups

to organizational contexts. Annual Review of Psychology, 44, 195-229. Neter, J., Wasserman, W., & Kutner, M.H. (1985). Applied linear statistical models. 2nd edition,

Homewood, IL: Irwin. Nunnally, J.C. (1978). Psychometric theory. 2nd edition. New York: McGraw-Hill. Pedhazur, E.J. (1982). Multiple regression in behavior research. New York: Holt, Rinehart, &

Winston. Perrow, C. (1970). Organizational analysis: A sociological view. Belmont, CA: Wadsworth. Peters, W.S., & Champoux, J.E. (1979). The role and analysis of moderator variables in

organizational research. In R.T. Mowday& R.M. Steers (Eds.), Research in organizations: Issues and controversies (pp. 239-253). Santa Monica, CA: Goodyear.

Peters, L.W., O’Connor, E.J., & Wise, S.L. (1984). The specification and testing of useful moderator variable hypotheses. In T.S. Bateman & G.R. Ferris (Eds.), Methodundanalysis in organizational research (pp. 128-139). Reston, VA: Reston Publishing.

Rizzo, J.R., House, R.J., & Lirtzman, S.E. (1970). Role conflict and ambiguity in complex organizations. Administrative Science Quarterly, 15, 150-163.

Roberts, K.H., Hulin, C.L., & Rousseau, D.M. (1978). Developing an interdisciplinury science of organizations. San Francisco: Jossey-Bass.

Robinson, W. (1950). Ecological correlations and the behavior of individuals. American Sociological Review, 75, 351-357.

Rousseau, D.M. (1985). Issues of level in organizational research: Multi-level and cross-level perspectives. Research in Organizational Behavior, 7, l-37.

Schriesheim, C.A. (1978). Development, application, and validation of new leadership behavior and expectancy research instruments. Unpublished Ph.D. dissertation, College of Administrative Science, Ohio State University.

Schriesheim, C.A. (1979). The similarity of individual-directed and group-directed leader behavior descriptions. Academy of Management Journal, 22, 345-355.

Schriesheim, C.A. (Forthcoming). Multivariate and moderated within- and betweenentity analysis (WABA) using hierarchical linear multiple regression. Leadership Quurterly, 6.

Schriesheim, C.A., & Bird, B.J. (1979). Contributions of the Ohio State studies to the field of leadership. Journal of Management, 5, 135-145.

Schriesheim, C.A., & Chames, J.M. (1995). A note on significance testing and using the rWG within-group agreement index. Unpublished manuscript, University of Miami.

Schriesheim, C.A., & Kerr, S. (1974). Psychometric properties of the Ohio State leadership scales. Psychological Bulletin, 81, 756-765.

Schriesheim, C.A., & Kerr, S. (1977). Theories and measures of leadership: A critical appraisal of current and future directions. In J.G. Hunt & L.L. Larson (Eds.), Leadership: The cutting edge (pp. 945). Carbondale, IL: Southern Illinois University Press.

Schriesheim, C.A., Kinicki, A.J., & Schriesheim, J.F. (1979). The effect of leniency on leader behavior descriptions. Organizational Behavior and Human Performance, 23, l-29.

Schriesheim, C.A., Mowday, R.T., & Stogdill, R.M. (1979). Crucial dimensions of leader-group interactions. In J.G. Hunt & L.L. Larson (Eds.), Crosscurrents in leadership (pp. 106-125). Carbondale, IL: Southern Illinois University Press.

Schriesheim, C.A., & Murphy, C.J. (1976). Relationships between leader behavior and subordinate satisfaction and performance: A test of some situational moderators. Journal of Applied Psychology, 61,634~641.

Schriesheim, C.A., Jr Schriesheim, J.F. (1974). Development and empirical verification of new response categories to increase the validity of multiple response alternative questionnaires. Educational and Psychological Measurement, 34, 877-884.

Multi-Method Examination 145

Schriesheim, C.A., & Stogdill, R.M. (1975). Differences in factor structure across three versions of the Ohio State Leadership Scales. Personnel Psychology, 28, 189-206.

Schuler, R.S., Aldag, R.J., 8z Brief, A.P. (1977). Role conflict and ambiguity: A scale analysis. Organizational Behavior and Human Performance, 20, 119-128.

Seeman, M.E. (1957). A comparison of general and specific leader behavior descriptions. In R.M. Stogdill & A.E. Coons (Eds.), Leader behavior: Its description and measurement (pp. 86- 102). Columbus, OH: Bureau of Business Research, Ohio State University.

Shaw, M.E. (1981). Group dynamics: Thepsychology of smallgroup behavior. 3rd edition. New York: McGraw-Hill.

Stogdill, R.M. (1963). Manual for the Leader Behavior Description Questionnaire-form XZZ. Columbus, OH: Bureau of Business Research, Ohio State University.

Stogdill, R.M., & Coons, A.E. (Eds.) (1957). Leader behavior: Zts description and measurement. Columbus, OH: Bureau of Business Research, Ohio State University.

Van Sell, M., Brief, A.P., & Schuler, R.S. (1981). Role confhct and ambiguity: Integration of the literature and directions for future research. Human Relations, 34, 43-71.

Vroom, V.H. (1964). Work and motivation. New York: Wiley. Wanous, J.P. (1974). A causal-correlational analysis of the job satisfaction and performance

relationship. Journal of Applied Psychology, 59, 139-144. Weiss, D.J., Dawis, R.V., England, G.W., 8t Lofquist, L.H. (1967). Manualfor the Minnesota

Satisfaction Questionnaire. Minneapolis, MN: Industrial Relations Center, University of Minnesota.

Yammarino, F. J. (1990). Individual- and group-directed leader behavior descriptions. Educational and Psychological Measurement, 50, 739-759.

Yammarino, F.J. (Forthcoming a). A conceptual-empirical approach for testing meso and multi- level theories. In H.L. Tosi (Ed.), Extensions of the environmentJorganization/person model (Monographs in Organizational Behavior and Industrial Relations, Vol. 17). Greenwich, CT: JAI Press.

Yammarino, F.J. (Forthcoming b). The varient/ WABA approach for multi-level theory testing. In D.R. Dalton & L.R. Williams (Eds.), Advances in research methodr and analyses for organizational studies (Vol. 1). Greenwich, CT: JAI Press.

Yammarino, F.J., & Dubinsky, A.J. (1992). Superior-subordinate relationships: A multiple levels of analysis approach. Human Relations, 45, 575-600.

Yammarino, F.J., Dubinsky, A.J., & Hartley, S.W. (1987). An approach for assessing individual versus group effects in performance evaluations. Journal of Occupational Psychology, 60, 157-167.

Yammarino, F.J., & Markham, S.E. (1992). On the application of within and between analysis: Are absence and affect really group-based phenomena? Journal of Applied Psychology, 77, 168-176.

Yukl, G.A., & Van Fleet, D.D. (1992). Theory and research on leadership in organizations. In M.D. Dunnette & L.M. Hough (Eds.), Handbook of industrial & organizational psychology, 2nd edition (Vol. 3, pp. 147-197). Palo Alto, CA: Consulting Psychologists Press.

Zedeck, S. (1971). Problems with the use of moderator variables. Psychological Bulletin, 76,295- 310.