LDRSHP WEEK 8

jacsti

reliability_and_validity.pdf

Home >Government homework help >LDRSHP WEEK 8

Reliability and validity of the “leadership competencies and engaging leadership scale”

Juliette Alban-Metcalfe Real World Group, Leeds, UK, and

Beverly Alimo-Metcalfe School of Management, University of Bradford, Bradford, UK

Abstract

Purpose – The aim of this paper is to present evidence of the reliability and validity of the “Local Government 360 (LG360)”, a diagnostic tool for assessing both competent and engaging leadership behaviour among managers and professionals in local government.

Design/methodology/approach – The paper argues that the assessment of both leadership competencies (the “what”) and engaging leadership behaviours (the “how”) is essential for a valid diagnosis of individuals’ strengths and developmental needs.

Findings – Evidence is presented of the internal consistency (reliability) and the criterion and discriminant validity of the tool, among an opportunity sample comprising 288 local government staff, of whom 143 were female and 77 male; 58 were from a Black and Minority Ethnic background and 220 were White.

Practical implications – Data on the impact of leadership behaviour on direct reports, provided by the LG360, are powerful in providing evidence to emphasise that the way in which a leader behaves can have a significant effect on staff attitudes and wellbeing at work.

Originality/value – Although a number of 360-tools are available, only in a small number of cases are their psychometric properties, particularly empirical evidence of their criterion and discriminant validity, reported.

Keywords Leadership competencies, Engaging leadership, Reliability, Validity, Impact on staff, Leaders, Leadership

Paper type Research paper

Introduction Our understanding of the nature of leadership, which has changed over the decades, can be seen to have gone through five main stages (Alban-Metcalfe and Alimo-Metcalfe, 2009). Thus, the “trait” or “Great Man” theories of the 1930s (stage 1) gave way to the “behavioural” theories of the 1950s (stage 2), out of which the notion of “managerial competency”, later “leadership competency”, arose. Both were criticised for failing to take proper account of situational factors; what emerged next were “situational” and “contingency” models (stage 3). The late 1970s and early 1980s saw the emergence of stage 4, which was prompted by recognition that leadership is fundamentally about handling constant change, creating a vision, and involving individuals in the means by which they are able to cope with ever-changing situations. In other words, dealing effectively with what Charles Handy has referred to as “never ending white water”. According to Northouse (2010, p. 3), “Leadership is a process

The current issue and full text archive of this journal is available at

www.emeraldinsight.com/0951-3558.htm

IJPSM 26,1

Received 7 April 2011 Revised 14 October 2011 Accepted 16 November 2011

International Journal of Public Sector Management Vol. 26 No. 1, 2013 pp. 56-73 q Emerald Group Publishing Limited 0951-3558 DOI 10.1108/09513551311294281

whereby an individual influences a group of individuals to achieve a common goal.” While a useful starting point, such a definition does not emphasise the role of leader as “defining organisational reality” (Bryman, 1996).

This “new paradigm” thinking resulted in models of “distant”, often “heroic” leadership, with emphases on “charisma”, “vision”, and “transformation” (e.g. Northouse, 2010). These models have, in turn, come in for criticism for a number of reasons (e.g. Alimo-Metcalfe and Alban-Metcalfe, 2008; Collins, 2001; Mintzberg, 1999), particularly following unedifying events such as the fall of Enron and Worldcom (e.g. Tourrish and Vatcha, 2005). Indeed, there is a growing awareness of what has come to be described as the “dark side of charisma” (Hogan et al., 1990). This relates to flaws in the personal characteristics of those occupying leadership roles, which has come to the fore. Hogan has been at the vanguard of research into the “dark side”, and has extolled organisations not only to focus on identifying the presence of certain positive characteristics of those in a leadership role, but equally importantly, to ensure the absence of “dark side” traits, particularly those that alienate other colleagues (most importantly, subordinates), and the inability to build and to support a team (Hogan et al., 1994).

Stage 5 thinking (what Collins (2001) refers to as “level 5” thinking) emerged almost simultaneously in the UK and the US. It is essentially about “nearby”, rather that “distant”, often “heroic”, leadership behaviour (Shamir, 1995). Although Collins did not propose a model for such leadership, relevant behaviours have been described as “ethical” (e.g. Brown et al., 2005; Iles and Macaulay, 2007), “authentic” leadership (e.g. Avolio and Walumbwa, 2006), and “engaging” leadership (Alimo-Metcalfe and Alban-Metcalfe, 2008). The approaches have much in common and overlap to a certain extent (Alimo-Metcalfe and Bradley, 2011; Walumbwa et al., 2009). The “engaging” leadership approach differs in that it used a Grounded Theory approach, among a sample that was deliberately inclusive with reference to age, ethnicity, gender, and level in the organisation, and assesses a wider range of relevant dimensions. The concept of “distributed” leadership (Spillane et al., 2001) falls into a different category since it is more concerned with “structural” arrangements – to whom, and at what level, is leadership delegated – than leadership behaviours per se.

Leadership competency and leadership style In drawing together this seventy years of research, it has been suggested (Alban-Metcalfe and Alimo-Metcalfe, 2009; Bolden and Gosling, 2006) that it is valuable to distinguish between what leaders do, and how they act. “What” leaders do can be assessed with reference to competency frameworks; “how” they act is a measure of their “leadership style”. This distinction was neatly summarised by Bolden and Gosling when they wrote:

A competency framework could be considered like sheet music, a diagrammatic representation of the melody. It is only in the arrangement, playing and performance, however, that the piece truly comes to life (Bolden and Gosling, 2006, p. 151).

The relationship between leadership competency and an “engaging” style of leadership is summarised in Figure 1.

Thus, person A can be seen to be highly competent as a leader, but not very engaging in their behaviour; perhaps the kind of person who is very detailed in their

Engaging leadership scale

planning, and who can devise very effective systems for quality control, but shows a lack of understanding of, or concern for, the needs of others.

Conversely, person B is someone who, perhaps, shows great concern for others, and creates a supportive environment in which all staff are valued, but who is unable to deliver what is required of them in terms of achieving goals or meeting agreed targets on time. Such a person’s style of leadership is highly engaging, but they show a low level of competency as a leader.

Person C, then, is the kind of manager or professional who, by acting in an engaging way, with all that entails, can use their competency as a leader in ways that are appropriate to the particular individual or situation.

The “leadership competencies and engaging leadership scale” (LCELS) Consistently with the “what” and “how” distinction, the “leadership competencies and engaging leadership scale (LCELS)” is a 360-degree multi-rater diagnostic feedback tool, on which managers are assessed in relation to their leadership competencies, and their engaging leadership behaviour.

Leadership competencies In spite of being almost ubiquitous in public and private sector organisations (Bolden et al., 2003), competency frameworks have been the subject of considerable criticism on conceptual and methodological grounds (Bolden and Gosling, 2006; Hollenbeck et al., 2006). Indeed, Bolden and Gosling (2006) commented that the “expansion of the concept of competencies raises further concerns because of its tendency to disguise and embed rather than expose and challenge certain assumptions about the nature and work of leadership”. Hollenbeck et al. (2006) went so far as to conclude that:

Figure 1. Relationship between “leader competencies” and “engaging” leadership behaviours

IJPSM 26,1

We see little evidence that these [competency] systems, in place for years now, are producing more and better leaders in organizations (p. 399).

The position adopted here is that the value of competency frameworks should be judged in terms of what they do, not in what they fail to do. What competency frameworks – when thoroughly researched, properly constructed, and differentiated to meet the particular needs of different groups of managers and professionals – can do is define and describe what a leader needs to be able to do in order to achieve the goals and targets appropriate to their role. Thus:

A competent leader may be defined as someone who enables the functioning of an organisation in a way that is goal directed, and is geared to developing processes and systems. This enables staff at all levels to plan effectively and efficiently, in order to achieve agreed goals.

High levels of competency can lead to a degree of consistency, and thereby enable staff to make day-to-day decisions and short-term predictions, with a measure of confidence. Leadership competencies, which are often largely closed-ended in nature, are necessary in order that staff can undertake strategic planning, and in this way help to turn the vision of an organisation, department or team into a reality (Alimo-Metcalfe and Alban-Metcalfe, 2008, pp. 15-16).

Acceptance of such a definition leads to the self-evident conclusion that being competent is an essential characteristic of anyone who occupies a management or leadership role. It is equally true, particularly in the light of the earlier criticism, that competency on its own is not enough.

The competencies that are assessed through the are based on an extensive of the leadership literature, and a factor analytic study of the competency framework to which the competencies can be traced (Alimo-Metcalfe, 1999). Within the, competencies are assessed on ten scales (Table I).

Leadership competencies Engaging leadership

Engaging individuals Respecting diversity Showing genuine concern Developing individual potential Enabling

Engaging the organisation Communication Supporting a developmental culture Developing the team Focusing team effort Planning

Engaging other stakeholders Stakeholder awareness Building shared vision Future orientation Facilitating change sensitively Commitment to excellence

Personal qualities and values Personal qualities Acting with integrity Reflective skills

Table I. Structure of the

leadership competencies and engaging leadership

scale (LCELS)

Engaging leadership scale

Engaging leadership In order to determine the nature of “nearby” transformational or “engaging” leadership, Alimo-Metcalfe and Alban-Metcalfe (2001, 2008) used Repgrid technique (Kelly, 1955) to elicited constructs of leadership from a total of 150 managers and professionals at all levels (Chief Executives; top, senior and middle managers) in local government and the health service in England and Wales. In addition to be inclusive in relation to level in the organisation, the sample was inclusive with reference to gender (50 per cent female; 50 per cent male), ethnicity (15 per cent BME background), and age. A “grounded theory” approach (Parry, 1998) was adopted to elicit the constructs. To elicit constructs of nearby leadership, individuals were asked to think of two current or previous line managers whom they rated as “good” in relation to having “a powerful effect on their motivation, self-confidence, self-efficacy, or performance”. They were then asked to think of two more line managers in the categories “not good” and “in-between”, and also to think about their own behaviour. Similarities and differences (the constructs) were recorded.

Content analysis of the 2,000 þ constructs that emerged led to the identification of 48 clusters of constructs, on the basis of which a pilot version of the “transformation leadership questionnaire (TLQ)” was developed by Alimo-Metcalfe and Alban-Metcalfe (2001, 2008). Completed responses were received from 1,464 local government, and 2,013 health service managers and professionals. Factor analyses of the data led to the development of the TLQ.

Use of the same technique among 1,022 police officers and staff at all levels from 28 of the 43 forces in England and Wales (Dobby et al., 2004) led to the validation of the TLQ with this population. The TLQ has also been validated in other organisations, for example, among three leading private sector companies in Scotland (Kelly et al., 2006). A private sector version of the TLQ has subsequently been developed and validated (Alban-Metcalfe and Alimo-Metcalfe, 2007). The different versions of the TLQ all show significant levels of criterion validity ( p , 0.01). Furthermore, a longitudinal empirical investigation of the leadership of multi-professional teams in the NHS provided evidence of a cause-effect relationship between an engaging style of leadership and productivity, measured objectively (Alimo-Metcalfe et al., 2007, Alimo-Metcalfe and Bradley, 2011).

Formally:

A “nearby” transformational or “engaging” leader may be defined as someone who encourages and enables the development of an organisation that is characterised by a culture based on integrity, openness and transparency, and the genuine valuing of others.

This shows itself in concern for the development and well-being of others, in the ability to unite different groups of stakeholders in articulating a joint vision, and in delegation of a kind that empowers and develops potential, coupled with the encouragement of questioning and of thinking which is critical as well as strategic.

Engaging leadership is essentially open-ended in nature, enabling organisations not only to cope with change, but also to be proactive in shaping their future. At all times engaging leadership behaviour is guided by ethical principles (Alimo-Metcalfe and Alban-Metcalfe, 2008, p. 16).

Items from the 14 scales that comprise the TLQ were combined to produce 7 “engaging leadership” scales within the LCELS (Table I).

IJPSM 26,1

Present study The decision to produce the LCELS was to provide a 360-tool that could be used by managers and professionals in a middle role who may need to develop their leadership competencies or skills, as well as their engaging leadership behaviours. The results presented here are based on a series of opportunity samples of local government managers and professionals (n ¼ 288), who completed the as part of a leadership development programme. The individuals were all in a “middle management” role, and completed the LCELS, as part of a leadership development programme. Although completion of the tool was voluntary, no-one declined the offer. Of these: 143 were female, 77 male; 220 were White, 68 were from a Black and Minority Ethic (BME) background. Other demographic data were not available. In each case, only the direct reports’ (subordinates’) ratings were analysed, since these have been shown to have the highest level of validity. Thus, self-ratings (those of the “target” manager, TM) are seen by some to be more ”lenient” than ratings provided by work colleagues (Atwater and Yammamino, 1992; Harris and Schaubroeck, 2002; Nilsen and Campbell, 1993; Podsakoff and Organ, 1986), though, conversely, Van der Heijden and Nijhof (2004) suggested that self-ratings are more likely to provide an accurate assessment of own performance since TMs are the ones who can evaluate it in all situations, and over time. Fletcher and Baldry (2000) suggested that when people rate their own behaviour they actually elevate their own ratings due to factors such as lack of self-awareness or a leniency bias so that they portray themselves in a favourable light, while Hoffman et al. (1991) have shown that self-ratings of performance have lower validity than supervisor ratings. Also, self-ratings are also less accurate than ratings from peers or direct reports, when compared to objective criterion measures (Fletcher and Baldry, 2000; Hough et al., 1983, cited in Atwater and Yammamino, 1992), while ratings by direct reports have been shown to be more valid than those by line managers (Hoffman et al., 1991; Hogan and Hogan, 2001; Shipper and Wilson, 1991, cited by Hogan and Hogan, 2001).

Results Reliability The psychometric characteristics of the scales are presented in Table II.

The alpha coefficients ranged from 0.76 to 0.97, which exceed the lower limit of 0.70 proposed by Cronbach (1951), and the inter-item correlations on each scale ranged from r ¼ 0.44 2 0.85, which exceed the lower limit of 0.30 proposed by Cortina (1993), using SPSS version 15. Thus, each of the scales shows a significant level of consistency, or internal reliability.

For the competency scales, the mean scores ranged from 4.70 to 4.96, with the standard deviations ranging from 0.79 to 1.03, while for the engaging leadership scales, the range of the mean scores was from 4.56 to 4.86, and of the standard deviations, 0.94 to 1.04. The coefficients of variation (CoVs), which is a measure of the “spread” of the scores ranged from 12.7 to 21.7 for the competences, and from 18.8 to 27.5 for engaging leadership (Yeomans, 1968). For both sets of scales, the range is well within the 1:2 ratio that is required if scales are to be rated with comparable discrimination between the items.

Engaging leadership scale

Validity Criterion validity. In order to test the criterion validity of the LCELS, product-moment correlation coefficients were calculated between the direct reports’ (subordinates’) ratings of their line manager, and direct reports’ ratings on each of six criterion variables. The criterion variables were [My line manager]: behaves in a way that raises my motivation to achieve; has a leadership style that increases my commitment to the organisation; leads in a way that increases my intention to stay in this organisation; behaves in a way that has a positive effect on my self-confidence; leads in a way that increases my job satisfaction; and reduces my job-related stress through their leadership style (e.g. Alban-Metcalfe et al., 2009; Alimo-Metcalfe and Alban-Metcalfe, 2008; Bass, 1998; van Breukelen et al., 2006; Walumbwa et al., 2009).

The coefficients ranged from r ¼ 0.61 for “planning” £ “reduces my job-related stress through their leadership style”, to r ¼ 0.86 for “showing genuine concern” £ “behaves in a way that has a positive effect on my self-confidence” ( p , .001, in all cases). When the sample was divided by gender, or by ethnicity, the coefficients were of the same order of magnitude.

Discriminant validity. The discriminant validity of the LCELS was tested using the discriminant functional analysis statistic, for the sample as a whole (Table III), and for females and males separately (Table IV). The total number of direct reports rating a BME manager was not large enough to permit use of this statistic with BME vs White managers.

Scales Items Alpha Inter-item

correlations Mean Standard deviation

Coefficient of variation

Competency Respecting diversity 4 0.92 0.62-0.81 4.96 1.03 21.4 Developing individual potential 9 0.96 0.56-0.85 4.79 0.95 18.8 Communication 4 0.88 0.62-0.68 4.70 1.01 21.7 Developing the team 7 0.95 0.66-0.85 4.77 1.02 21.8 Planning 4 0.88 0.54-0.81 4.88 0.84 14.5 Stakeholder awareness 3 0.85 0.56-0.77 4.80 0.91 17.3 Future orientation 7 0.91 0.47-0.72 4.90 0.79 12.7 Commitment to excellence 4 0.88 0.60-0.76 4.80 0.97 19.6 Personal qualities 7 0.92 0.50-0.73 4.80 0.93 18.0 Reflective skills 5 0.89 0.45-0.76 4.91 0.81 13.4

Engaging leadership Showing genuine concern 5 0.92 0.70-0.81 4.69 0.94 18.8 Enabling 3 0.76 0.44-0.57 4.80 1.06 23.4 Supporting a developmental culture 3 0.86 0.54-0.75 4.56 1.12 27.5 Focusing team effort 3 0.82 0.53-0.74 4.73 1.00 21.1 Building shared vision 3 0.86 0.61-0.74 4.78 1.02 23.9 Facilitating change sensitively 3 0.89 0.69-0.75 4.67 1.01 21.8 Acting with integrity 4 0.92 0.71-0.77 4.86 1.04 22.3

Note: n=228

Table II. Number of items, Cronbach’s alpha coefficients, inter-item correlations, means, standard deviations, and coefficients of variation for the seventeen LCELS scales

IJPSM 26,1

For the sample as a whole, the “enabling” scale was a significant predictor of five of the six impact measures, while “developing the team”, “commitment to excellence”, and “Showing genuine concern”, were each a significant predictor of three of the six. “respecting diversity”, “personal qualities”, and “acting with integrity” were each significant predictors of 2 impact measures, while “communication”, “future orientation”, “reflective skills”, and “supporting a developmental culture”, were significant predictors of one impact measure ( p , 0.05, in each case). For six of the scales, there was no statistically significant predictive link with any of the impact measures.

Put another way round, six of the leadership scales had a predictive link with “reduced job-related stress”, five scales with “job satisfaction”, four with “organisational commitment” and “intention to stay”, and three with “motivation” and “self-confidence”.

When the sample was divided by gender, a slightly different pattern emerged (Table IV).

Scale/impact measure Motivation Organisational commitment

Intention to stay

Self- confidence

Job satisfaction Stress

Competency Respecting diversity – – – – 0.16 0.17 Developing individual potential – – – – – – Communication – – – – 0.16 – Developing the team – – – 0.21 0.18 0.18 Planning – – – – – – Stakeholder awareness – – – – – – Future orientation – 0.22 – – – – Commitment to excellence 0.32 – 0.22 – 0.16 – Personal qualities 0.32 – – – – 0.17 Reflective skills – – – – – 0.17

Engaging leadership Showing genuine concern 0.37 – 0.23 0.21 – – Enabling – 0.22 0.26 0.20 0.17 0.17 Supporting a developmental culture – 0.23 – – – 0.17 Focusing team effort – – – – – – Building shared vision – – – – – – Facilitating change sensitively – – – – – – Acting with integrity – 0.23 0.22 – – –

Notes: n=228; Wilk’s lambda ( p , 0.05); Motivation – behaves in a way that raises my motivation to achieve; Organisational commitment – has a leadership style that increases my commitment to the organisation; Intention to stay – leads in a way that increases my intention to stay in this organisation; Self-confidence – behaves in a way that has a positive effect on my self-confidence; Job satisfaction – leads in a way that increases my job satisfaction; Stress – reduces my job-related stress through their leadership style

Table III. Discriminant functional

analysis of the relationship between

leadership scales and impact on direct reports,

using Wilk’s lambda

Engaging leadership scale

M ot

iv at

io n

O rg

an is

at io

n al

co m

m it

m en

t In

te n

ti on

to st

ay S

el f-

co n

fi d

en ce

Jo b

sa ti

sf ac

ti on

S tr

es s

S ca

le /i

m p

ac t

m ea

su re

M F

C om

pe te n cy

R es

p ec

ti n

g d

iv er

si ty

0. 30

D ev

el op

in g

in d

iv id

u al

p ot

en ti

al C

om m

u n

ic at

io n

D ev

el op

in g

th e

te am

0. 35

0. 28

0. 17

0. 25

0. 24

P la

n n

in g

0. 14

S ta

k eh

ol d

er aw

ar en

es s

F u

tu re

or ie

n ta

ti on

0. 31

0. 33

C om

m it

m en

t to

ex ce

ll en

ce 0.

14 0.

35 P

er so

n al

q u

al it

ie s

0. 16

R efl

ec ti

v e

sk il

ls 0.

E n ga gi n g le a d er sh ip

S h

ow in

g g

en u

in e

co n

ce rn

0. 24

0. 22

0. 38

0. 20

0. 26

E n

ab li

n g

0. 32

0. 31

0. 29

0. 18

0. 20

S u

p p

or ti

n g

a d

ev el

op m

en ta

l cu

lt u

re 0.

18 0.

31 0.

35 0.

15 0.

18 0.

21 F

oc u

si n

g te

am ef

fo rt

0. 16

B u

il d

in g

sh ar

ed v

is io

n F

ac il

it at

in g

ch an

g e

se n

si ti

v el

y A

ct in

g w

it h

in te

g ri

ty 0.

17 0.

N o te s :

F em

al es

(n =

14 3)

;M al

es (n

= 77

); W

il k

’s la

m b

d a

(p ,

0. 05

); M

ot iv

at io

n –

b eh

av es

in a

w ay

th at

ra is

es m

y m

ot iv

at io

n to

ac h

ie v

e; O

rg an

is at

io n

al co

m m

it m

en t

– h

as a

le ad

er sh

ip st

y le

th at

in cr

ea se

s m

y co

m m

it m

en t

to th

e or

g an

is at

io n

;I n

te n

ti on

to st

ay –

le ad

s in

a w

ay th

at in

cr ea

se s

m y

in te

n ti

on to

st ay

in th

is or

g an

is at

io n

; S

el f-

co n

fi d

en ce

– b

eh av

es in

a w

ay th

at h

as a

p os

it iv

e ef

fe ct

on m

y se

lf -c

on fi

d en

ce ;

Jo b

sa ti

sf ac

ti on

– le

ad s

in a

w ay

th at

in cr

ea se

s m

y jo

b sa

ti sf

ac ti

on ;

S tr

es s

– re

d u

ce s

m y

jo b

-r el

at ed

st re

ss th

ro u

g h

th ei

r le

ad er

sh ip

st y

Table IV. Discriminant functional analysis of the relationship between leadership scales and impact on direct reports, for females and males, using Wilk’s lambda

IJPSM 26,1

There, the following significant relationships were found among the leadership competency scales:

. “respecting diversity” was a predictor of 1 impact measure among both the females and the males;

. “developing the team” was a predictor of 4 impact measures among the females, 2 among the males;

. “planning” was a predictor of 1 impact measure, only among the males;

. “future orientation” was a predictor of 2 impact measures among the females, 1 among the males;

. “commitment to excellence” was a predictor of 1 impact measure among the females and the males;

. “personal qualities” was a predictor of 1 impact measure, only among the males; and

. “reflective skills” was a predictor of 1 impact measure, only among the females ( p , 0.05, in each case).

Among the engaging leadership scales, the significant relationships were between: . “showing genuine concern” and 2 impact measures among the females, 3 among

the males; . “enabling” and 3 impact measures among the females, 1 among the males; . “supporting a developmental culture” and 4 impact measures among the females,

2 among the males; and . “acting with integrity” and 1 impact measure among both females and males

( p , 0.05, in each case).

Discussion Reliability The psychometric properties of the LCELS suggest that it shows a significant level of internal consistency (reliability). On each of the scales, the alpha coefficient was 0.76 or above, and the inter-item correlations were 0.46 or above. The latter statistic is important because Cortina (1993) has demonstrated that, unless the individual items are inter-correlated at the r $ 0.30 level, it is possible for a scale for which a ¼ 0.70 actually to comprise two different scales. The coefficients of variation for the competency scales were of the same order of magnitude as each other in that the largest coefficient did not exceed two times the smallest (Yeomans, 1968). This was also true when the coefficients for the engaging leadership scales were compared. The engaging leadership scores were slightly more widely distributed that the competencies scores, but, importantly, in neither case were the ratings “bunched” around the mean.

Validity Criterion validity. The significant product-moment correlations between each of the scales and each of six impact which have been used in other research to establish the criterion validity of a leadership tool (e.g. Alban-Metcalfe et al., 2009; Alimo-Metcalfe

Engaging leadership scale

and Alban-Metcalfe, 2008; Bass, 1998; van Breukelen et al., 2006; Walumbwa et al., 2009) present evidence of criterion validity of the LCELS ( p , 0.001, for each of the scales). There is also evidence, from others studies, that one of the impact measures (“job satisfaction”) is a significant predictor of organisational performance, measured objectively (Patterson et al., 2004; Xenikou and Simosi, 2006).

Discriminant validity. The discriminant validity of an instrument or technique is a measure of the extent to which differential relationships exist between what is assessed or achieved, and different criterion variables. The psychological significance of the discriminant analyses reported here is:

. That they focus attention on the way in which different aspects of leadership behaviour, as measured by the different scales, are particularly relevant to different aspects of staff attitudes and wellbeing, and thus

. That they provide evidence: that, in spite of themselves being significantly correlated with each of the criterion variables, each of the scales measures a distinct aspect of competent or engaging leadership; and thus of the capacity of the instrument to discriminate between different patterns of relationships.

In other words, within the context of all the scales being valid (as evidenced by the correlational data), there is evidence that the leadership behaviour of some of the scales has an impact on staff attitudes and sense of wellbeing at work that cannot be accounted for by links with the other scales.

This particular kind of relevance may be described as a scale’s “focus of convenience”. In personal contract theory (Kelly, 1955), this term is defined as, “a set of events which its user finds can be most conveniently ordered within its context” (Bannister and Mair, 1968, p. 19). This is in contrast to the “range of convenience” which is “a broader set of events which the construct can deal with, if sometimes less effectively” (ibid.). In the present context, the focus of convenience of a particular scale refers to those aspects of job-related behaviour or feelings (staff attitudes and wellbeing) to which that scale is most relevant. In some cases, the focus of convenience of a scale relates to an aspect of behaviour or feelings among a group or a sub-group of staff (in this case, females or males). To suggest that a scale does not have a particular focus of convenience does not have implications for its range of convenience. The range of convenience of a scale is the wider range of feelings and behaviours, among a wider range of groups and sub-groups of managers, for which the scale still has relevance (as evidenced by the correlational data).

Thus, the correlational data suggest that each of the six measures of staff attitudes and wellbeing is within the range of convenience of each of the fourteen scales. On the other hand, stepwise discriminant analyses identify those variables (scales) which, independently of the other scales, account for statistically significant amounts of variance, and thus identify the focus of convenience of these scales and combinations of scales.

Contextual and situational factors. There are underlying reasons for not expecting either the product-moment or the discriminant analysis data to be identical for different groups or sub-groups of individuals. One of these is the effect of contextual factors. At an organisational or team level, the impact of a particular kind of leadership behaviour on particular aspects of staff attitude to work, or sense of wellbeing at work, will be affected by contextual organisational or team factors (e.g. Liden and Antonakis, 2009).

IJPSM 26,1

Some of these contextual factors can be related to: the kind of work place, in this case, a particular local authority department; the nature of the service they offer; socio-economic factors; differences in attendant career constraints and opportunities; and organisational contexts, including the leader’s vision. Overlapping, and sometimes related to these, are personal factors and local situational factors. Personal factors that have been shown to have a modulating effect are: high need for independence; indifference to organisational rewards; and professional orientation. To these, should be added age, ethnicity, gender, level in the organisation, and sexual orientation. Examples of situational factors include degree of work group autonomy, and the extent to which there is routine or programmed work (e.g. Alban-Metcalfe and Alimo-Metcalfe, 2000a, b; Bass, 1998; Dobby et al., 2004; Griffin, 2007; Griffin et al., 2010; Gronn, 1999; Howell, 1997; Howell et al., 1986; Jermier and Kerr, 1997; Kerr and Jermier, 1978; Parker et al., 2003; Podsakoff and MacKwenzie, 1997). Research by Stordeur et al. (1999) among nurses in a Belgium hospital indicated a significant moderating effect of such factors on the relationship between leadership scores and criterion variables that included job satisfaction and satisfaction with leadership style.

Another (broader) contextual factor, though not one that was examined here, relates to cultural differences. Thus, Misumi (1985) noted that “Variation [in perceptions of leadership] occurs because the same concepts may contain thought processes, beliefs, implicit understandings, or behaviours in one culture but not in another”. This was evident in the Globe Studies (Den Hartog et al., 1999; House et al., 1999), and cautions against the imposition of “one-size-fits-all” leadership assessment tools.

In the present context, other factors that may cause the differences between the 17 leadership scales and the six impact measures include differences in a range of contextual factors, such as the culture that exists within the organisation, for example, whether it is generally supportive of “adaptivity” and “proactivity” in the workplace (e.g. Griffin et al., 2010), or whether there is an imperative to develop cross-organisational or cross-sector links (e.g. Alban-Metcalfe and Alimo-Metcalfe, 2009; Crosby and Bryson, 2010).

A source of difference that was examined in the present study is between female and male staff. Since the sample comprised individuals from different organisations, it is inappropriate to speculate about the reasons for the evident differences in the predictive evidence for female versus male staff. What is evident, however, is that there are differences in the impact on the men and women in the sample. This is consistent with evidence from Antonakis et al. (2003) that factors that may be positively related in one context may negatively related in another. The conclusion to be drawn here is that gender would appear to be a significant contextual factor modulating the relationship between what a leader does, and the impact that this has on her/his direct reports.

Conclusions and implications What emerges is evidence of the reliability and validity of the LCELS among an opportunity sample comprising significant proportions of female and male, BME and White, direct reports. Generically, the evidence is that the way in which a leader acts has a significant effect on the attitudes to work and the wellbeing at work of her/his direct reports. There is also evidence that the contextual factor of whether the direct report is male or female affects the extent to which certain leadership behaviours have an impact on their attitude to, and well being at, work.

Engaging leadership scale

At a practical level, when used in the contexts of personal or organisational development, discriminant functional analysis data can be used:

. to identify those factors, operating within a particular organisation, or within a particular team, that can have a significant positive impact on its staff (Alban-Metcalfe et al., 2009; Alimo-Metcalfe and Alban-Metcalfe, 2008), and through them on organisational performance (Patterson et al., 2004; Xenikou and Simosi, 2006); and

. to emphasise that the way a leader behaves has a real impact on their staff. Where the number of direct reports rating their line manager is of the order of n ¼ 100, discriminant functional analyses can be calculated for that particular group.

Thus, the impact of those leadership behaviours that have been rated highest can be identified, and celebrated. Conversely, the impact of those rated lowest can also be identified, and the impact of those behaviours on staff attitudes to work and their wellbeing at work can be brought into high relief. Experience shows that evidence presented in this way can have a powerful effect, even on those who question the value of 360-data. Thus, the LCELS can be seen to assess those leadership behaviour that have been shown to have a significant impact on staff attitudes and wellbeing, and through these, on organisational performance.

Evidence of different patterns of relationships between leadership behaviours and impact on staff, among females and males, emphasises the more general need for sensitivity to contextual factors. In particular, the evidence points to the importance of recognising that the same leadership behaviours may be valued differently by different groups and subgroups of staff.

Where this leaves us is with a recognition: . that there is the need to distinguish between what a leader does in performing

her/his leadership role (acting more or less competently), and how he/she enacts these competencies (in a more or less engaging way); and

. that conceptual analyses and empirical evidence (e.g. Alimo-Metcalfe and Alban-Metcalfe, 2008; Alimo-Metcalfe and Bradley, 2011) indicate that competent leader behaviours on their own, are not sufficient for effective leadership.

The LCELS differs from the TLQ in that it assesses both competencies and engaging behaviours. This said, in order to be manageable, the number of engaging leadership items is reduced in the LCELS.

The reliability and validity data presented here are for a limited number of participants (total 288). Although the participants were from a wide range of local authorities, and different ethnic backgrounds, further analyses, based on participants from other sectors, private as well as not-for-profit, would be valuable, and provide evidence of generalisability.

With regard to competencies and competency frameworks, the fundamental issue is one of “fitness for purpose”. This can be achieved if:

IJPSM 26,1

. empirical sector-specific research be undertaken to ensure that what is assessed is relevant to managers and professionals working in that particular area, rather than being generic; and

. it is recognised that being competent is only part of the story.

In relation to sector specificity, the evidence suggests that the engaging leadership scales of the TLQ (on which the LCELS are based) are relevant to managers and professionals at different levels, and in different roles, in a wide range of private and not-for-profit organisations.

Caution must, however, be exercised when using these particular leadership competencies among staff at different levels, and in different roles, in different organisations. What this paper reports is the reliability and validity of the competency elements of the LCELS among middle managers and professionals in local government. The competency dimensions may need to be customised, to a greater or lesser extent, to meet sector-specific needs.

With regard to the second point, one might conclude by paraphrasing a comment made by Neil Kinnock in another context: competencies can be likened to Brighton Pier, very fine in their own way, but not a good way of getting to France[1] (cf. Alban-Metcalfe and Alimo-Metcalfe, 2009).

Note

1. Neil Kinnock when describing the 1981 Special Education Act for England and Wales.

References

Alban-Metcalfe, J. and Alimo-Metcalfe, B. (2007), “Development of a private sector version of the (Engaging) Transformational Leadership Questionnaire”, Leadership & Organization Development Journal, Vol. 28, pp. 104-21.

Alban-Metcalfe, J. and Alimo-Metcalfe, B. (2009), “Engaging leadership part one: Competencies are like Brighton Pier”, Internal Journal of Leadership in Public Services, Vol. 5 No. 1, pp. 10-18.

Alban-Metcalfe, J., Alban-Metcalfe, J. and Alimo-Metcalfe, B. (2009), “Engaging leadership part two: an integrated model of leadership development”, International Journal of Leadership in Public Services, Vol. 5 No. 2, pp. 5-13.

Alban-Metcalfe, R.J. and Alimo-Metcalfe, B. (2000a), “An analysis of the convergent and discriminant validity of the transformational leadership questionnaire”, International Journal of Selection & Assessment, Vol. 8, pp. 158-75.

Alban-Metcalfe, R.J. and Alimo-Metcalfe, B. (2000b), “The Transformational Leadership Questionnaire (TLQ-LGV): a convergent and discriminant validity study”, Leadership & Organization Development Journal, Vol. 21, pp. 280-96.

Alimo-Metcalfe, B. (1999), “Do male and female managers, their bosses, direct reports, and colleagues agree as to the nature of managerial/leadership effectiveness, and does gender make a difference? A factor analysis of 360 feedback”, in Pepermans, R., Flion, I., Ardts, J.C.A. and Jansen, P.G.W. (Eds), Managerial Behaviour: Empirical Studies of Management Behaviour, Development and Socialization, ACCO, Leuven.

Alimo-Metcalfe, B. and Alban-Metcalfe, J. (2008), Engaging Leadership: Creating Organisations that Maximise the Potential of their People, Chartered Institute for Personnel Development, London.

Engaging leadership scale

Alimo-Metcalfe, B. and Alban-Metcalfe, R.J. (2001), “The development of a new transformational leadership questionnaire”, Journal of Occupational & Organizational Psychology, Vol. 74, pp. 1-27.

Alimo-Metcalfe, B. and Bradley, M. (2011), “The impact of engaging leadership and contextual factors on organizational performance: evidence of a cause-effect relationship”, (submitted for publication).

Alimo-Metcalfe, B., Alban-Metcalfe, J., Bradley, M., Mariathasan, J. and Samele, C. (2007), “The impact of leadership factors in implementing change in complex health and social care environments”, Research project supported by NHS Grant SDO/22/2002.

Antonakis, J., Avolio, B.J. and Sivasubramaniam, N. (2003), “Context and leadership: an examination of the nine-factor full-range leadership theory using the multifactor leadership questionnaire”, Leadership Quarterly, Vol. 14, pp. 261-95.

Atwater, L.E. and Yammamino, F.J. (1992), “Does self-other agreement on leadership perceptions moderate the validity of leadership and performance predictions?”, Personnel Psychology, Vol. 45, pp. 141-64.

Avolio, B.J. and Walumbwa, F.O. (2006), “Authentic leadership: moving HR leaders to a higher level”, Research in Personnel and Human Resource Management, Vol. 25, pp. 273-304.

Bannister, D. and Mair, J.M.M. (1968), The Evaluation of Personal Constructs, Academic Press, London.

Bass, B.M. (1998), Transformational Leadership: Industry, Military, and Educational Impact, Erlbaum, Mahwah, NJ.

Bolden, R. and Gosling, J. (2006), “Leadership competencies: time to change the tune?”, Leadership, Vol. 2, pp. 147-63.

Bolden, R., Gosling, J., Marturano, A. and Dennison, P. (2003), “A review of leadership theory and competency frameworks”, Report for Chase Consulting and the Management Standards Centre, Centre for Leadership Studies, University of Exeter, Exeter, June.

Brown, M.E., Treviño, L.K. and Harrison, D.A. (2005), “Ethical leadership: a social learning perspective for construct development and testing”, Organizational Behavior & Human Decision Processes, Vol. 97, pp. 117-34.

Bryman, A. (1996), “Leadership in organizations”, in Clegg, S.R., Hardy, C. and Nord, W.R. (Eds), Handbook of Organizational Studies, Sage, London, pp. 276-92.

Collins, J. (2001), Good to Great, Harper Business, New York, NY.

Cortina, J.M. (1993), “What is coefficient alpha? An examination of theory and applications”, Journal of Applied Psychology, Vol. 78, pp. 98-104.

Cronbach, L.J. (1951), “Coefficient alpha and the internal structure of tests”, Psychometrika, Vol. 16, pp. 297-334.

Crosby, B.C. and Bryson, J.M. (2010), “Integrative leadership and the creation and maintenance of cross-sector collaborations”, Leadership Quarterly, Vol. 21, pp. 211-30.

Den Hartog, D.N., House, R.J., Hanges, P.J., Dorfman, P.W., Ruiz-Quintana, A. and GLOBE Associates (1999), “Culture specific and cross-culturally generalizable implicit leadership theories: are attributes of charismatic/transformational leadership universally endorsed?”, Leadership Quarterly, Vol. 10, pp. 219-56.

Dobby, J., Anscombe, J. and Tuffin, R. (2004), “Police leadership: expectations and impact”, Home Office Online Report 20/04, London.

Fletcher, C. and Baldry, C. (2000), “The study of individual differences and self-awareness in the context of multisource feedback”, Journal of Occupational and Organizational Psychology, Vol. 73, pp. 303-19.

IJPSM 26,1

Griffin, M. (2007), “Specifying organizational contexts: systematic links between contexts and processes in organizational behaviour”, Journal of Organizational Behavior, Vol. 28, pp. 859-63.

Griffin, M.A., Parker, S.K. and Mason, C.M. (2010), “Leader vision and the development of adaptive and proactive performance: a longitudinal study”, Journal of Applied Psychology, Vol. 95, pp. 174-82.

Gronn, R. (1999), “Substituting for leadership: the neglected role of the leadership couple”, Leadership Quarterly, Vol. 10, pp. 41-62.

Harris, M.M. and Schaubroeck, J.A. (2002), “A meta-analysis of self-superior, self-peer, and peer-supervisor ratings”, Personnel Psychology, Vol. 41, pp. 43-62.

Hoffman, C.C., Nathan, B.R. and Holden, L.M. (1991), “A comparison of validation criteria: objective versus subjective performance measures and self-and superior-ratings”, Personnel Psychology, Vol. 44, pp. 601-19.

Hogan, R. and Hogan, J. (2001), “Assessing leadership: a view from the dark side”, International Journal of Selection & Assessment, Vol. 9 Nos 1/2, pp. 40-51.

Hogan, R., Curphy, G.J. and Hogan, J. (1994), “What we know about leadership”, American Psychologist, Vol. 493, p. 504.

Hogan, R.J., Raskin, R. and Fazzini, D. (1990), “The dark side of charisma”, in Clark, K.E. and Clark, M.B. (Eds), Measures of Leadership, Leadership Library of America, West Orange, NJ, pp. 343-54.

Hollenbeck, G.P., McCall, M.W. Jr and Silzer, R.F. (2006), “Leadership competency models”, Leadership Quarterly, Vol. 17, pp. 398-413.

House, R.J., Hanges, P.J., Ruiz-Quintanilla, S.A., Dorfman, P.W., Javidan, M., Dickson, M. and Gupta, V. (1999), “Cultural influences on leadership and organizations: Project Globe”, Advances in Global Leadership, Vol. 1, pp. 171-233.

Howell, J.P. (1997), “‘Substitutes for leadership: their meaning and measurement’ – an historical assessment”, Leadership Quarterly, Vol. 8, pp. 113-6.

Howell, J.P., Dorfman, J.W. and Kerr, S. (1986), “Moderator variables in leadership research”, Academy of Management Review, Vol. 11, pp. 88-102.

Iles, P. and Macaulay, M. (2007), “Putting principles into practice: developing ethical leadership in local government”, International Journal of Public Sector Leadership, Vol. 3, pp. 15-28.

Jermier, J.M. and Kerr, S. (1997), “‘Substitutes for leadership: Their meaning and measurement’ – Contextual recollections and current observations”, Leadership Quarterly, Vol. 8, pp. 95-101.

Kelly, A., Johnson, P. and Gill, R. (2006), “Time for a change: a UK model of transformational leadership”, paper presented at the British Academy of Management conference, Belfast.

Kelly, G.A. (1955), Psychology of Personal Constructs, Vols. 1 and 2, Norton, New York, NY.

Kerr, S. and Jermier, J. (1978), “Substitutes for leadership: their meaning and measurement”, Organizational Behavior and Human Performance, Vol. 22, pp. 374-403.

Liden, R. and Antonakis, J. (2009), “Considering context in psychological leadership research”, Human Relations, Vol. 62, pp. 1587-605.

Misumi, J. (1985), The Behavioral Science of Leadership: an Interdisciplinary Research Program, University of Michigan, Ann Arbor, MI.

Mintzberg, H. (1999), “Managing quietly”, Leader to Leader, Vol. 12, Spring, pp. 24-30.

Nilsen, D. and Campbell, D.P. (1993), “Self-observer ratings discrepancies: Once an overrater, always an overrater?”, Human Resources Management, Vol. 32, pp. 265-81.

Engaging leadership scale

Northouse, P.G. (2010), Leadership: Theory and Practice, 5th ed., Sage, London.

Parker, C.P., Baltes, B.B., Young, S.A., Huff, J.W., Altman, R.A., Lacost, H.A. and Roberts, J.E. (2003), “Relationship between psychological climate perceptions and work outcomes: a meta-analytic review”, Journal of Organizational Behavior, Vol. 24, pp. 389-416.

Parry, K.W. (1998), “Grounded theory and social processes: a new direction for leadership research”, Leadership Quarterly, Vol. 9, pp. 85-105.

Patterson, M., Warr, P. and West, M. (2004), “Organizational climate and company productivity: the role of employee affect and employee level”, Journal of Occupational & Organizational Psychology, Vol. 77, pp. 193-216.

Podsakoff, P.M. and MacKwenzie, S.B. (1997), “Kerr and Jermier’s substitutes for leadership model: Background, empirical assessment, and suggestions for further research”, Leadership Quarterly, Vol. 8, pp. 117-25.

Podsakoff, P.M. and Organ, D.W. (1986), “Self-reports in organizational research: Problems and prospects”, Journal of Management, Vol. 12, pp. 531-44.

Shamir, B. (1995), “Social distance and charisma: theoretical notes and an exploratory study”, Leadership Quarterly, Vol. 6, pp. 19-47.

Spillane, J., Halverson, R. and Diamond, J. (2001), “Towards a theory of leadership practice: a distributed perspective”, working paper, Institute for Policy Research, Northwestern University, Evanston, IL.

Stordeur, S., Vandenberghe, C. and D’Hoore, W. (1999), “An examination of substitutes for leadership within a Belgium nursing context”, paper presented at the VI European Conference on Organizational Psychology and Health Care, University of Ghent and Jan Palfijn Hospital, Belgium.

Tourrish, D. and Vatcha, N. (2005), “Charismatic leadership and corporate cultism at Enron: The elimination of dissent, the promotion of conformity and organizational collapse”, Leadership, Vol. 1, pp. 455-80.

van Breukelen, W., Schyns, B. and Le Blanc, P. (2006), “Leader-member exchange theory and research: accomplishments and future challenges”, Leadership, Vol. 2, pp. 295-316.

Van der Heijden, B. and Nijhof, A. (2004), “The value of subjectivity: problems and prospects for 360-degree appraisal systems”, International Journal of Human Resource Management, Vol. 15, pp. 493-511.

Walumbwa, F.O., Luthans, F., Avey, J.B. and Oke, A. (2009), “Authentically leading groups: the mediating role of collective psychological capital and trust”, Journal of Organizational Behavior, online.

Xenikou, A. and Simosi, M. (2006), “Organisational culture and transformational leadership as predictors of business unit performance”, Journal of Management Psychology, Vol. 21, pp. 566-79.

Yeomans, K.A. (1968), Introducing Statistics, Vol. 1, Penguin, Harmondsworth.