ASAP reflection papers (3) based on 5 articles each - Human Resource Management

Measuring Executive Coaching Efficacy? The Answer Was Right Here All the Time

Ruth L. Orenstein Rutgers, The State University of New Jersey and Princeton Consulting Resources, Inc.

This article demonstrates that executive coach- ing efficacy can be measured empirically. It describes the application of C. P. Alderfer’s & L. D. Brown’s (1972) “Empathic Organic Ques- tionnaire” to executive coaching by summariz- ing a case in which it was adapted and utilized, by detailing the instrument’s construction and administration, and by reporting the results of paired sample t tests of 40 items rated by 20 respondents representing a diverse sample of the client’s organization. The findings support the hypothesis that the coaching client would be rated as changing most the behaviors directly related to stated coaching objectives; next, be- haviors indirectly related to objectives, and least, behaviors not addressed in coaching. It concludes by considering the inextricability of sound practice and sound measurement.

Keywords: executive coaching efficacy, out- come measurement, Empathic Organic Ques- tionnaire, valuation methodology, organiza- tional psychology

Over the course of the last decade, for- midable dramas have been inexorably un- folding across the organizational land- scapes of this country. Leadership crises in every sector of our society have eroded trust and confidence in those running our businesses, governmental agencies, and re- ligious institutions. The pervasiveness of downsizing as a “quick fix” solution to declining stock prices has virtually elimi- nated middle management and placed heavy burdens, both professionally and psychologically, on those who remain. The increasing disappearance in executive of- fices of individuals beyond early middle age has vastly diminished a resource with

the natural capacity for wisdom and mentorship.

Against this backdrop, and arguably as a direct consequence, a remarkable phenom- enon has emerged: the staggering demand for executive coaching, a one-on-one inter- vention with a senior manager for the pur- pose of improving or enhancing manage- ment skills. Reported to be a market worth $1 billion that is expected to double by the end of 2005 (“Corporate therapy,” 2003), executive coaching is provided across cor- porate, governmental and nonprofit sectors by, according to recent reports, over 10,000 practitioners in the United States (Rivera, 2002) and 15,000 worldwide (Greco, 2001). Executive coaches come from a multiplicity of disciplines with a broad range of credentials and use a diverse array of techniques in service of improving the

Ruth L. Orenstein is President of Princeton Consulting Resources, Inc., and on the visiting faculty of the Graduate School of Applied and Professional Psychology at Rutgers, the State University of New Jersey. She is a member of the Board of Directors of the Graduate School Alliance for Executive Coaching.

I would like express my gratitude to Skylar Buono, Nancy Fagley, Cary Cherniss, and An- drew Simon for their valuable contributions to the creation of this article and my indebtedness to Clayton P. Alderfer for inspiring me to con- tinuously strive for professional excellence.

Correspondence concerning this article should be addressed to Ruth L. Orenstein, Princeton Con- sulting Resources, Inc., 625 Sayre Drive, Prince- ton, NJ 08540. E-mail: [email protected]

Copyright 2006 by the American Psychological Association and the Society of Consulting Psychology, 1065-9293/06/$12.00 DOI: 10.1037/1065-9293.58.2.106

Consulting Psychology Journal: Practice and Research, Vol. 58, No. 2, 106 –116

106

performance of the managers with whom they work (Judge & Cowell, 1997).

In view of the enormous growth in the practice, a substantive literature regarding the field has remained relatively limited. Even more limited are contributions re- garding outcome evaluation and the effi- cacy of executive coaching. Empirical stud- ies that currently exist, primarily the result of dissertations and fieldwork by graduate students, appear promising but remain in- conclusive (Kampa-Kokesch & Anderson, 2001). Recent articles clearly articulate the need for scientific evaluation but note the difficulty of measuring efficacy in an evolving field (Kilburg, 2004; Lowman, 2005). Consequently, the authors promote the creation of case studies to produce a database from which a statistically sound evaluation process may eventually emerge, and they look to the medical and psycho- therapeutic traditions for narrative proto- types (Kilburg, 2004; Lowman, 2001). Doing so, however, preserves the very con- ceptual frameworks that, although predom- inant in executive coaching, have thus far failed to offer a viable construct for mea- suring its efficacy.

There exists, however, another body of knowledge that provides a robust context for disciplined inquiry in executive coach- ing: organizational psychology. With its emphasis on systems thinking, organiza- tional psychology broadens focus beyond the individual and provides the mechanism with which to view executive coaching as a complex, multidimensional organizational intervention (Orenstein, 2000, 2002). When guided by this theoretical perspec- tive, executive coaching is not a fledgling profession but, rather, another facet of a time-honored field— one with well estab- lished empirical diagnostic and evaluation methods that are as applicable to executive coaching as to any other aspect of organi- zational practice.

The purpose of this article is to demon- strate how executive coaching efficacy can

be measured empirically: by introducing a methodology drawn from the field of orga- nizational psychology; by summarizing an executive coaching case in which it was adapted and utilized; by describing how the evaluation instrument—the Empathic Or- ganic Questionnaire—was constructed and administered; by reporting and discussing results; and finally, by considering the in- extricability of sound practice and sound measurement.

The Empathic Organic Questionnaire

The Empathic Organic Questionnaire is an instrument whose content is derived from the language and experience of orga- nizational members in a system under study. It is designed for maximum rele- vance to the client organization (Alderfer & Brown, 1972).

The methodology originated during a 4-year system-wide consultation by Alder- fer and Brown (1975). Countering the pre- vailing method of superimposing on the organization the pristine theoretical per- spective of researchers who attempted to remove themselves from their subjects, Al- derfer and Brown engaged in a deliberate effort to involve organizational members in a joint process of inquiry. Utilizing the information gleaned in earlier phases of diagnosis through in-depth interviewing and observation, the methodology demon- strated the ability of the consultants to iden- tify significant areas of organizational ex- perience from the perspective of those within the system. Benefits included in- creased relevance, decreased distance be- tween respondents and consultants, and as a result, greater respondent openness and concomitant improvement in the breadth and depth of information flow, particularly in those areas most threatening to the organization.

A new theory of organizational consul- tation emerged during the course of the work. This theory emphasized the signifi- cance of the mutuality of the relationship

107Consulting Psychology Journal: Practice and Research Spring 2006

between the client and the consultant as well as the psychological acceptance of change by all relevant individuals and groups in the system, and it presented a consultation process that consisted of four iterative and interdependent phases: entry, data collection and diagnosis, implementa- tion, and evaluation. Given the nature of the process, each phase not only built upon the work of those preceding it but also contained elements of all others. The Em- pathic Organic Questionnaire, first con- ceived as a tool for the diagnostic phase, was thus able to be utilized again in the evaluation phase to measure the effective- ness of the entire consultation.

This work is the foundation upon which the use of the organic instrument for the evaluation of executive coaching can be solidly built.

Evaluating Executive Coaching: An Illustrative Case1

The Case History

The request to work with this client, a senior executive in a large state agency, came from his immediate manager. The latter explained that the client had fla- grantly dismissed the results of a recently conducted 360-degree assessment which had indicated, in addition to weakness in the areas of communication, interpersonal relations, and motivation, that the execu- tive’s self-evaluation in all areas was con- siderably higher than those of his peers, subordinates, and manager. The manager also described the executive as arrogant, domineering, and defensive.

When I met with the client, he was cold and aloof. He stated that the assessment had been of no value to him because although the results of the standardized instrument indicated what people thought, it explained neither why they arrived at their conclu- sions nor how he could improve. Further- more, his manager had chosen the respon- dents and insisted that he would not iden-

tify them, so the client was precluded from speaking with them to discover the answers to his questions. The client had concluded that the results were meaningless because he assumed they had been skewed by one individual on his staff with whom he had had significant problems for many years. Interactions with his manager had become strained over the course of the previous year, so communication had been limited between them and their meeting to discuss the results had worsened their relationship. He was unwilling to undergo any similar assessment process.

When I described the process that I uti- lized (Orenstein, 2000, 2002), the client became more responsive. He quickly rec- ognized how the emphasis on his (rather than his manager’s) ownership of the pro- cess, qualitative interviewing, and observa- tion to uncover the reasons for perceptions, in-depth oral as well as written feedback, establishing specific coaching objectives, and personalized coaching sessions, all with complete confidentiality of data, dif- ferentiated this from the experience he had recently undergone. He decided, albeit still with some skepticism, to engage in execu- tive coaching. He selected 15 people, in- cluding senior managers, peers, subordi- nates, and customers, with whom semi- structured, one-on-one interviews were to be conducted, and the assessment phase (Orenstein, 2000) of the coaching process began.

The feedback that I shared with him after the interview data had been aggre- gated and analyzed was revelatory to him. He was disappointed (and more than a little surprised) that it corroborated some of the standardized data and that it could not be

1 The purpose of the case as discussed here is to demonstrate the use of the organic instrument. It therefore intentionally omits details regarding the specific coaching interventions utilized. For a more complete discussion of the author’s ap- proach, readers are directed to Orenstein, 2002.

108 Consulting Psychology Journal: Practice and Research Spring 2006

attributed to a single disgruntled employee, but he was relieved that he could ask ques- tions and learn the underlying reasons for the findings. In addition, he was gratified to learn of the strengths that had been identi- fied and that they could be leveraged during the coaching phase. He also seemed fasci- nated by the explanation of organizational and group forces that had surfaced over the course of the interviews. The coaching ob- jectives he formulated fell within the orig- inally identified areas of motivation, com- munication, and interpersonal skills, but were much more specific: to increase rec- ognition of staff; to become more approach- able, caring and patient; and to eliminate public criticism, fear, and intimidation.

During the intensive coaching that fol- lowed, the client remained highly moti- vated. An initial reluctance to divulge any personal information to me disappeared af- ter the feedback session, and we were able to discuss how systemic forces were trig- gering unconscious and self-defeating de- fense structures. Our interactions, which had previously caused me to question whether a successful working alliance could be established, now evoked compas- sion and empathy. This mirrored the posi- tive responses he was receiving from his staff, including many of those who had been involved in the interview process. They reinforced his efforts, and he was excited and pleased. At the end of 6 months, he felt confident that he had achieved the stated coaching objectives, and we met with the client’s manager for his corroboration. The latter agreed that he had seen dramatic differences in the cli- ent’s behavior and expressed a desire to conduct another 360-degree assessment that would provide some quantitative indi- cation of outcomes. When I described the organic questionnaire and suggested using it in place of the standardized instrument, both the client and the manager readily agreed. We decided to wait another 6 months before conducting the evaluation in

order to make certain that the behavior changes, and perceptions about them, were permanent.

The Evaluation Method

The evaluation was conducted in a 4-step process:

1. Designing the instrument; 2. Selecting the respondents; 3. Administering the questionnaire; 4. Analyzing the data.

Designing the Instrument

The executive coaching process in the preceding section was guided by the same theoretical and implementational frame- work as the previously discussed 4-phased organizational diagnosis process (Alderfer & Brown, 1975). Replicating the design of the Empathic Organic Questionnaire was therefore a natural outcome. The only dis- tinction was focus. Heretofore, the instru- ment had been used to address systemic issues (Alderfer & Brown, 1972; Alderfer, Alderfer, Tucker, & Tucker, 1980) as part of the general method of organizational di- agnosis (Alderfer, 1980); this time its focus was on individual change within an orga- nizational role. Consequently, in keeping with the very nature of the organic instru- ment, the content would change, but the steps in the design process would remain virtually the same. They are described below.

First, broad categories for examination were selected. A review of the client’s feedback report, the client’s specific coach- ing objectives, and the focus throughout one-on-one coaching, clearly indicated that emphasis should be placed on outcomes in the areas of motivation, communication, and interpersonal skills. Next, direct quotes and phrases relevant to the identified cate- gories were extracted from the 15 qualita- tive interviews conducted at the start of the executive coaching process; some were di- rectly related to the specific coaching ob-

109Consulting Psychology Journal: Practice and Research Spring 2006

jectives, some indirectly. The quotes and phrases were then converted into question- naire items by expressing a single thought for each, presenting each thought positively and negatively to eliminate bias (Miller & Fagley, 1991), and then selecting an ap- proximately equal number of items for each category. To enhance statistical soundness, additional methods for arriving at the data were then added (Campbell & Fiske, 1959): to strengthen reliability and convergent va- lidity, an open-ended question was insert- ed; to ensure discriminant validity and to counteract the demand characteristic caused by retrospective ratings, a control category consisting of items considered strengths (and therefore areas for which change was neither desired nor sought) was included; and to further eliminate bias, the order of all items was randomly mixed. Finally, the items were converted to past tense and again randomly reordered. The final instru- ment consisted of the past and present tense versions (see Appendix).

Going into the evaluation, the hypothe- sis was as follows: the coaching client will be rated by relevant others as changing most the behaviors directly related to the stated coaching objectives (i.e., coaching outcomes); next, the behaviors that are in- directly related to the objectives (i.e., in the same broad categories but not specifically identified as outcomes); and least, the be- haviors not addressed in the coaching pro- cess (the control items).

Selecting the Respondents

Twenty respondents were identified— the original 15 and 5 additional individuals selected for diversity in age, gender, race, ethnicity, and hierarchical level in the man- agement structure. This resulted in a re- spondent group representative of the demo- graphics of the organization: 12 males (10 Caucasians, 1 African American, and 1 Hispanic) ranging in age from 26 to 63, and 8 females (3 Caucasians, 2 African Amer- icans, and 2 Hispanics) ranging in age from

32 to 52, representing levels from first-line supervisor through vice president.

Administering the Questionnaire

The client called the respondents to re- quest participation in the evaluation pro- cess and to notify them that a consultant would be calling. In order to avoid bias, the questionnaire was administered by a con- sultant whom the respondents had not pre- viously met; and in order to assure comple- tion, the consultant called each respondent to arrange an individual appointment, ask- ing that the respondent secure a private room in order to complete the question- naire. At the scheduled time, the consultant reviewed the procedure, including both the voluntary nature of participation and the manner in which confidentiality would be safeguarded. She then gave each individual the instrument together with an unmarked envelope in which to seal it, left the room, and waited outside until the respondent was finished and handed her the sealed envelope.

Analyzing the Data

When all the questionnaires were col- lected, the open-ended data in each document was reviewed and compared to the numerical responses for validation purposes (i.e., to make certain that positive responses matched positive statements, and vice versa). All open-ended data were then copied verbatim to a separate document. Next, each of the past tense items was matched to the corresponding present tense item. Negatively worded items were reverse scored in order to achieve nu- merical consistency for analytical purposes. A paired samples t test was performed on all items with missing data eliminated on an analysis-by-analysis basis.

Results

The results of the t tests are presented in three categories. The first category (see Ta- ble 1) is composed of those items directly

110 Consulting Psychology Journal: Practice and Research Spring 2006

related to the coaching objectives; the sec- ond category (see Table 2) consists of those items indirectly related to the coaching ob- jectives; and the third category (see Table 3) comprises the control items, considered unrelated to the coaching objectives.

As shown, 15 of the 19 items in the first category are significant, six at an alpha level of .05, six at an alpha level of .01, and three at an alpha level of .001. In the sec- ond category, four of the 11 items are sig-

nificant, two at an alpha of .05 and two at .01. In the third category, none of the items is significant.

The results are summarized in Table 4, which shows the degrees of change in each category as a whole.

The hypothesis, then, was supported by the results: the coaching client was rated by relevant others as changing most the behav- iors directly related to the stated coaching objectives; next, the behaviors that were

Table 1 Behaviors Directly Related to Coaching Objectives

Item Mean past Mean present N p

He belittles me. (R) 3.8 4.1 18 .38 He cares about me. 2.7 3.6 18 .01** I find him personable and approachable. 2.8 4.1 20 .00** He does not value my opinion. (R) 3.4 3.7 19 .43 He gives me credit when I do a good job. 3.7 4.2 18 .01* I consider him a bully. (R) 2.9 3.8 18 .00** He interrupts me when I speak. (R) 2.7 3.5 19 .00*** He helps me be the best I can be. 3.2 3.8 19 .02* I see him criticize others in public. (R) 2.6 4.2 17 .00*** He is impatient with me. (R) 2.3 3.6 19 .00*** He is cold to me. (R) 3.2 4.6 20 .01** I don’t know how to read him. (R) 2.7 3.7 19 .00** He is not there when I need him. (R) 3.8 4.3 19 .04* He isolates himself from his staff. (R) 3.5 4.4 19 .03* He listens when I speak to him. 3.7 4.2 20 .04* He welcomes it when I challenge him. 2.8 3.2 18 .05* I am afraid to be honest with him. (R) 3.8 4.3 20 .07 He intimidates me. (R) 3.4 3.8 20 .23 When I meet with him, I am on guard. (R) 2.9 3.9 19 .00**

Note. (R) indicates reverse scored item * p � .05. ** p � .01. *** p � .001.

Table 2 Behaviors Indirectly Related to Coaching Objectives

Item Mean past Mean present N p

He behaves in unpredictable ways. (R) 3.1 3.5 18 .36 I notice that his moods changed quickly. (R) 2.6 3.4 18 .08 He is fair to me. 3.8 4.1 20 .08 He is not interested in details. (R) 2.1 2.6 19 .04* He is not visible to customers. (R) 3.4 4.2 19 .01* He talks above my level. (R) 4.1 4.3 19 .27 I consider him opinionated. (R) 1.8 2.1 18 .24 He does not hold a grudge. 3.4 3.7 19 .40 I find him easy to get along with. 3.3 3.6 20 .27 I observe him leading by example. 3.6 4.2 18 .00** I see him get upset quickly. (R) 2.2 3.7 18 .00**

Note. (R) indicates reverse scored item * p � .05. ** p � .01.

111Consulting Psychology Journal: Practice and Research Spring 2006

indirectly related to the objectives; and least, the behaviors not addressed in the coaching process.

Conclusion

As demonstrated by the above results, executive coaching efficacy and outcomes can be measured, and the organic question- naire is a statistically sound methodology with which to do so. It thus appears to be a highly promising vehicle for contributing to the empirical base for coaching efficacy.

There are, nevertheless, two major con- siderations for its use. First and foremost is the inextricability of sound practice and sound measurement. Even the best of mea- surements cannot replace the essentials of effective executive coaching—a motivated client, a supportive system, a qualified con- sultant, and a well conceived approach. The Empathic Organic Questionnaire is no ex- ception. It can only be utilized within the confines of a comprehensive process that supports it. This process must be one that encompasses a number of fundamental and

interdependent elements. The first element is the appropriate diagnosis and navigation of organizational boundaries for successful entry. The second is the establishment of a strong working alliance with the coaching client. The third is the conducting of in- depth qualitative interviews with relevant participants that provide the basis for the content for the organic instrument, involve the respondents in the client’s process of change, and enable the development of re- lationships that will increase the likelihood of willing participation in postcoaching as- sessment. The fourth is intensive feedback aggregated from the interview data as well as from the internal experience of the coach. The fifth is the formulation of spe- cific coaching objectives and outcome cri- teria. The sixth is the formulation and ex- ecution of interventions appropriate to the attainment of those outcomes. The seventh is the maintenance of incontrovertible con- fidentiality boundaries throughout.

The other consideration involves the rig- ors of instrument design and administra- tion. The organic questionnaire must be constructed on a case-by-case basis, each time following the requirements for statis- tical validity. By its very nature and what it seeks to measure, the content cannot be standardized, even when individuals being coached are part of the same client system. In addition, to ensure reliability, it must be administered in such a way as to maximize

Table 3 Strengths (Control Category)

Item Mean past Mean present N p

He backs me up when I made a mistake. 3.6 3.8 18 .24 He demonstrates strong organizational skills. 4.8 4.8 20 1.00 He gets the job done. 4.6 4.6 19 1.00 I consider him a visionary. 3.9 4.2 17 .08 He gives the right job to the right person. 3.9 4.2 18 .22 He hinders me from growing professionally. (R) 3.8 4.2 18 .11 I enjoy his sense of humor. 3.7 3.8 18 .50 He is straightforward and clear in our communications. 4.1 4.4 20 .08 I trust him. 3.7 3.9 20 .23 He sets high standards of performance. 4.6 4.5 19 .58

Note. (R) indicates reverse scored item

Table 4 Mean Differences by Category

Category Mean past

Mean present

Mean difference

Directly related 3.1 4.0 0.9 Indirectly related 3.0 3.6 0.6 Control 4.1 4.2 0.1

112 Consulting Psychology Journal: Practice and Research Spring 2006

willing participation from selected respon- dents who constitute a representative sam- ple of the universe of relevant others. It is thus a time consuming and labor-intensive tool, and all those involved must believe in the worthiness of the evaluation effort.

There are, of course, a number of impli- cations for consulting psychologists. One of the most salient relates to education and train- ing for executive coaches. To date, discussion regarding the knowledge and skills required for the field have centered on what is needed for practice (Kilburg, 1996; Brotman, Liberi, & Wasylyshyn, 1998). Using the evaluation methodology described here also makes req- uisite competence in research, with an em- phasis on research design and statistical anal- ysis. Equally significant is the necessity of adopting a replicable coaching process that is capable of being clearly articulated to all those who become involved, whether as cli- ent or relevant other. Finally, and perhaps most compelling, is the obligation to take an investigative and reflective stance in relation to one’s own work, thereby being willing to accept, with brutal honesty, when that work has not been effective and when it must be refined, transformed, or even abandoned in favor of more efficacious approaches.

The methodology is therefore not for ev- eryone involved in the practice of executive coaching. Those in the field, however, who are guided by theory, who are committed to substance, and who are serious about efficacy may find it a promising avenue for further investigation and research. It is for the latter that this article has been written.

References

Alderfer, C. P. (1980). The methodology of or- ganization diagnosis. Professional Psychol- ogy, 11, 462– 485.

Alderfer, C. P., Alderfer, C. J., Tucker, R. C., & Tucker, L. M. (1980). Diagnosing race rela- tions in management. Journal of Applied Be- havioral Science, 16, 135–166.

Alderfer, C. P., & Brown, L. D. (1972). Design- ing an “empathic questionnaire” for organi-

zational research. Journal of Applied Psy- chology, 56, 456 – 460.

Alderfer, C. P., & Brown, L. D. (1975). Learn- ing from changing: Organizational diagno- sis and development. Beverly Hills: Sage.

Brotman, L. E., Liberi, W. P., & Wasylyshyn, K. M. (1998). Executive coaching: The need for standards of competence. Consulting Psychology Journal: Practice and Re- search, 50, 40 – 46.

Campbell, D. T., & Fiske, D. W. (1959). Con- vergent & discriminant validation by the multi-trait multi-method matrix. Psycholog- ical Bulletin, 56, 81–105.

Corporate Therapy. (2003, November 23). The Economist print ed., 1–2.

Greco, J. (2001). Hey, coach. The Journal of Business Strategy, 22, 28 –31.

Judge, W. Q., & Cowell, J. (1996). The brave new world of executive coaching. Business Horizons, July-August, 71–77.

Kampa-Kokesch, S., & Anderson, M. Z. (2001). Executive coaching: A comprehensive re- view of the literature. Consulting Psychology Journal: Practice and Research, 53, 205– 228.

Kilburg, R. R. (1996). Toward a conceptual understanding and definition of executive coaching. Consulting Psychology Jour- nal, 48, 134 –144.

Kilburg, R. R. (2004). Trudging toward dodoville: Conceptual approaches and case studies in executive coaching. Consulting Psychology Journal: Practice and Re- search, 56, 203–213.

Lowman, R. L. (2001). Constructing a literature from case studies: Promises and limitations of the method. Consulting Psychology Jour- nal: Practice and Research, 53, 119 –123.

Lowman, R. L. (2005). Executive coaching: The road to dodoville needs paving with more than good intentions. Consulting Psychology Journal: Practice and Research, 57, 90 –96.

Miller, P. M., & Fagley, N. S. (1991). The effects of framing, problem variations, and providing rationale on choice. Personality and Social Psychology Bulletin, 17, 517–522.

Orenstein, R. L. (2000). Executive coaching: An integrative model. Dissertation Abstracts In- ternational, 61, AAT 9971459. chap. 4, pp. 34 –36; chap. 7, pp. 73– 84.

Orenstein, R. L. (2002). Executive coaching: It’s not just about the executive. Journal of Applied Behavioral Science, 38, 355–374.

Rivera, P. R. (2002, September 29). Averting risks of coach craze. The Boston Globe, p. G2.

113Consulting Psychology Journal: Practice and Research Spring 2006

Appendix follows on next page

Appendix

Executive Coaching Evaluation

Dear Evaluator: The purpose of this evaluation is to discover the effects, if any, of executive coaching on the

behaviors of those who undergo the process. Therefore, your perceptions in the present and the past are crucial.

This evaluation has four sections: • Section I asks you to reflect on your interactions with XXXX approximately 9–12 months ago

and to describe his professional behavior at that time. • Section II contains a series of statements about past behaviors. Please indicate the extent to

which you agree or disagree with each in relation to XXXX. • Section III asks you to reflect on your interactions with XXXX currently and to describe his

professional behavior at the present time. • Section IV contains the same statements about present behaviors. Again, please indicate the

extent to which you agree or disagree with each in relation to XXXX.

All of your responses will be treated with complete confidentiality. Thank you very much for providing this information.

Section I

Directions: Please reflect on your interactions with XXXX approximately 9 –12 months ago. In the space below, briefly describe what XXXX was like as a professional. Then please answer the questions in the following sections.

Section II

Directions: Please indicate to what extent you agree or disagree with the following statements about XXXX as they describe his behavior approximately 9 –12 months ago. Please place a check mark in the appropriate box.

Key: 1. Disagree completely; 2. Disagree somewhat; 3. Neither agree nor disagree; 4. Agree somewhat; 5. Agree completely

1 2 3 4 5

1. He backed me up when I made a mistake.

2. He behaved in unpredictable ways.

3. He belittled me.

4. He cared about me.

5. I found him personable and approachable.

6. I noticed that his moods changed quickly.

7. He demonstrated strong organizational skills.

8. He did not value my opinion.

9. He got the job done.

114 Consulting Psychology Journal: Practice and Research Spring 2006

Section II. (continued)

1 2 3 4 5

10. He gave me credit when I did a good job.

11. I considered him a visionary.

12. I considered him a bully.

13. He gave the right job to the right person.

14. He hindered me from growing professionally.

15. He interrupted me when I spoke.

16. He was fair to me.

17. He helped me be the best I could be.

18. I saw him criticize others in public.

19. He was impatient with me.

20. He was cold to me.

21. He was not interested in details.

22. I didn’t know how to read him.

23. I enjoyed his sense of humor.

24. He was not there when I needed him.

25. He was not visible to customers.

26. He was straightforward and clear in our communications.

27. He isolated himself from his staff.

28. He listened when I spoke to him.

29. He talked above my level.

30. He welcomed it when I challenged him.

31. I was afraid to be honest with him.

32. I considered him opinionated.

33. He did not hold a grudge.

34. I found him easy to get along with.

35. I observed him leading by example.

36. He intimidated me.

37. I saw him get upset quickly.

38. I trusted him.

39. When I met with him, I was on guard.

40. He set high standards of performance.

Section III

Directions: Please reflect on your interactions with XXXX currently. In the space below, briefly describe what XXXX is like as a professional. Then please answer the questions in the following section.

Appendix continues

115Consulting Psychology Journal: Practice and Research Spring 2006

Section IV

Directions: Please indicate to what extent you agree or disagree with the following statements about XXXX as they describe his behavior currently. Please place a check mark in the appropriate box.

Key: 1. Disagree completely; 2. Disagree somewhat; 3. Neither agree nor disagree; 4. Agree somewhat; 5. Agree completely

1 2 3 4 5

1. He gives the right job to the right person.

2. I find him personable and approachable.

3. He is not interested in details.

4. He is fair to me.

5. He isolates himself from his staff.

6. He listens when I speak to him.

7. He is not there when I need him.

8. I observe him leading by example.

9. I find him easy to get along with.

10. He helps me be the best I can be.

11. He hinders me from growing professionally.

12. He behaves in unpredictable ways.

13. When I meet with him, I am on guard.

14. He gives me credit when I do a good job.

15. I am afraid to be honest with him.

16. I see him criticize others in public.

17. I trust him.

18. I consider him opinionated.

19. I notice that his moods change quickly.

20. He is impatient with me.

21. I don’t know how to read him.

22. I enjoy his sense of humor.

23. He cares about me.

24. I consider him a visionary.

25. I consider him a bully.

26. He demonstrates strong organizational skills.

27. He interrupts me when I speak.

28. He does not value my opinion.

29. He backs me up when I make a mistake.

30. He welcomes it when I challenge him.

31. He intimidates me.

32. He sets high standards of performance.

33. He belittles me.

34. He is straightforward and clear in our communications.

35. I see him get upset quickly.

36. He is not visible to customers.

37. He is cold to me.

38. He does not hold a grudge.

39. He talks above my level.

40. He gets the job done.

116 Consulting Psychology Journal: Practice and Research Spring 2006