homework
© P
ix m
an n/
Im ag
ez oo
/ G
et ty
Im ag
es
6
Collecting, Analyzing, and Feeding Back Diagnostic Information
learning objectives
Understand the importance of the diagnostic relationship in the organization development (OD) process.
Describe the methods for collecting diagnostic data.
Understand the primary techniques used to analyze diagnostic data.
Outline the process issues associated with data feedback.
Describe and evaluate the survey feedback intervention.
Organization development is vitally depen-dent on collecting diagnostic informationthat will be shared with the client in jointly assessing how the organization is functioning and determining the best change intervention. The qual- ity of the information gathered and the effective- ness of the feedback process, therefore, are critical parts of the OD process. In this chapter, we discuss several key issues associated with col- lecting, analyzing, and feeding back diagnostic data on how an organization or department functions.
Data collection involves gathering information on specific organizational features, such as the inputs, design components, and outputs presented in Chapter 5. The process begins by establishing an effective relationship between the organization development (OD) practitioner and those from
whom data will be collected and then choosing data collection techniques. Four methods can be used to collect data: questionnaires, interviews, observations, and unobtrusive measures. Data analysis organizes and examines the information to make clear the underlying causes of an organizational problem or to identify areas for future development. Data feedback presents diagnostic information to organizational members so they can understand it and draw action implications from it. Effective feedback involves attention to both the content and the process of data feedback. A popular technique for feeding back questionnaire data is called survey feedback. Its central role in many large-scale OD efforts warrants a special look. The overall process of data collection, analysis, and feedback is shown in Figure 6.1.
6-1 The Diagnostic Relationship In most cases of planned change, OD practitioners play an active role in gathering data from organization members for diagnostic purposes. For example, they might interview members of a work team about causes of conflict among members; they might survey
123
employees at a large industrial plant about factors contributing to poor product quality. Before collecting diagnostic information, practitioners need to establish a relationship with those who will provide and subsequently use it. Because the nature of that relation- ship affects the quality and usefulness of the data collected, it is vital that OD practitioners clarify for organization members who they are, why the data are being collected, what the data gathering will involve, and how the data will be used.1 That information can help allay people’s natural fears that the data might be used against them and gain members’ participation and support, which are essential to developing successful interventions.
Establishing the diagnostic relationship between the OD practitioner and relevant organization members is similar to forming a contract. It is meant to clarify expectations and to specify the conditions of the relationship. In those cases where members have been directly involved in the entering and contracting process described in Chapter 4, the diagnostic contract will typically be part of the initial contracting step. In situations where data will be collected from members who have not been directly involved in enter- ing and contracting, however, OD practitioners will need to establish a diagnostic con- tract as a prelude to diagnosis. The answers to the following questions provide the substance of the diagnostic contract:2
1. Who am I? The answer to this question introduces the OD practitioner to the orga- nization, particularly to those members who do not know the consultant and yet will be asked to provide diagnostic data.
2. Why am I here, and what am I doing? These answers are aimed at defining the goals of the diagnosis and data-gathering activities. The consultant needs to present the objectives of the action research process and to describe how the diagnostic activities fit into the overall developmental strategy.
3. Who do I work for? This answer clarifies who has hired the OD practitioner, whether it be a manager, a group of managers, or a group of employees and man- agers. One way to build trust and support for the diagnosis is to have those people directly involved in establishing the diagnostic contract. Thus, for example, if the
FIGURE 6.1
The Cycle of Data Collection and Feedback
SOURCE: Figure adapted from D. Nadler, Feedback and Organization Development, 1977, Pearson Education, Upper Saddle River, NJ.
124 PART 2 THE PROCESS OF ORGANIZATION DEVELOPMENT
consultant works for a joint labor–management committee, representatives from both sides of that group could help the consultant build the proper relationship with those from whom data will be gathered.
4. What do I want from you, and why? Here, the OD practitioner needs to specify how much time and effort people will need to give to provide valid data and subse- quently to work with these data in solving problems. Because some people may not want to participate in the diagnosis, it is important to specify that such involvement is voluntary.
5. How will I protect your confidentiality? This answer addresses member concerns about who will see their responses and in what form. This is especially critical when employees are asked to provide information about their attitudes or perceptions. Either OD practitioners can ensure confidentiality or state that full participation in the change process requires open information sharing. In the first case, employees are frequently concerned about privacy and the possibility of being punished for their responses. To alleviate concern and to increase the likelihood of obtaining honest responses, the con- sultant may need to assure employees of the confidentiality of their information, perhaps through explicit guarantees of response anonymity. In the second case, full involvement of the participants in their own diagnosis may be a vital ingredient of the change pro- cess. If sensitive issues arise, assurances of confidentiality can coopt the OD practitioner and thwart meaningful diagnosis. The consultant is bound to keep confidential the issues that are most critical for the group or organization to understand.3 OD practi- tioners must think carefully about how they want to handle confidentiality issues.
6. Who will have access to the data? Respondents typically want to know whether they will have access to their data and who else in the organization will have similar access. The OD practitioner needs to clarify access issues and, in most cases, should agree to provide respondents with their own results. Indeed, the collaborative nature of diagnosis means that organization members will work with their own data to dis- cover causes of problems and to devise relevant interventions.
7. What is in it for you? This answer is aimed at providing organization members with a clear delineation of the benefits they can expect from the diagnosis. This usu- ally entails describing the feedback process and how they can use the data to improve the organization.
8. Can I be trusted? The diagnostic relationship ultimately rests on the trust estab- lished between the OD practitioner and those providing the data. An open and hon- est exchange of information depends on such trust, and the practitioner should provide ample time and face-to-face contact during the contracting process to build this trust. This requires the consultant to listen actively and discuss openly all questions raised by participants.
Careful attention to establishing the diagnostic relationship helps to promote the three goals of data collection.4 The first and most immediate objective is to obtain valid information about organizational functioning. Building a data collection contract can ensure that organization members provide honest, reliable, and complete information.
Data collection also can rally energy for constructive organizational change. A good diagnostic relationship helps organization members start thinking about issues that con- cern them, and it creates expectations that change is possible. When members trust the OD practitioner, they are likely to participate in the diagnostic process and to generate energy and commitment for organizational change.
Finally, data collection helps to develop the collaborative relationship necessary for effecting organizational change. The diagnostic stage of action research is probably the
CHAPTER 6 COLLECTING, ANALYZING, AND FEEDING BACK DIAGNOSTIC INFORMATION 125
first time that most organization members meet the OD practitioner, and it can be the basis for building a longer-term relationship. The data collection contract and subsequent data-gathering and feedback activities provide members with opportunities for seeing the consultant in action and for knowing him or her personally. If the consultant can show employees that he or she is trustworthy, is willing to work with them, and is able to help improve the organization, then the data collection process will contribute to the longer- term collaborative relationship so necessary for carrying out organizational changes.
6-2 Collecting Data The four major techniques for gathering diagnostic data are questionnaires, interviews, observations, and unobtrusive measures. Table 6.1 briefly compares the methods and lists their major advantages and problems. No single method can fully measure the kinds of diagnostic variables important to OD because each has certain strengths and weaknesses.5 For example, perceptual measures, such as questionnaires and surveys, are open to self-report biases, such as respondents’ tendency to give socially desirable answers rather than honest opinions. Observations, on the other hand, are susceptible to observer biases, such as seeing what one wants to see rather than what is really there. Because of the biases inherent in any data collection method, more than one method should be used when collecting diagnostic data. If data from the different meth- ods are compared and found to be consistent, it is likely that the variables are being
TABLE 6.1
Strengths and Weaknesses of Different Data Collection Methods
Data Collection Method Primary Strengths Primary Weaknesses
Surveys and questionnaires
Member beliefs and attitudes can be quantified easily
Can gather large amount of data from many people
Inexpensive on a per-person basis
Relatively impersonal Mechanistic and rigid—assumes all
the right questions are asked Easy to “over interpret” the data Response bias
Interviews Very flexible—can adapt to interviewee and data collection subject
Data is “rich” Interview process builds rapport and
empathy
Relatively expensive Interviewer responses can be biased Difficult to code and interpret Self-report bias
Observations Collects data on actual behavior, rather than reports of behavior
Real time, not retrospective Adaptive and objective
Difficult to code and interpret Sampling may be inconsistent Observer bias and reliability can be
questioned Can be expensive
Unobtrusive measures
No response bias High face validity Easily quantified
Privacy, access, and retrieval difficulties Validity concerns Difficult to code and interpret
© Ce
ng ag
e Le
ar ni
ng 20
15
126 PART 2 THE PROCESS OF ORGANIZATION DEVELOPMENT
measured validly. For example, questionnaire measures of job discretion could be supple- mented with observations of the number and kinds of decisions employees are making. If the two kinds of data support each other, job discretion is probably being assessed accu- rately. If the two kinds of data conflict, the validity of the measures should be examined further—perhaps by using a third method, such as interviews.
6-2a Questionnaires One of the most efficient ways to collect data is through questionnaires. Because they typi- cally contain fixed-response queries about various features of an organization, these mea- sures can be administered to large numbers of people simultaneously. Also, they can be analyzed quickly, especially with the use of computers, thus permitting quantitative com- parison and evaluation. As a result, data can easily be fed back to employees. Numerous basic resource books on survey methodology and questionnaire development are available.6
Questionnaires can vary in scope, some measuring selected aspects of organizations and others assessing more comprehensive organizational characteristics. They also can vary in the extent to which they are either standardized or tailored to a specific organi- zation. Standardized instruments generally are based on an explicit model of organiza- tion, group, or individual effectiveness and contain a predetermined set of questions that have been developed and refined over time. For example, Table 6.2 presents a stan- dardized questionnaire for measuring the job-design dimensions identified in Chapter 5: skill variety, task identity, task significance, autonomy, and feedback about results. The questionnaire includes three items or questions for each dimension, and a total score for each job dimension is computed simply by adding the responses for the three rele- vant items and arriving at a total score from 3 (low) to 21 (high). The questionnaire has wide applicability. It has been used in a variety of organizations with employees in both blue-collar and white-collar jobs.
Several research organizations have been highly instrumental in developing and refining surveys. The Institute for Social Research at the University of Michigan (http:// home.isr.umich.edu) and the Center for Effective Organizations at the University of Southern California (http://ceo.usc.edu) are two prominent examples. Two of the Insti- tute’s most popular measures of organizational dimensions are the Survey of Organiza- tions and the Michigan Organizational Assessment Questionnaire. Few other instruments are supported by such substantial reliability and validity data.7 Other exam- ples of packaged instruments include Weisbord’s Organizational Diagnostic Question- naire, Dyer’s Team Development Survey, Cameron and Quinn’s Organizational Culture Assessment Instrument, and Hackman and Oldham’s Job Diagnostic Survey.8 In fact, so many questionnaires are available that rarely would an organization have to create a totally new one. However, because every organization has unique problems and special jargon for referring to them, almost any standardized instrument will need to have organization-specific additions, modifications, or omissions.
On the other hand, customized questionnaires are tailored to the needs of a particu- lar organization. Typically, they include questions composed by OD practitioners or organization members, receive limited use, and do not undergo longer-term develop- ment. They can be combined with standardized instruments to provide valid and reliable data focused toward the particular issues facing an organization.
Questionnaires, however, have a number of drawbacks that need to be taken into account in choosing whether to employ them for data collection. First, responses are limited to the questions asked in the instrument. They provide little opportunity to probe for additional data or to ask for points of clarification. Second, questionnaires tend to be impersonal, and
CHAPTER 6 COLLECTING, ANALYZING, AND FEEDING BACK DIAGNOSTIC INFORMATION 127
TABLE 6.2
Job-Design Questionnaire
Here are some statements about your job. How much do you agree or disagree with each?
My Job: Strongly Disagree Disagree
Slightly Disagree Undecided
Slightly Agree Agree
Strongly Agree
1. provides much variety … [1] [2] [3] [4] [5] [6] [7] 2. permits me to be left on my
own to do my own work … [1] [2] [3] [4] [5] [6] [7]
3. is arranged so that I often have the opportunity to see jobs or projects through to completion …
[1] [2] [3] [4] [5] [6] [7]
4. provides feedback on how well I am doing as I am working …
[1] [2] [3] [4] [5] [6] [7]
5. is relatively significant in our organization …
[1] [2] [3] [4] [5] [6] [7]
6. gives me considerable opportunity for independence and freedom in how I do my work …
[1] [2] [3] [4] [5] [6] [7]
7. gives me the opportunity to do a number of different things …
[1] [2] [3] [4] [5] [6] [7]
8. provides me an opportunity to find out how well I am doing …
[1] [2] [3] [4] [5] [6] [7]
9. is very significant or important in the broader scheme of things …
[1] [2] [3] [4] [5] [6] [7]
10. provides an opportunity for independent thought and action …
[1] [2] [3] [4] [5] [6] [7]
11. provides me with a great deal of variety at work …
[1] [2] [3] [4] [5] [6] [7]
12. is arranged so that I have the opportunity to complete the work I start …
[1] [2] [3] [4] [5] [6] [7]
13. provides me with the feeling that I know whether I am performing well or poorly …
[1] [2] [3] [4] [5] [6] [7]
14. is arranged so that I have the chance to do a job from the beginning to the end (i.e., a chance to do the whole job) …
[1] [2] [3] [4] [5] [6] [7]
15. is one where a lot of other people can be affected by how well the work gets done …
[1] [2] [3] [4] [5] [6] [7]
Scoring: Skill variety ......................................................................................................................... questions 1, 7, 11 Task identity ....................................................................................................................... questions 3, 12, 14 Task significance ............................................................................................................... questions 5, 9, 15 Autonomy .......................................................................................................................... questions 2, 6, 10 Feedback about results ..................................................................................................... questions 4, 8, 13
SOURCE: Reproduced by permission of E. Lawler, S. Mohrman, and T. Cummings, Center for Effective Organizations, University of Southern California.
128 PART 2 THE PROCESS OF ORGANIZATION DEVELOPMENT
employees may not be willing to provide honest answers. Third, questionnaires often elicit response biases, such as the tendency to answer questions in a socially acceptable manner. This makes it difficult to draw valid conclusions from employees’ self-reports.
6-2b Interviews A second important measurement technique is the individual or group interview. Inter- views are probably the most widely used technique for collecting data in OD. They permit the interviewer to ask the respondent direct questions. Further probing and clarification is, therefore, possible as the interview proceeds. This flexibility is invaluable for gaining pri- vate views and feelings about the organization and for exploring new issues that emerge during the interview.
Interviews may be highly structured—resembling questionnaires—or highly unstructured—starting with general questions that allow the respondent to lead the way. Structured interviews typically derive from a conceptual model of organization function- ing; the model guides the types of questions that are asked. For example, a structured interview based on the organization-level design components identified in Chapter 5 would ask managers specific questions about strategy, technology, organization structure, management processes, human resources systems, and organization culture.
Unstructured interviews are more general and include the following broad questions about organizational functioning:
• What are the major goals or objectives of the organization or department? • How does the organization currently perform with respect to these purposes? • What are the strengths and weaknesses of the organization or department? • What barriers stand in the way of good performance?
Although interviewing typically involves one-to-one interaction between an OD practitioner and an employee, it can be carried out in a group context. Group interviews save time and allow people to build on others’ responses. A major drawback, however, is that group settings may inhibit some people from responding freely.
A popular type of group interview is the focus group or sensing meeting.9 These are unstructured meetings conducted by a manager or a consultant. A small group of 10 to 15 employees is selected to represent a cross section of functional areas and hierarchical levels or a homogeneous grouping, such as minorities or engi- neers. Group discussion is frequently started by asking general questions about organizational features and functioning, an OD intervention’s progress, or current performance. Group members are then encouraged to discuss their answers more fully. Consequently, focus groups and sensing meetings are an economical way to obtain interview data and are especially effective in understanding particular issues in greater depth. The richness and validity of the information gathered will depend on the extent to which the manager or the OD practitioner develops a trusting relationship with the group and listens to member opinions.
Another popular unstructured group interview involves assessing the current state of an intact work group. The manager or the consultant generally directs a question to the group, calling its attention to some part of group functioning. For example, group members may be asked how they feel the group is progressing on its stated task. The group might respond and then come up with its own series of questions about barriers to task performance. This unstructured interview is a fast, simple way to collect data about group behavior. It enables members to discuss issues of immediate concern and to engage actively in the questioning and
CHAPTER 6 COLLECTING, ANALYZING, AND FEEDING BACK DIAGNOSTIC INFORMATION 129
answering process. This technique is limited, however, to relatively small groups and to settings where there is trust among employees and managers and a commit- ment to assessing group processes.
Interviews are an effective method for collecting data in OD. They are adaptive, allowing the interviewer to modify questions and to probe emergent issues during the interview process. They also permit the interviewer to develop an empathetic relationship with employees, frequently resulting in frank disclosure of pertinent information.
A major drawback of interviews is the amount of time required to conduct and ana- lyze them. Interviews can consume a great deal of time, especially if interviewers take full advantage of the opportunity to hear respondents out and change their questions accord- ingly. Personal biases also can distort the data. Like questionnaires, interviews are subject to the self-report biases of respondents and, perhaps more important, to the biases of the interviewer. For example, the nature of the questions and the interactions between the interviewer and the respondent may discourage or encourage certain kinds of responses. These problems suggest that interviewing takes considerable skill to gather valid data. Interviewers must be able to understand their own biases, to listen and establish empathy with respondents, and to change questions to pursue issues that develop during the course of the interview.
6-2c Observations One of the more direct ways of collecting data is simply to observe organizational beha- viors in their functional settings. The OD practitioner may do this by walking casually through a work area and looking around or by simply counting the occurrences of specific kinds of behaviors (e.g., the number of times a phone call is answered after three rings in a service department). Observation can range from complete participant observation, in which the OD practitioner becomes a member of the group under study, to more detached observation, in which the observer is clearly not part of the group or situation itself and may use film, videotape, and other methods to record behaviors.
Observations have a number of advantages. They are free of the biases inher- ent in self-report data. They put the OD practitioner directly in touch with the behaviors in question, without having to rely on others’ perceptions. Observations also involve real-time data, describing behavior occurring in the present rather than the past. This avoids the distortions that invariably arise when people are asked to recollect their behaviors. Finally, observations are adaptive in that the consultant can modify what he or she chooses to observe, depending on the circumstances.
Among the problems with observations are difficulties interpreting the meaning underlying the observations. OD practitioners may need to devise a coding scheme to make sense out of observations, and this can be expensive, take time, and introduce biases into the data. When the observer is the data collection instrument, the data can be biased and subjective unless the observer is trained and skilled in knowing what to look for; how, where, and when to observe; and how to record data systematically. Another problem concerns sampling: Observers not only must decide which people to observe, but they also must choose the time periods, territory, and events in which to
130 PART 2 THE PROCESS OF ORGANIZATION DEVELOPMENT
make those observations. Failure to attend to these sampling issues can result in highly biased samples of observational data.
When used correctly, observations provide insightful data about organization and group functioning, intervention success, and performance. For example, observations are particularly helpful in diagnosing the interpersonal relations of members of work groups. As discussed in Chapter 5, interpersonal relationships are a key component of work groups; observing member interactions in a group setting can provide direct infor- mation about the nature of those relationships.
6-2d Unobtrusive Measures Unobtrusive data are not collected directly from respondents but from secondary sources, such as company records and archives. These data are generally available in organizations and include records of absenteeism or tardiness; grievances; quan- tity and quality of production or service; financial performance; meeting minutes; and correspondence with key customers, suppliers, or governmental agencies.
Unobtrusive measures are especially helpful in diagnosing the organization, group, and individual outputs presented in Chapter 5. At the organization level, for example, market share and return on investment usually can be obtained from company reports. Similarly, organizations typically measure the quantity and qual- ity of the outputs of work groups and individual employees. Unobtrusive measures also can help to diagnose organization-level design components—structure, man- agement processes, and human resources systems. A company’s organization chart, for example, can provide useful information about organization structure. Information about management processes usually can be obtained by examining the firm’s management information system, operating procedures, and accounting practices. Data about human resources systems often are included in a company’s employee manual.
Unobtrusive measures provide a relatively objective view of organizational function- ing. They are free from respondent and consultant biases and are perceived as being “real” by many organization members. Moreover, unobtrusive measures tend to be quan- tified and reported at periodic intervals, permitting statistical analysis of behaviors occur- ring over time. Examining monthly absenteeism rates, for example, might reveal trends in employee withdrawal behavior.
The major problems with unobtrusive measures occur in collecting such infor- mation and drawing valid conclusions from it. Company records may not include data in a form that is usable by the OD practitioner. If, for example, individual performance data are needed, the consultant may find that many firms only record production information at the group or department level. Unobtrusive data also may have their own built-in biases. Changes in accounting procedures and in methods of recording data are common in organizations, and such changes can affect company records independently of what is actually happening in the organi- zation. For example, observed changes in productivity over time might be caused by modifications in methods of recording production rather than by actual changes in organizational functioning.
Despite these drawbacks, unobtrusive data serve as a valuable adjunct to other diagnostic measures, such as interviews and questionnaires. Archival data can be
CHAPTER 6 COLLECTING, ANALYZING, AND FEEDING BACK DIAGNOSTIC INFORMATION 131
used in preliminary diagnosis, identifying those organizational units with absentee- ism, grievance, or production problems. Then, interviews might be conducted or observations made in those units to discover the underlying causes of the problems. Conversely, unobtrusive data can be used to cross-check other forms of information. For example, if questionnaires reveal that employees in a department are dissatisfied with their jobs, company records might show whether that discon- tent is manifested in heightened withdrawal behaviors, in lowered quality work, or in similar counterproductive behaviors.
6-3 Sampling Before discussing how to analyze data, the issue of sampling needs to be emphasized. Application of the different data collection techniques invariably raises the following questions: “How many people should be interviewed and who should they be?” “What events should be observed and how many?” “How many records should be inspected and which ones?”10
Sampling is not an issue in many OD cases. Because OD practitioners collect inter- view or questionnaire data from all members of the organization or department in ques- tion, they do not have to worry about whether the information is representative of the organization or unit.
Sampling becomes an issue in OD, however, when data are collected from selected members, behaviors, or records. This is often the case when diagnosing organization- level issues or large systems. In these cases, it may be important to ensure that the sam- ple of people, behaviors, or records adequately represents the characteristics of the total population. For example, a sample of 50 employees might be used to assess the percep- tions of all 300 members of a department. A sample of production data might be used to evaluate the total production of a work group. OD practitioners often find that it is more economical and quicker to gather a sampling of diagnostic data than to collect all possi- ble information. If done correctly, the sample can provide useful and valid information about the entire organization or unit.
Sampling design involves considerable technical detail, and consultants may need to become familiar with basic references in this area or to obtain professional help.11 The first issue to address is sample size, or how many people, events, or records are needed to carry out the diagnosis or evaluation. This question has no simple answer: The necessary sample size is a function of population size, the con- fidence desired in the quality of the data, and the resources (money and time) available for data collection.
First, the larger the population (for example, the number of organization members or total number of work outcomes) or the more complex the client sys- tem (e.g., the number of salary levels that must be sampled or the number of dif- ferent functions), the more difficult it is to establish a “right” sample size. As the population increases in size and complexity, simple measures, such as an overall average score on a questionnaire item, are less meaningful. Because the population comprises such different types of people or events, more data are needed to ensure an accurate representation of the potentially different subgroups. Second, the larger the proportion of the population that is selected, the more confidence one
132 PART 2 THE PROCESS OF ORGANIZATION DEVELOPMENT
can have about the quality of the sample. If the diagnosis concerns an issue of great importance to the organization, then extreme confidence may be needed, indicative of a very large sample size. Third, limited resources constrain sample size. If resources are limited but the required confidence is high, then question- naires will be preferred over interviews because more information can be collected per member per dollar.
The second issue to address is sample selection. Probably the most common approach to sampling diagnostic data in OD is a simple random sample, in which each member, behavior, or record has an equal chance of being selected. For example, assume that an OD practitioner would like to select 50 people randomly out of the 300 employees at a manufacturing plant. Using a complete list of all 300 employees, the consultant can generate a random sample in one of two ways. The first method is to use a random number table printed in the back of almost any statistics text; the consultant would pick out the employees corresponding to the first 50 numbers under 300 beginning anywhere in the table. The second method is to pick every sixth name (300/50 6) starting anywhere in the list.
If the population is complex, or many subgroups need to be represented in the sam- ple, a stratified sample may be more appropriate than a random one. In a stratified sample, the population of members, events, or records is segregated into a number of mutually exclusive subpopulations and a random sample is taken from each subpopulation. For example, members of an organization might be divided into three groups (managers, white-collar workers, and blue-collar workers), and a random sample of members, beha- viors, or records could be selected from each grouping to reach diagnostic conclusions about each of the groups.
Adequate sampling is critical to gathering valid diagnostic data, and the OD litera- ture has paid little attention to this issue. OD practitioners should gain rudimentary knowledge in this area and use professional help if necessary.
6-4 Analyzing Data Data analysis techniques fall into two broad classes: qualitative and quantitative. Qualita- tive techniques generally are easier to use because they do not rely on numerical data. That fact also makes them more open to subjective biases but also easier to understand and interpret. Quantitative techniques, on the other hand, can provide more accurate readings of the organizational problem.
6-4a Qualitative Tools Of the several methods for summarizing diagnostic data in qualitative terms, two of the most important are content analysis and force-field analysis.
Content Analysis A popular technique for assessing qualitative data, especially interview data, is content analysis, which attempts to summarize comments into meaningful categories. When done well, a content analysis can reduce hundreds of interview comments into a few themes that effectively summarize the issues or attitudes of a group of respondents. The process of content analysis can be quite
CHAPTER 6 COLLECTING, ANALYZING, AND FEEDING BACK DIAGNOSTIC INFORMATION 133
formal, and specialized references describe this technique in detail.12 In general, however, the process can be broken down into three major steps. First, responses to a particular question are read to gain familiarity with the range of comments made and to determine whether some answers are occurring over and over again. Second, based on this sampling of comments, themes are generated that capture recurring comments. Themes consolidate different responses that say essentially the same thing. For example, in answering the question “What do you like most about your job?,” different respondents might list their coworkers, their supervi- sors, the new machinery, and a good supply of tools. The first two answers con- cern the social aspects of work, and the second two address the resources available for doing the work. Third, the respondents’ answers to a question are then placed into one of the categories. The categories with the most responses rep- resent those themes that are most often mentioned.
Force-Field Analysis A second method for analyzing qualitative data in OD derives from Kurt Lewin’s three-step model of change described in Chapter 2. Called force-field analysis, this method organizes information pertaining to organi- zational change into two major categories: forces for change and forces for main- taining the status quo or resisting change.13 Using data collected through interviews, observations, or unobtrusive measures, the first step in conducting a force-field analysis is to develop a list of all the forces promoting change and all those resisting it. Then, based either on the OD practitioner’s personal belief or per- haps on input from several organization members, the most powerful positive and negative forces are determined. One can either rank the order or rate the strength of the different forces.
Figure 6.2 illustrates a force-field analysis of the performance of a work group. The arrows represent the forces, and the length of the arrows corresponds to the strength of the forces. The information could have been collected in a group interview in which members were asked to list those factors maintaining the current level of group performance and those factors pushing for a higher level. Members also could have been asked to judge the strength of each force, with the average judgment shown by the length of the arrows.
This analysis reveals two strong forces pushing for higher performance: pressures from the supervisor of the group and competition from other work groups performing similar work. These forces for change are offset by two strong forces for maintaining the status quo: group norms supporting present levels of performance and well- learned skills that are resistant to change. According to Lewin, efforts to change to a higher level of group performance, shown by the darker band in Figure 6.2, should focus on reducing the forces maintaining the status quo. This might entail changing the group’s performance norms and helping members to learn new skills. The reduc- tion of forces maintaining the status quo is likely to result in organizational change with little of the tension or conflict typically accompanying change caused by increas- ing the forces for change.
Application 6.1 describes another installment in the change evaluation process at Alegent Health. (The introduction of this longitudinal case began in Chapter 4.) In this application, the research team collected data from interviews and questionnaires, but also used observation and unobtrusive measures. The analysis used a combination of
134 PART 2 THE PROCESS OF ORGANIZATION DEVELOPMENT
qualitative and quantitative techniques. What do you see as the strengths and weaknesses of the data collection and analysis process at Alegent?
6-4b Quantitative Tools Methods for analyzing quantitative data range from simple descriptive statistics of items or scales from standard instruments to more sophisticated, multivariate analysis of the underlying instrument properties and relationships among measured variables.14 The most common quantitative tools are means, standard deviations, and frequency distribu- tions; scattergrams and correlation coefficients; and difference tests. These measures are routinely produced by most statistical computer software packages. Therefore, mathe- matical calculations are not discussed here.
Means, Standard Deviations, and Frequency Distributions One of the most economical and straightforward ways to summarize quantitative data is to compute a mean and standard deviation for each item or variable measured. These represent the respondents’ average score and the spread or variability of the responses, respectively. These two numbers easily can be compared across different measures or subgroups. For example, Table 6.3 shows the means and standard deviations for six questions asked of 100 employees concerning the value of different kinds of organizational rewards. Based on the 5-point scale ranging from 1 (very low value) to 5 (very high value),
FIGURE 6.2
Force-Field Analysis of Work-Group Performance
© Ce
ng ag
e Le
ar ni
ng
CHAPTER 6 COLLECTING, ANALYZING, AND FEEDING BACK DIAGNOSTIC INFORMATION 135
a p
p lica
tio n
6 1
COLLECTING AND ANALYZING DIAGNOSTIC DATA AT ALEGENT HEALTH
T he two applications in Chapter 4 described the entering and contracting processes at the Alegent Health (AH) organization. As a result of a recent merger and the hiring of a
new CEO and chief innovation officer (CIO), the organization had implemented a series of large group interventions, known as decision accel- erators (DAs), to generate innovative strategies in the six clinical service areas of women’s and children’s services, oncology, behavioral health, neuroscience, orthopedics, and cardiology. Alegent Health then hired two OD researchers to evaluate its change progress. The evaluation was intended to help AH understand what had changed, what had been learned, the impact of those changes, and how they might extend those changes and learnings into the future. The diagnostic phase involved the collection and analysis of unobtrusive, interview, and survey data.
UNOBTRUSIVE MEASURES
Immediately following each DA, the Right Track office (a group set up to manage the DA experience) compiled a report listing partic- ipant names and affiliations, an agenda, instructions and elapsed times for each activity and process, photographs of different activities and all small-group outputs, and nearly verba- tim transcripts of the large-group report-outs, activity debriefings, and discussions.
These reports were analyzed to under- stand the process and outcomes associated with each DA. The researchers created a cod- ing scheme and process to capture the charac- teristics of the participants, the nature of the process, and a description of the DA outputs. Two coders analyzed the data to ensure the reliability of the analysis.
First, the results suggested that the DAs varied in their composition. For example, some DAs were composed of higher percentages of physicians or community members than other DAs. Second, some DAs were more “intense” than others as indicated by the amount of debate over decisions or issues, the number
of different stakeholders who participated in the debates and discussions, and the extent to which the DA’s activities deviated from the preset agenda. Finally, some DAs produced comprehensive visions and strategies for their clinical area, while others produced visions that were more narrowly focused.
INTERVIEW MEASURES
A second data set consisted of interviews with various stakeholder groups. Initial interviews were conducted with executives and physicians about (1) the context of change at Alegent, including organization history, strategy, and recent changes; (2) their reflections on the DA process; and (3) clinical area implementation progress. The researchers conducted a second round of interviews with people who were closely connected with the implementation of each clinical service-area strategy. They were asked questions about the clarity of action plans, the level of involvement of different peo- ple, and implementation progress. Finally, a third set of interviews were conducted with a sample of staff nurses who had not participated in the original DAs or been directly involved in implementation activities, such as steering committees or design teams.
Each set of interview data was content analyzed for key themes and perspectives. A few of the summary results from the initial interviews are presented here.
When asked, “How clear were the action plans coming out of the DA?,” the executives were evenly split in their beliefs that the action plans were clear as opposed to the plans being essentially absent. Executives were also asked, “What is going well/not so well in implementa- tion of the different service line strategies?” About 20% of executives believed that the strategies were aligned with the mission/vision of the health system and that the DAs had pro- vided a clear vision to guide change. However, more than half of executives expressed concern that the organization lacked a real change capa- bility. Executives were also concerned about
136 PART 2 THE PROCESS OF ORGANIZATION DEVELOPMENT
the data suggest that challenging work and respect from peers are the two most highly valued rewards. Monetary rewards, such as pay and fringe benefits, are not as highly valued.
However, the mean can be a misleading statistic. It only describes the average value and thus provides no information on the distribution of the responses.
being overwhelmed by change, insufficient commu- nication, and the need to involve stakeholders more.
When asked, “What would you list as the ‘high points’ or ‘best success stories’ of the DA process?” and “What have been some of the least successful activities/concerns?,” the answers were more positive than negative. Nearly all of the interviewees noted the improved relationships with physicians, and more than a third of executives said there had been some good learning on how to increase the speed of decision making. Both of these results reflected cultural changes in the orga- nization that were among the purposes for conduct- ing the DAs. On the negative side, a small percentage of executives noted the continued diffi- culties associated with coordinating the operations of a multihospital system.
Another area of interview data concerned executive perceptions of how the DA might evolve in the future. There was a strong belief that the DA needed to evolve to fit the changed organizational conditions and a widespread perception that this should include a more explicit focus on execution, better change governance, and better follow-up and communication.
In addition to these initial interview results, data from the second round of implementation interviews were used to develop six cases studies, one for each clinical service area. They described the initial DA event and the subsequent decisions, activities, and events for the 18 months following the forma- tion of the clinical strategies. Importantly, the case studies listed the organizational changes that most people agreed had been implemented in the first 18 months. Each case study was given to the VP in charge of the clinical area for validation.
SURVEY MEASURES
The researchers also collected two sets of survey data. The first survey, administered during the
initial round of executive and physician interviews, asked them to rate several dimensions of clinical area strategy and progress. The second survey was administered to people who attended a “review DA” for three of the six clinical areas. It too measured perceptions of clinical strategy and progress.
The survey data were organized into three categories and analyzed by a statistical program. The first category measured five dimensions of strategy for each clinical area: comprehensiveness, innovativeness, aggressiveness, congruence with Alegent’s strategy, and business focus. Both execu- tives and managers rated the clinical strategies highest on comprehensiveness and lowest on congruence with Alegent’s mission. Executives also rated the strategies lower on innovativeness. In all dimensions and for each clinical area, managers rated the five dimensions higher than executives did.
The second category measured how well the implementation process was being managed. Executives “somewhat agreed” that the clinical area strategies were associated with a clear action plan; however, there was considerable variance, suggesting that some clinical areas had better action plans than others. Similarly, managers “somewhat agreed” that change governance sys- tems exist and that change was coordinated.
The third category assessed implementation success. As with the strategy dimensions, man- agers rated overall implementation progress higher than executives did, but both groups were some- what guarded (between neutral and agree) in their responses. Managers were asked a more detailed set of questions about implementation. There was more agreement that the clinical strategies were the “right thing to do” and had helped to “build social capital” in the organization, but they were neutral with respect to whether “people feel involved” in the change.
CHAPTER 6 COLLECTING, ANALYZING, AND FEEDING BACK DIAGNOSTIC INFORMATION 137
Different patterns of responses can produce the same mean score. Therefore, it is important to use the standard deviation along with the frequency distribution to gain a clearer understanding of the data. The frequency distribution is a graphical method for displaying data that shows the number of times a particular response was given. For example, the data in Table 6.3 suggest that both pay and praise from the supervisor are equally valued with a mean of 4.0. However, the standard deviations for these two measures are very different at 0.71 and 1.55, respectively. Table 6.4 shows the frequency distributions of the responses to the questions about pay and praise from the supervisor. Employees’ responses to the value of pay are distributed toward the higher end of the scale, with no one rating it of low or very low value. In contrast, responses about the value of praise from the supervisor fall into two distinct groupings: Twenty-five employees felt that supervisor praise has a low or very low value, whereas 75 people rated it high or very high. Although both rewards have the same mean value, their standard deviations and frequency distri- butions suggest different interpretations of the data.
In general, when the standard deviation for a set of data is high, there is consider- able disagreement over the issue posed by the question. If the standard deviation is small, the data are similar on a particular measure. In the example described above, there is disagreement over the value of supervisory praise (some people think it is important, but others do not), but there is fairly good agreement that pay is a reward with high value.
Scattergrams and Correlation Coefficients In addition to describing data, quanti- tative techniques also permit OD practitioners to make inferences about the relationships between variables. Scattergrams and correlation coefficients are measures of the strength of a relationship between two variables. For example, suppose the problem being faced by an organization is increased conflict between the manufacturing department and the engineering design department. During the data collection phase, information about the number of conflicts and change orders per month over the past year is collected. The data are shown in Table 6.5 and plotted in a scattergram in Figure 6.3.
TABLE 6.3
Descriptive Statistics of Value of Organizational Rewards
Organizational Rewards Mean Standard Deviation
Challenging work 4.6 0.76
Respect from peers 4.4 0.81
Pay 4.0 0.71
Praise from supervisor 4.0 1.55
Promotion 3.3 0.95
Fringe benefits 2.7 1.14
Number of respondents 100 1 very low value; 5 very high value
© Ce
ng ag
e Le
ar ni
ng
138 PART 2 THE PROCESS OF ORGANIZATION DEVELOPMENT
A scattergram is a diagram that visually displays the relationship between two vari- ables. It is constructed by locating each case (person or event) at the intersection of its value for each of the two variables being compared. For example, in the month of August, there were eight change orders and three conflicts, whose intersection is shown in Figure 6.3 as an .
Three basic patterns can emerge from a scattergram, as shown in Figure 6.4. The first pattern is called a positive relationship because as the values of x increase, so do the values of y. The second pattern is called a negative relationship because as the values of x increase, the values of y decrease. Finally, there is the “shotgun” pattern wherein no relationship between the two variables is apparent. In the example shown in Figure 6.3, an apparently strong positive relationship exists between the number of change orders and the number of conflicts between the engineering design department and the manufacturing department. This suggests that change orders may contribute to the observed conflict between the two departments.
The correlation coefficient is simply a number that summarizes data in a scatter- gram. Its value ranges between 1.0 and 1.0. A correlation coefficient of 1.0 means that there is a perfectly positive relationship between two variables, whereas a correlation
TABLE 6.4
Frequency Distributions of Responses to “Pay” and “Praise from Supervisor” Items
Pay (Mean 4.0)
Response Number Checking
Each Response Graph*
(1) Very low value 0
(2) Low value 0
(3) Moderate value 25 XXXXX
(4) High value 50 XXXXXXXXXX
(5) Very high value 25 XXXXX
Praise from Supervisor (Mean 4.0)
Response Number Checking
Each Response Graph*
(1) Very low value 15 XXX
(2) Low value 10 XX
(3) Moderate value 0
(4) High value 10 XX
(5) Very high value 65 XXXXXXXXXXXX
*Each X five people checking the response
© Ce
ng ag
e Le
ar ni
ng
CHAPTER 6 COLLECTING, ANALYZING, AND FEEDING BACK DIAGNOSTIC INFORMATION 139
of 1.0 signifies a perfectly negative relationship. A correlation of 0 implies a “shotgun” scattergram where there is no relationship between two variables.
Difference Tests The final technique for analyzing quantitative data is the differ- ence test. It can be used to compare a sample group against some standard or norm to determine whether the group is above or below that standard. It also can be used to determine whether two samples are significantly different from each other. In the first case, such comparisons provide a broader context for understanding the mean- ing of diagnostic data. They serve as a “basis for determining ‘how good is good or how bad is bad.’ ”15 Many standardized questionnaires have standardized scores based on the responses of large groups of people. It is critical, however, to choose a comparison group that is similar to the organization being diagnosed. For example, if 100 engineers take a standardized attitude survey, it makes little sense to compare their scores against standard scores representing married males from across the country. On the other hand, if industry-specific data are available, a comparison of sales per employee (as a measure of productivity) against the industry average would be valid and useful.
The second use of difference tests involves assessing whether two or more groups differ from one another on a particular variable, such as job satisfaction or absenteeism. For example, job satisfaction differences between an accounting department and a sales department can be determined with this tool. Given that each group took the same questionnaire, their means and standard deviations can be used to compute a difference score (t-score or z-score) indicating whether the two groups are statistically different.
TABLE 6.5
Relationship between Change Orders and Conflicts
Month Number of
Change Orders Number of Conflicts
April 5 2
May 12 4
June 14 3
July 6 2
August 8 3
September 20 5
October 10 2
November 2 1
December 15 4
January 8 3
February 18 4
March 10 5
© Ce
ng ag
e Le
ar ni
ng 20
15
140 PART 2 THE PROCESS OF ORGANIZATION DEVELOPMENT
The larger the difference score relative to the sample size and standard deviation for each group, the more likely that one group is more satisfied than the other.
Difference tests also can be used to determine whether a group has changed its score on job satisfaction or some other variable over time. The same questionnaire can be given to the same group at two points in time. Based on the group’s means and standard
FIGURE 6.3
Scattergram of Change Order versus Conflict
FIGURE 6.4
Basic Scattergram Patterns
© Ce
ng ag
e Le
ar ni
ng ©
Ce ng
ag e
Le ar
ni ng
CHAPTER 6 COLLECTING, ANALYZING, AND FEEDING BACK DIAGNOSTIC INFORMATION 141
deviations at each point in time, a difference score can be calculated. The larger the score, the more likely the group actually changed its job satisfaction level.
The calculation of difference scores can be very helpful for diagnosis but requires the OD practitioner to make certain assumptions about how the data were collected. These assumptions are discussed in most standard statistical texts, and OD practitioners should consult them before calculating difference scores for purposes of diagnosis or evaluation.16
6-5 Feeding Back Data Perhaps the most important step in the diagnostic process is feeding back diagnostic infor- mation to the client organization. Although the data may have been collected with the client’s help, the OD practitioner often organizes and presents them to the client. Properly analyzed and meaningful data can have an impact on organizational change only if organization members can use the information to devise appropriate action plans. A key objective of the feedback process is to be sure that the client has ownership of the data.
As shown in Figure 6.5, the success of data feedback depends largely on its ability to arouse organizational action and to direct energy toward problem solving. Whether feed- back helps to energize the organization depends on the content of the feedback data and on the process by which they are fed back to organization members.
6-5a Content of Feedback In the course of diagnosing the organization, a large amount of data is collected—often, more information than the client needs or can interpret in a realistic period of time. If too many data are fed back, the client may decide that changing is impossible. Therefore, OD practitioners need to summarize the data in ways that enable clients to understand the information and draw action implications from it. The techniques for data analysis described earlier in this chapter can inform this task. Additional criteria for determining the content of diagnostic feedback are described below.
Several characteristics of effective feedback data have been described in the litera- ture.17 They include the following nine properties:
1. Relevant. Organization members are likely to use feedback data for problem solving when they find the information meaningful. Including managers and employees in the initial data collection activities can increase the relevance of the data.
2. Understandable. Data must be presented to organization members in a form that is readily interpreted. Statistical data, for example, can be made understandable through the use of graphs and charts.
3. Descriptive. Feedback data need to be linked to real organizational behaviors if they are to arouse and direct energy. The use of examples and detailed illustrations can help employees gain a better feel for the data.
4. Verifiable. Feedback data should be valid and accurate if they are to guide action. Thus, the information should allow organization members to verify whether the findings really describe the organization. For example, questionnaire data might include information about the sample of respondents as well as frequency distribu- tions for each item or measure. Such information can help members verify whether the feedback data accurately represent organizational events or attitudes.
5. Timely. Data should be fed back to members as quickly as possible after being col- lected and analyzed. This will help ensure that the information is still valid and is linked to members’ motivations to examine it.
142 PART 2 THE PROCESS OF ORGANIZATION DEVELOPMENT
6. Limited. Because people can easily become overloaded with too much information, feedback data should be limited to what employees can realistically process at one time.
7. Significant. Feedback should be limited to those problems that organization mem- bers can do something about because it will energize them and help direct their efforts toward realistic changes.
8. Comparative. Feedback data can be ambiguous without some benchmark as a refer- ence. Whenever possible, data from comparative groups should be provided to give organization members a better idea of how their group fits into a broader context.
9. Unfinalized. Feedback is primarily a stimulus for action and thus should spur fur- ther diagnosis and problem solving. Members should be encouraged, for example, to use the data as a starting point for more in-depth discussion of organizational issues.
FIGURE 6.5
Possible Effects of Feedback
© Ce
ng ag
e Le
ar ni
ng 20
15
CHAPTER 6 COLLECTING, ANALYZING, AND FEEDING BACK DIAGNOSTIC INFORMATION 143
6-5b Process of Feedback In addition to providing effective feedback data, it is equally important to attend to the process by which that information is fed back to people. Typically, data are provided to organization members in a meeting or series of meetings. Feedback meetings provide a forum for discussing the data, drawing relevant conclusions, and devising preliminary action plans. Because the data might include sensitive material and evaluations about organization members’ behaviors, people may come to the meeting with considerable anxiety and fear about receiving the feedback. This anxiety can result in defensive beha- viors aimed at denying the information or providing rationales. More positively, people can be stimulated by the feedback and the hope that desired changes will result from the feedback meeting. Because people are likely to come to feedback meetings with anxiety, fear, and hope, OD practitioners need to manage the feedback process so that construc- tive discussion and problem solving occur. The most important objective of the feedback process is to ensure that organization members own the data. Ownership is the opposite of resistance to change and refers to people’s willingness to take responsibility for the data, their meaning, and the consequences of using them to devise a change strategy.18
If the feedback session results in organization members rejecting the data as invalid or useless, then the motivation to change is lost and members will have difficulty engaging in a meaningful process of change.
Ownership of the feedback data is facilitated by the following five features of successful feedback processes:19
1. Motivation to work with the data. Organization members need to feel that working with the feedback data will have beneficial outcomes. This may require explicit sanc- tion and support from powerful groups so that people feel free to raise issues and to identify concerns during the feedback sessions. If members have little motivation to work with the data or feel that there is little chance to use the data for change, then the information will not be owned by the client system.
2. Structure for the meeting. Feedback meetings need some structure or they may degenerate into chaos or aimless discussion. An agenda or outline for the meeting and the presence of a discussion leader can usually provide the necessary direction. If the meeting is not kept on track, especially when the data are negative, ownership can be lost in conversations that become too general. When this happens, the energy gained from dealing directly with the problem is lost.
3. Appropriate attendance. Generally, organization members who have common pro- blems and can benefit from working together should be included in the feedback meeting. This may involve a fully intact work team or groups comprising members from different functional areas or hierarchical levels. Without proper representation in the meeting, ownership of the data is lost because participants cannot address the problem(s) suggested by the feedback.
4. Appropriate power. It is important to clarify the power possessed by the group receiving the feedback data. Members need to know on which issues they can make necessary changes, on which they can only recommend changes, and over which they have no control. Unless there are clear boundaries, members are likely to have some hesitation about using the feedback data for generating action plans. Moreover, if the group has no power to make changes, the feedback meeting will become an empty exercise rather than a real problem-solving session. Without the power to address change, there will be little ownership of the data.
5. Process help. People in feedback meetings require assistance in working together as a group. When the data are negative, there is a natural tendency to resist the
144 PART 2 THE PROCESS OF ORGANIZATION DEVELOPMENT
implications, deflect the conversation onto safer subjects, and the like. An OD prac- titioner with group process skills can help members stay focused on the subject and improve feedback discussion, problem solving, and ownership.
When combined with effective feedback data, these features of successful feedback meet- ings enhance member ownership of the data. They help to ensure that organization members fully discuss the implications of the diagnostic information and that their con- clusions are directed toward relevant and feasible organizational changes.
Application 6.2 presents excerpts from some training materials that were delivered to a group of internal OD facilitators at a Fortune 100 telecommunications company.20
It describes how the facilitators were trained to deliver the results of a survey concerning problem solving, team functioning, and perceived effectiveness.
6-6 Survey Feedback Survey feedback is a process of collecting and feeding back data from an organization or department through the use of a questionnaire or survey. The data are analyzed, fed back to organization members, and used by them to diagnose the organization and to develop interventions to improve it. Because questionnaires often are used in organization diag- nosis, particularly in OD efforts involving large numbers of participants, and because it is a powerful intervention in its own right, survey feedback is discussed here as a special case of data feedback.
As discussed in Chapter 1, survey feedback is a major technique in the history and development of OD. Originally, this intervention included only data from questionnaires about members’ attitudes. However, attitudinal data can be supplemented with interview data and more objective measures, such as productivity, turnover, and absenteeism.21
Another trend has been to combine survey feedback with other OD interventions, including work design, structural change, large group interventions, and intergroup rela- tions. These change methods are the outcome of the planning and implementation phase following from survey feedback and are described fully in Chapters 10–20.
6-6a What Are the Steps? Survey feedback generally involves the following five steps:22
1. Members of the organization, including those at the top, are involved in preliminary planning of the survey. In this step, all parties must be clear about the level of anal- ysis (organization, group, or job) and the objectives of the survey. Because most sur- veys derive from a model about organization or group functioning, organization members must, in effect, approve that diagnostic framework. This is an important initial step in gaining ownership of the data and in ensuring that the right problems and issues are addressed by the survey.
Once the objectives are determined, the organization can use one of the stan- dardized questionnaires described earlier in this chapter, or it can develop its own survey instrument. If the survey is developed internally, pretesting the questionnaire is essential to ensure that it has been constructed properly. In either case, the survey items need to reflect the objectives established for the survey and the diagnostic issues being addressed.
2. The survey instrument is administered to all members of the organization or work group. This breadth of data collection is ideal, but it may be appropriate to administer
CHAPTER 6 COLLECTING, ANALYZING, AND FEEDING BACK DIAGNOSTIC INFORMATION 145
a p
p lica
tio n
6 2
TRAINING OD PRACTITIONERS IN DATA FEEDBACK
A s part of a large-scale, employee involve- ment (EI) program, a large telecommunica- tions company and the Communications Workers of America union were working
to build an internal organization development consulting capability. This involved the hiring and training of several union and management employees to work with managers, facilitate EI problem-solving team meetings, and assist in the implementation of recommended changes. The implementation process included an eval- uation component and the EI facilitators were expected to collect and feed back data to the organization.
The data collected included observation of various work processes and problem-solving meetings; unobtrusive measures such as minutes from all meetings, quarterly income statements, operational reports, and communi- cations; and questionnaire and interview data. A three-page questionnaire was administered every three months and it asked participants on EI problem-solving teams for their percep- tions of team functioning and performance. Internal EI facilitators were appointed from both management and union employees, and part of their work required them to feed back the results of the quarterly surveys.
To provide timely feedback to the problem- solving teams, the EI facilitators were trained to deliver survey feedback. Some of the mate- rial developed for that training is summarized below.
I. PLANNING FOR A SURVEY-FEEDBACK SESSION
The success of a survey-feedback meeting often has more to do with the level of preparation for the meeting than with anything else. There are several things to do in preparing for a survey-feedback meeting. A. Distribute copies of the feedback report
in advance. This enables people to devote more time at the meeting to problem solving and less to just digest- ing the data. This is especially important
when a large quantity of data is being presented.
B. Think about substantive issues in advance. Formulate your own view of what the data suggest about the strengths and weaknesses of the group. Does the general picture appear to be positive or problematic? Do the data fit the experience of the group as you know it? What issues do the data suggest need group attention? Is the group likely to avoid any of these issues? If so, how will you help the group confront the difficult issues?
C. Make sure you can answer likely tech- nical questions about the data. Survey data have particular strengths and weaknesses. Be able to acknowledge that the data are not perfect, but that a lot of effort has gone into ensuring that they are reliable and valid.
D. Plan your introduction to the survey- feedback portion of the meeting. Make the introduction brief and to the point. Remind the group of why it is considering the data, set the stage for problem solving by pointing out that many groups find such data helpful in tracking their progress, and be prepared to run through an example that shows how to understand the feedback data.
II. PROBLEM SOLVING WITH SURVEY- FEEDBACK DATA A. Chunk the feedback. If a lot of data are
being fed back, use your knowledge of the group and the data to present small portions of data. Stop periodically to see if there are questions or comments about each section or “chunk” of data.
B. Stimulate discussion on the data. What follows are various ways to help get the discussion going. 1. Help clarify the meaning of the data
by asking • What questions do you have
about what the data mean?
146 PART 2 THE PROCESS OF ORGANIZATION DEVELOPMENT
• What does [a specific number] mean?
• Does anything in the data surprise you?
• What do the data tell you about how we’re doing as a group?
2. Help develop a shared diagnosis about the meaning of the data by commenting
• What I hear people saying is… Does everyone agree with that?
• Several people are saying that… is a problem. Do we agree that this is something the group needs to address?
• Some people seem to be saying… while other comments suggest… Can you help me understand how the group sees this?
• The group has really been strug- gling with [specific issue that the facilitator is familiar with], but the data say that we are strong on this. Can someone explain this?
3. Help generate action alternatives by asking
• What are some of the things we can do to resolve… ?
• Do we want to brainstorm some action steps to deal with… ?
C. Focus the group on its own data. The major benefit of survey feedback for EI teams will be in learning about the group’s own behav- ior and outcomes. Often, however, groups will avoid dealing with issues concerning their own group in favor of broader and less helpful discussions about what other groups are doing right and wrong. Com- ments you might use to help get the group on track include: 1. What do the data say about how we
are doing as a group? 2. There isn’t a lot we can do about what
other groups are doing. What can we do about the things that are under our control?
3. The problem you are mentioning sounds like one this group also is fac- ing [explain]. Is that so?
D. Be prepared for problem-solving discus- sions that are only loosely connected to the data. It is more important for the group to use the data to understand itself better and to solve problems than it is to follow any particular steps in analyzing the data. Groups often are not very systematic in how they analyze survey-feedback data. They may ignore issues that seem obvious to them and instead focus on one or two issues that have meaning for them.
E. Hot issues and how to deal with them. Sur- vey data can be particularly helpful in addressing some hot issues within the group that might otherwise be overlooked. For example, a group often will prefer to portray itself as very effective even though group members privately acknowledge that such is not the case. If the data show pro- blems that are not being addressed, you can raise this issue as a point for discus- sion. If someone denies that group mem- bers feel there is a problem, you can point out that the data come from the group and that group members reported such-and- such on the survey. Be careful not to use a parental tone; if you sound like you’re wagging your finger at or lecturing the group, you’re likely to get a negative reac- tion. Use the data to raise issues for discus- sion in a less emotional way.
Ultimately, the group must take responsibility for its own use of the data. There will be times when the OD practitioner sees the issues differ- ently from the way group members see them or times when it appears certain to the practitioner that the group has a serious problem that it refuses to acknowledge. A facilitator cannot push a group to do something it’s not ready to do, but he or she can poke the group at times to find out if it is ready to deal with tough issues. “A little irritation is what makes a pearl in the oyster.”
CHAPTER 6 COLLECTING, ANALYZING, AND FEEDING BACK DIAGNOSTIC INFORMATION 147
the instrument to only a sample of members because of cost or time constraints. If so, the size of the sample should be as large as possible to improve the motivational basis for participation in the feedback sessions.
3. The OD practitioner usually analyzes the survey data, tabulates the results, suggests approaches to diagnosis, and trains client members to lead the feedback process.
4. Data feedback usually begins at the top of the organization and cascades downward to groups reporting to managers at successively lower levels. This waterfall approach ensures that all groups at all organizational levels involved in the survey receive appropriate feedback. Most often, members of each organization group at each level discuss and deal with only that portion of the data involving their particular group. They, in turn, prepare to introduce data to groups at the next lower organi- zational level if appropriate.
Data feedback also can occur in a “bottom-up” approach. Initially, the data for specific work groups or departments are fed back and action items proposed. At this point, the group addresses problems and issues within its control. The group notes any issues that are beyond its authority and suggests actions. That information is combined with information from groups reporting to the same manager, and the combined data are fed back to the managers who review the data and the recom- mended actions. Problems that can be solved at this level are addressed. In turn, their analyses and suggestions regarding problems of a broader nature are combined, and feedback and action sessions proceed up the hierarchy. In such a way, the peo- ple who most likely will carry out recommended action get the first chance to propose suggestions.
5. Feedback meetings provide an opportunity to work with the data. At each meeting, members discuss and interpret their data, diagnose problem areas, and develop action plans. OD practitioners can play an important role during these meetings,23
facilitating group discussion to produce accurate understanding, focusing the group on its strengths and weaknesses, and helping to develop effective action plans.
Although the preceding steps can have a number of variations, they generally reflect the most common survey-feedback design. Application 6.3 presents a contemporary example of how the survey-feedback methodology can be adapted to serve strategic pur- poses. The application describes how Cambia Health Solutions used a survey and survey feedback process to initiate a strategic change effort.
6-6b Survey Feedback and Organizational Dependencies Traditionally, the steps of survey feedback have been applied to work groups and organi- zational units with little attention to dependencies among them. Research suggests, how- ever, that the design of survey feedback should vary depending on how closely the participating units are linked with one another.24 When the units are relatively indepen- dent and have little need to interact, survey feedback can focus on the dynamics occur- ring within each group and can be applied to the groups separately. When there is greater dependency among units and they need to coordinate their efforts, survey feed- back must take into account relationships among the units, paying particular attention to the possibility of intergroup conflict. In these situations, the survey-feedback process needs to be coordinated across the interdependent groups. The process will typically be managed by special committees and task forces representing the groups. They will facili- tate the intergroup confrontation and conflict resolution generally needed when relations across groups are diagnosed.
148 PART 2 THE PROCESS OF ORGANIZATION DEVELOPMENT
a p
p li
ca ti
o n
6 3 SURVEY FEEDBACK AND PLANNED CHANGEAT CAMBIA HEALTH SOLUTIONS
C ambia Health Solutions (www.cambia health.com) is a nonprofit total health solu- tions company dedicated to transforming the way people experience the health care
system. Located in the Pacific Northwest and intermountain region of the United States, Cam- bia’s portfolio of companies spans health care information technology and software develop- ment; retail health care; health insurance plans; pharmacy benefit management; life, dis- ability, dental, vision, and other lines of protec- tion; alternative solutions to health care access; and freestanding health and wellness solutions. The largest business in the portfolio is Regence Health, a health insurance plan associated with the Blue Cross and Blue Shield brands. Regence Health is over 90 years old and operates in Washington, Oregon, Idaho, and Utah.
To support this increasingly broad portfolio, Cambia had restructured itself into two divi- sions: Regence Insurance Holding Company and Direct Health Solutions. All of the start-up, alternative health care products and services were housed in the direct health solutions divi- sion. In 2009, the organization was concerned about the health care reform initiatives taking place in Washington, D.C. and more specifically the implications of the recently passed “Obamacare” legislation. What were the impli- cations of establishing regional health exchanges and accountable care organizations? How would the organization have to change? In particular, was the organization’s culture “fit for the future?”
As corporate sponsors of USC’s Center for Effective Organizations, the vice president of human resources and the director of organiza- tion development called the Center to talk about the latest thinking in organization culture and how they might go about managing cultural change. After several conversations about dif- ferent approaches and the research being done at the Center regarding organization design, change, and agility, the researchers pro- posed an assessment process of Cambia’s cur- rent organization in terms of how people saw
the strategies, structures, systems, and culture. A design team composed of the executive vice president of corporate services, the VP of HR, the director of OD, and an internal HR business partner worked with the researcher to make the assessment relevant.
In early 2011, a three-page diagnostic sur- vey was administered to all managers with titles of assistant director or above, a popula- tion of about 150 people. In addition, 16 senior leaders were interviewed from the headquar- ters and regional organizations. The leaders represented a good mix of functions and ten- ure with the organization.
The survey consisted of about 50 items to be rated on a scale of 1 to 5 where 1 “Not at all” and a 5 “To a great extent.” These pre- tested items fell into 14 dimensions, including the extent to which the organization formu- lated “robust strategies,” engaged in future focused environmental scanning, had flat and responsive structures, rewarded performance and change, leveraged information systems, developed its talent well, and managed resources flexibly. In addition, the survey asked several questions about the organiza- tion’s cultural values and how members perceived leaders spending their time. The hour-long interviews asked questions addres- sing similar issues in terms of strategies, pro- cesses, and culture but were focused more on gathering rich stories and examples that might help the survey data “come alive.”
The results of the survey were placed into a spreadsheet and analyzed with statistical pro- grams that generated summary tables and charts of the data. The interview data was summarized using content analysis procedures and preliminary themes were discussed with design team members to ensure that the inter- view responses and categories had meaning for the organization.
The summary results were then placed into three categories: “Positive issues,” “Areas of Concern,” and “Problems.” Compared to the overall scores from other firms, Cambia’s scores
CHAPTER 6 COLLECTING, ANALYZING, AND FEEDING BACK DIAGNOSTIC INFORMATION 149
were generally below the overall average of other firms but were similar to other financial services firms. The economic recession and financial crises of the time had affected the culture of many of these firms and it was not surprising that the finan- cial services sector scores were lower.
The key “positive issue” was that people reported a strong sense of shared purpose in the organization. Captured by “The Cause,” a statement announced in early 2004 stating that the Cambia organization wanted to be a “catalyst for change” in health care, there was broad support for this clear direction. The Cause and the organization’s history also supported a clear “member-focused” culture. People liked working for a not-for-profit insurer and believed that such a corporate form was an important differentiator in the way the organization did busi- ness. This belief was reflected in the survey data as a very healthy balance between driving for results and taking care of people.
In the areas of concern category, and despite the strong shared sense of purpose scores, people struggled with what The Cause meant to their day- to-day behaviors. It was one thing to be clear about “being a catalyst for change” in health care, but how did that translate into how organization mem- bers were supposed to treat customers? In this sense, people were concerned about “who we are” as an organization and did not see how the Cause helped them have a real “voice” in making day-to-day decisions.
The recent reorganization into a health care business and a set of entrepreneurial “start-up” businesses that were intended to explore the future of the health care industry clearly reflected The Cause. However, people were concerned about what it meant for the culture. A lot of senior management’s attention was focused on the inno- vative nature of these new businesses, and some people in the insurance division felt left out. The culture of Cambia was clearly changing, but was the Regence culture expected to change as well and if so in what direction? The Cause helped peo- ple understand where the organization was headed, but it didn’t really help people answer the question “who are we?” and how to make decisions. People were frustrated by this.
In general, people were also concerned about how well the new direction was being supported by different organizational systems. They believed
that recent structural and reward systems changes were heading in the right direction, but other com- ments raised questions over other features, such as the way organizational and individual goals were set, how the organization responded to opportu- nities, and the way information and communication moved throughout the organization. These sys- tems were not changing and did not necessarily align with the new direction. The IT systems, in particular, had a very bad reputation. A complex systems changeover was generally regarded as an example of poor execution, and was producing a number of headaches around the organization.
Finally, two big problems loomed. First, there was widespread agreement that the organization did not have the change and learning capabilities to execute a change of this magnitude. As a 90-year-old organization in a slow-moving and reg- ulated industry, there was little expertise in the organization regarding how to manage change. Second, in a related way, the organization was rely- ing on innovation in both the new start-up busi- nesses and the traditional health care business as part of The Cause. However, the organization lacked the resources, processes, and experience to generate new product/service ideas or identify and implement process improvements. The pro- cesses that had helped them to adapt in the past were unlikely to be effective in the future.
The summary data were fed back in multiple forums. The first forum was an all-day meeting of the design team. A PowerPoint deck provided both detailed summaries and analyses of the data as well as charts that made interpretation more intui- tive. For example, the 14 scales regarding strategy, structure, and processes were presented as bar charts that allowed the design team to “see” how their data compared to other organizations and overall averages.
The data presentation was broken up by catego- ries. First, the “good news” was presented and dis- cussed. The organization had important strengths that any change process would naturally want to leverage or reinforce. The strong sense of shared pur- pose in the organization would provide an important base. The discussion among design team members centered on the acknowledgement that a strong his- tory in the different regions had created a “members first” culture. While it was acknowledged as a strength, the design team also wondered whether
150 PART 2 THE PROCESS OF ORGANIZATION DEVELOPMENT
such a legacy orientation would be strength or a weakness if change was necessary.
The areas of concern and problems were pre- sented next. The group spent quite a bit of time discussing their implications. There was ready agreement on the problems. Design team mem- bers believed that the organization needed (and lacked) change, learning, and innovation capabili- ties. But they also believed that just building these capabilities was not enough and might be a waste of time. They needed to be focused on changing and innovating the right things.
Much of the conversation then centered around the implication that there was a distinction to be made between the clear direction provided by The Cause and the concern that there was no guidance for decision making. The interviews clearly pointed to a frustration about how hard it was to “get things done” in the organization. There was a perception that too many decisions were pushed up to the top for resolution and that silos in the organization pre- vented the required cross-group collaboration.
From here, the diagnostic conversation turned to a broader subject. The design team members were concerned that not being able to “get stuff done” and pushing decisions up the hierarchy was indicative of a more basic problem. People gener- ally did not have clear goals (“it’s hard to get stuff done when you don’t know where you are going”) and were not held accountable (“it’s not my deci- sion”). The culture of “Northwest Nice” was work- ing against such culture change objectives. The design team believed that if change and innovation capability building could be focused on helping the organization more effectively execute specific strategies and goals, then that would represent an important impetus for culture change.
Before the meeting ended, the design team believed it was important to share the data and their conclusions with the CEO to gauge his level of interest in moving a change process forward. The team spent a considerable amount of time sort- ing through the data to find the most central and most influential data points to tell a story. The CEO’s summary was only two pages long and con- sisted of the high-level summary of positives, con- cerns, and negatives as well as a summary of the survey scale scores compared to other firms.
The CEO and the VP of HR met with the researcher. After a few brief comments about the
data, the CEO began by inquiring about the diag- nostic process. He wanted to know if the data he was looking at was “good” data or not. Once sat- isfied that a sound process had been followed in terms of sampling and analysis, he turned his attention to the actual data. Like the design team, he asked some clarifying questions about the dis- tinction between strategic direction and cultural influences. He also asked some insightful ques- tions about specific words that had been chosen to capture the design team’s “sense” of the data.
His attention was mostly on the concerned and negative themes. Many of the issues (both positive and negative) raised were familiar to him and he doubted that the organization could fulfill the promise of The Cause with this set of weak- nesses. On the spot, he commissioned the HR vice president with leading the design team to for- mulate a change strategy to address the issues raised in the assessment.
The HR vice president and the researcher reconvened the design team and added members from other departments, such as IT and the regional organizations, to better represent the overall enter- prise. They began to develop an action plan for the change. It began with feedback of the assessment data to other parts of the organization. This hap- pened in two primary ways. First, the results were fed back to the existing senior leadership team. They were tasked with committing to the change and formulating statements that would represent an organizational future state. Second, the data were fed back to the top 150 leaders at the organi- zation’s annual leadership summit. This group had been the primary group sampled in the survey and they were given a chance to review the data, ask questions, and provide guidance on a proposed action plan.
The design team also formally commissioned four initiative task forces to address specific issues in the assessment. One team took on the challenge of revising the human capital management process (see Application 15.1 for a summary of this effort). A second task force was charted to diagnose and explore in more detail the issues surrounding peo- ple’s beliefs that it was hard to “get stuff done” at Cambia. A third team addressed the related issue of strategic planning and corporate communication. Was there a clear, well-understood, and shared pro- cess for setting organization objectives that were
CHAPTER 6 COLLECTING, ANALYZING, AND FEEDING BACK DIAGNOSTIC INFORMATION 151
6-6c Limitations of Survey Feedback Although the use of survey feedback is widespread in contemporary organizations, the following limits and risks have been identified:25
1. Ambiguity of purpose. Managers and staff groups responsible for the survey- feedback process may have difficulty reaching sufficient consensus about the pur- poses of the survey, its content, and how it will be fed back to participants. Such confusion can lead to considerable disagreement over the data collected and paraly- sis about doing anything with them.
2. Distrust. High levels of distrust in the organization can render the survey feedback ineffective. Employees need to trust that their responses will remain anonymous and that management is serious about sharing the data and solving problems jointly.
3. Unacceptable topics. Most organizations have certain topics that they do not want examined. This can severely constrain the scope of the survey process, particularly if the neglected topics are important to employees.
4. Organizational disturbance. The survey-feedback process can unduly disturb orga- nizational functioning. Data collection and feedback typically infringe on employee work time. Moreover, administration of a survey can call attention to issues with which management is unwilling to deal, and can create unrealistic expectations about organizational improvement.
6-6d Results of Survey Feedback Survey feedback has been used widely in business organizations, schools, hospitals, fed- eral and state governments, and the military. The navy has used survey feedback in more than 500 navy commands. More than 150,000 individual surveys were completed, and a large bank of computerized research data was generated. Promising results were noted among survey indices on nonjudicial punishment rates, incidence of drug abuse reports, and performance of ships undergoing refresher training (a post overhaul training and evaluation period).26 Positive results have been reported in such diverse areas as an industrial organization in Sweden and the Israeli Army.27
relevant to the managers and departments in the organization and how were those objectives com- municated? Finally, a fourth team was given the task of creating and implementing an organization- wide change-management process.
The design team and VP of HR worked on a vari- ety of organization changes. These included a realign- ment of the senior leadership group, supporting key leadership changes, changes in the leadership devel- opment programs, and the reorganization of several functional groups, including HR, and a complete redesign of the performance management process.
After one year of implementation, the design team commissioned a midpoint review to gauge
progress on the action plan. Interviews with the design team members, a sample of managers who had participated on the task forces, and a sample of managers and executives who had not been directly involved in the change effort were conducted. In general, the interview data supported that the change was heading in the right direction. Many people believed that, in fact, the culture was changing and that the work of the design team was an important contributor to that change. The interviewees also made a variety of suggestions for continuing different initiatives as well as suggestions for “next steps.”
152 PART 2 THE PROCESS OF ORGANIZATION DEVELOPMENT
One of the most important studies of survey feedback was done by Bowers, who conducted a five-year longitudinal study (the Intercompany Longitudinal Study) of 23 organizations in 15 companies involving more than 14,000 people in both white- collar and blue-collar positions.28 In each of the 23 organizations studied, repeat measurements were taken. The study compared survey feedback with three other OD interventions: interpersonal process consultation, task process consultation, and labora- tory training. The study reported that survey feedback was the most effective of the four interventions and the only one “associated with large across-the-board positive changes in organization climate.”29 Although these findings have been questioned on a number of methodological grounds,30 the original conclusion that survey feedback is effective in achieving organizational change was supported. The study suggested that any conclusions to be drawn from action research and survey-feedback studies should be based, at least in part, on objective operating data.
Comprehensive reviews of the literature reveal differing perspectives on the effects of survey feedback. In one review, survey feedback’s biggest impact was on attitudes and per- ceptions of the work situation. The study suggested that survey feedback might best be viewed as a bridge between the diagnosis of organizational problems and the implementa- tion of problem-solving methods because little evidence suggests that survey feedback alone will result in changes in individual behavior or organizational output.31 This view is sup- ported by research suggesting that the more the data were used to solve problems between initial surveys and later surveys, the more the data improved.32 Similarly, Church and his colleagues, based on a longitudinal evaluation of a survey feedback process in a large multi- national corporation, found that groups that shared feedback data and acted on that data were more likely to report positive attitudes about the company, their manager, job training, and support for work-life balance.33 The authors stated, “Put another way, the impact of sharing and acting on survey data on overall employee attitudes is (a) significant and pro- nounced, (b) replicable over time, (c) applies across different employee groups/levels, and (d) applies across content areas and overall rating tendencies. If there was ever a reason to decide to take action from an organizational survey effort, this is a clear mandate.”
Another study suggested that survey feedback has positive effects on both outcome variables (for example, productivity, costs, and absenteeism) and process variables (for example, employee openness, decision making, and motivation) in 53% and 48%, respec- tively, of the studies measuring those variables. When compared with other OD approaches, survey feedback was only bettered by interventions using several approaches together—for example, change programs involving a combination of survey feedback, process consultation, and team building.34 On the other hand, another review found that, in contrast to laboratory training and team building, survey feedback was least effective, with only 33% of the studies that measured hard outcomes reporting success. The success rate increased to 45%, however, when survey feedback was combined with team building.35 Finally, a meta-analysis of OD process interventions and individual atti- tudes suggested that survey feedback was not significantly associated with overall satis- faction or attitudes about coworkers, the job, or the organization. Survey feedback was able to account for only about 11% of the variance in satisfaction and other attitudes.36
Studies of specific survey-feedback interventions identify conditions that improve the success of this technique. One study in an urban school district reported difficulties with survey feedback and suggested that its effectiveness depends partly on the quality of those leading the change effort, members’ understanding of the process, the extent to which the survey focuses on issues important to participants, and the degree to which the values expressed by the survey are congruent with those of the respondents.37 Another study in the military concluded that survey feedback works best when supervisors play an active
CHAPTER 6 COLLECTING, ANALYZING, AND FEEDING BACK DIAGNOSTIC INFORMATION 153
role in feeding back data to employees and helping them to work with the data.38 Simi- larly, a field study of funeral cooperative societies concluded that the use and dissemina- tion of survey results increased when organization members were closely involved in developing and carrying out the project and when the consultant provided technical assis- tance in the form of data analysis and interpretation.39 Finally, a long-term study of survey feedback in an underground mining operation suggested that continued, periodic use of survey feedback can produce significant changes in organizations.40
SUMMARY
This chapter described methods for collecting, analyz- ing, and feeding back diagnostic data. Because diagnos- ing is an important step that occurs frequently in the planned change process, a working familiarity with these techniques is essential. Methods of data collection include questionnaires, interviews, observation, and unobtrusive measures. Methods of analysis include qualitative techniques, such as content analysis and force-field analysis, and quantitative techniques, such as the determination of mean, standard deviation, and frequency distributions; scattergrams and correlation
coefficients; as well as difference tests. Feeding back data to a client system is concerned with identifying the content of the data to be fed back and designing a feedback process that ensures ownership of the data. If members own the data, they will be motivated to solve organizational problems. A special application of the data collection and feedback process is called survey feedback, which enables OD practitioners to collect diagnostic data from a large number of organization members and to feed back that information for pur- poses of problem solving.
NOTES
1. S. Mohrman, T. Cummings, and E. Lawler III, “Creating Useful Knowledge with Organizations: Relationship and Process Issues,” in Producing Useful Knowledge for Orga- nizations, ed. R. Kilmann and K. Thomas (New York: Praeger, 1983): 613–24; C. Argyris, R. Putnam, and D. Smith, eds., Action Science (San Francisco: Jossey-Bass, 1985); E. Lawler III, A. Mohrman, S. Mohrman, G. Ledford Jr., and T. Cummings, Doing Research That Is Useful for Theory and Practice (San Francisco: Jossey-Bass, 1985).
2. D. Nadler, Feedback and Organization Development: Using Data-Based Methods (Reading, MA: Addison- Wesley, 1977): 110–14.
3. W. Nielsen, N. Nykodym, and D. Brown, “Ethics and Organizational Change,” Asia Pacific Journal of Human Resources 29 (1991).
4. Nadler, Feedback, 105–7.
5. W. Wymer and J. Carsten, “Alternative Ways to Gather Opinion,” HR Magazine, April 1992, 71–78.
6. Examples of basic resource books on survey methodology include L. Rea and R. Parker, Designing and Conducting Survey Research: A Comprehensive Guide (San Francisco:
Jossey-Bass, 2012); W. Saris and I. Gallhofer, Design, Evaluation, and Analysis for Survey Research (New York: Wiley-Interscience, 2007); S. Seashore, E. Lawler III, P. Mirvis, and C. Cammann, Assessing Organizational Change (New York: Wiley-Interscience, 1983); E. Lawler III, D. Nadler, and C. Cammann, Organiza- tional Assessment: Perspectives on the Measurement of Organizational Behavior and the Quality of Work Life (New York: Wiley-Interscience, 1980).
7. J. Taylor and D. Bowers, Survey of Organizations: A Machine-Scored Standardized Questionnaire Instrument (Ann Arbor: Institute for Social Research, University of Michigan, 1972); C. Cammann, M. Fichman, G. Jenkins, and J. Klesh, “Assessing the Attitudes and Perceptions of Organizational Members,” in Assessing Organizational Change: A Guide to Methods, Measures, and Practices, ed. S. Seashore, E. Lawler III, P. Mirvis, and C. Cammann (New York: Wiley-Interscience, 1983), 71–138.
8. M. Weisbord, “Organizational Diagnosis: Six Places to Look for Trouble with or without a Theory,” Group and Organi- zation Studies 1 (1976): 430–37; R. Preziosi, “Organizational Diagnosis Questionnaire,” in The 1980 Handbook for Group
154 PART 2 THE PROCESS OF ORGANIZATION DEVELOPMENT