Assignment - w11 - Assessing

Performance Appraisal for Groups: Models and Methods for Assessing Group Processes and Outcomes for Development and Evaluation

Manuel London State University of New York at Stony Brook

This paper guides consulting psychologists in how to help managers and group leaders assess group members’ reactions, behaviors, and per- formance. The results may be used for develop- ment in improving group performance and for evaluation in making decisions about group members’ pay and assignments. Individual and group-level measures of conditions (pressures and opportunities), input, process, and out- comes are considered. The paper discusses who seeks group assessment, the multiple purposes of assessment, models of group process to guide assessment, what is assessed and when, methods for assessment, and who contributes to the as- sessment process. Implications for inculcating a culture of assessment and continuous learning within groups and organizations.

Keywords: group performance appraisal, group assessment, group development and evaluation, group diagnosis and intervention

The purpose of this paper is to help guide consultants in encouraging and sup- porting group assessment for development and the evaluation of individuals and groups. The paper examines ways to assess the behavior and performance of individu- als in groups and to assess processes and performance at the group level. I begin by considering why assessment of groups and group members is important in organiza- tions today, and I provide several case ex- amples. Then I discuss who seeks assess- ment, the multiple purposes of assessment, models of group process to guide assess- ment, what is assessed and when, methods for assessment, and who contributes to the assessment process.

Background

Jobs in the world of today are complex, interdependent, and fluid. Much work is done in groups (Hackman, 2002; Kozlowski & Ilgen, in press). Projects and tasks cut across functions and organizational levels. Professionals, technical experts, managers, and staff members are asked to contribute to more than one group at a time. They may be leaders of some initiatives and contrib- utors to others. They may have a key role in some groups and a minor role in others. An employee may report to one supervisor in a given department and function but work on tasks in a variety of departments and groups. Some groups are ongoing. Others are short term. Some have clear goals and tight deadlines. Others have ambiguous goals and long time periods before they are required to report to a higher authority. In some cases, group members are colocated and can easily meet together in person. In other cases, they are geographically dis- persed and may communicate mainly through an ever-expanding array of elec- tronic technology (e.g., email, blogs, Web sites, instant messaging, video conferenc- ing, pod casts, and cell phones). These dy- namic group structures and multiple modes of communication pose challenges for super- visors to guide and evaluate the contributions

Correspondence concerning this article should be addressed to Manuel London, Har- riman Hall, SUNY-Stony Brook, Stony Brook, NY 11794-3775. E-mail: manuel [email protected]

Copyright 2007 by the American Psychological Association and the Society of Consulting Psychology, 1065-9293/07/$12.00 DOI: 10.1037/1065-9293.59.3.175 Consulting Psychology Journal: Practice and Research, Vol. 59, No. 3, 175–188

175

of individual employees and to assess and evaluate the performance of work groups. Because managers may have employees in groups that are not under their control, they will need to coordinate with other managers to decide which groups to evaluate and how to evaluate the results. In addition, they need to decide whether to collect data on individ- ual performance and/or the performance of the group as a whole.

Groups handle a range of tasks and take a variety of forms. These include commit- tees, boards of advisors and directors, councils, quality improvement teams, re- search and development teams, and task forces. Work groups can be classified by type (e.g., executive, negotiation, produc- tion, advisory, service), function (plan, di- rect, integrate, display, treat), and settings (corporate, medicine, transportation, fast food, law) (Devine, 2002). Groups may engage in problem solving (e.g., error de- tection and correction), decision making (a personnel selection committee), planning (strategy formulation), and/or implementa- tion (product roll out, customer service, event management).

Executives and managers are responsi- ble for these groups—starting them, lead- ing them, providing oversight, and staffing them with employees in their units. As such, they may be concerned about the groups’ progress, effectiveness, and out- comes. Consider several examples:

A hospital executive commissions contin- uous quality improvement (CQI) teams for a host of issues, such as emergency room re- sponsiveness, patient monitoring, safety, lab test accuracy, and so forth The groups are composed of staff from a variety of organi- zational levels and functions in the hospital, including physicians, nurses, lab technicians, therapists, and other professional and support staff. The executive wants to know how these groups are doing. Are people participating actively? Are they taking the time they need to collect data and delve into the problems? Are they implementing solutions and achiev-

ing gains in productivity and quality of ser- vice? Have they learned CQI techniques that they can apply to future quality improvement efforts?

A manager is responsible for the imple- mentation of an enterprise-wide, compre- hensive data system to handle customer, personnel, facilities, and financial data. The work is distributed to a number of sub- groups, each consisting of functional and technical specialists. The manager wants to know whether the members of these groups are working effectively with each other and whether the groups are making progress. Are the groups in synch with each other? Should members be rewarded for their ex- tra service contributions to this important and costly initiative?

A multinational automobile corporation has research and development teams for new vehicle design and technology. The members of many of these groups represent different functions (engineering, operations, market- ing), partner organizations (suppliers, distrib- utors, sales), and geographic regions, includ- ing international offices. Employees are likely to be on more than one group on un- related projects. Some groups rarely meet in person. Most rely on various electronic media to communicate, share knowledge and ideas, examine issues, experiment, and make deci- sions. Regional time differences, varying lan- guage capabilities, and cultural differences make group leadership and member interac- tion difficult. Some groups stay intact for years, working on ambitious, complex projects that have long time lines. Members of these groups may shift as the focus moves from the idea stage to design, implementa- tion, and delivery. The vice president for re- search and development wants to track these groups to know who is doing what and whether the groups are working effectively.

Considerable research and practice fo- cuses on performance appraisal of individ- uals (cf. Tziner, Murphy, & Cleveland, 2005). The challenge of measuring group members’ performance is that the output of

176 Consulting Psychology Journal: Practice and Research September 2007

the group can be observed and measured, but not necessarily the contributions of each member (Petersen, 1994). In addition, in many groups, the results of a given mem- ber depend on the efforts of all the workers. Wageman, Hackman, and Lehman (2005) noted that there are many instruments for measuring overall group process and per- formance, including consultant-developed, often online, rating formats that are meant to diagnose and assess group dynamics. As examples, they cited Cornelius Associates (2004); Lefton and Buzzotta (2005); Link- age Assessment Services (2004); Reliable Surveys Online (2005), and Parker’s (1998) print compilation of 25 such tools. These instruments tend to have high face validity, and they are used to give feedback and interpret the results in comparison to norms derived from a large number of other groups. However, they are not necessarily based on established theory and research on variables that are indeed important for per- formance (Wageman et al., 2005). In addi- tion, instruments for helping teams perform better must measure variables that are known to affect performance, can be ma- nipulated, and are applicable in many dif- ferent situations so that norms can be es- tablished for understanding a group’s scores. Instruments that are used in re- search need to be generalizable—that is, they need to measure concepts that are im- portant to describing a group and the con- ditions under which it is operating, and be psychometrically sound, with high reliabil- ity and convergent and discriminant valid- ity (Wageman et al., 2005). The value of measures of group output quality and quan- tity depend on the adequacy of information systems, the reliability of the measures, controllability of the constructs (i.e., they are constructs that group leaders and man- agers can do something about), and avail- ability of the results as feedback to group members (Reilly & McGourty, 1998).

Throughout this paper, I use the term assessment because it implies multiple pur-

poses. Assessment can examine group pro- cesses to diagnose the group members’ in- teractions, determine progress, and provide feedback to help the group perform better. In addition, assessment can measure group outcomes to reward its members and make decisions about the individuals and the group. Importantly, assessment is a pro- cess, not a one-time collection of data. This is similar to performance appraisal or 360 degree feedback surveys in the context of performance improvement and career de- velopment (London & Tornow, 1998). As a process, group assessment refers to deter- mining what is to be assessed, why, and who is involved. It includes designing the assessment, implementing it, using the in- formation (accepting and interpreting the results and taking action), and later mea- surement.

Who Seeks Assessment?

Group assessment may be useful to dif- ferent stakeholders. These include manag- ers and executives who commissioned the group or have oversight over the group. They need to evaluate the contributions of the people who work for them who are assigned to one or more groups with differ- ent leaders. They want to track group progress to know what is happening.

The group’s leader, as coach of the group, and the group’s facilitator, if there is one, would want to diagnose the group’s needs for support and development to fos- ter its effectiveness. The company’s orga- nization development officer, human re- source professional, director of training, in- house organizational psychologist, or outside consulting psychologist may want to assess group process and performance to determine needs for training and facilita- tion. In addition, they may work with ex- ecutives to select people for new groups or to form or restructure existing groups to enhance organizational performance and meet corporate needs (for instance, form a

177Consulting Psychology Journal: Practice and Research September 2007

task force to develop a new product in response to the competition). Psychological consultants may work with professionals and managers to design and implement group assessment methods and generally encourage and support group appraisal for development and evaluation.

Purposes for Assessment

Assessment may be for individual and group development or evaluation. Develop- ment refers to the compilation of learning over time. As a group develops, members learn more interdependent forms of inter- action (Kasl, Marsick, & Dechant, 1997; Kozlowski, Gully, Nason, & Smith, 1999). Assessment can diagnose learning gaps and determine interventions that will improve group process. Evaluation refers to deter- mining the quality and quantity of group and individual outputs, often to make deci- sions about the individuals in the group (pay, promotion, continuing them on the group) and about the group as a whole (e.g., whether to disband or maintain it, change its goals, reward all group members equally, etc.).

Data for development may focus on ex- amining the input to the group (whether the group has the talent it needs) and the con- ditions under which the group is operating (time pressure, availability of resources). Assessment data can determine group com- petency—the ability of the group to accom- plish its goals and the ability of members to contribute to the group. Group and individ- ual competency requirements depend on the nature of the group (e.g., problem solv- ing, decision making, production), the na- ture of the task (e.g., its complexity and the degree of member interdependence; Kasl, Marsick, & Dechant, 1997), the conditions under which the group is operating (time pressure, resource availability), and mem- bers’ readiness to participate actively and cooperate (London & Sessa, 2006a; Reilly & McGourty, 1998). In addition, data for

development may focus on group pro- cess—member interactions, conflict resolu- tion, negotiations, attendance, and meeting commitments, to cite a few examples.

Another reason for assessment data is to evaluate the effectiveness of different group intervention methods. For instance, a facilitator might stop the group process to talk about how things are working or ways of negotiating and resolving conflict. Does this help? Are some group interventions better than others under certain conditions? Group assessment may track changes in group process and performance over time, determine changes in members’ behaviors toward each other, determine the effects of these changes on group outputs, and com- pare groups that are treated differently as a field experiment.

Assessment can also be used by manag- ers and group leaders to evaluate members’ contributions and incorporate this informa- tion in the members’ annual performance appraisal as well as provide members with feedback to improve their performance. If the manager’s employees work interdepen- dently on a daily basis in their natural work team, the manager may want a way to as- sess their independent and collective effort as part of the annual performance appraisal. Such data can be used to hold the leader and the members of the group accountable for the group’s results. In addition, mem- bers of the group may share equally in an award that recognizes the group’s accom- plishments. Group skills and members’ abilities can be assessed and used as input for future assignments. Executives can use the results to make decisions about the group, such as whether the group should continue, whether new members should be added and others dropped, and/or whether the group needs a change in leadership.

The purpose for collecting the data may affect the data itself. For instance, raters may be more lenient if they know that the data will be used to make administrative decisions than if the data will be used

178 Consulting Psychology Journal: Practice and Research September 2007

solely for development; Kozlowski, Chao, & Morrison, 1998; London, 2001; London, Smither, & Adsit, 1997). The purpose may also affect how the data are perceived and used as feedback, for instance, whether the group leader and members will be held accountable for their performance (London & Sessa, 2006b).

Group performance data can be com- pared within the organization to determine effective leadership behaviors, task struc- tures, goals, and resources. Surveys of group members’ feelings about the group can evaluate group climate. When averaged across groups, survey data can measure or- ganizational climate. The results may have implications for employee retention as well as the ability of the organization to be ef- fective—to develop and implement new products and services that create new mar- kets or beat the competition. Assessment across groups can also be a way for an executive to determine if the organization has adopted the culture that the executive wants to promulgate.

Deciding Which Groups to Assess

Managers are likely to be responsible for different types of groups. These may include natural work teams (the employees who report directly to the manager), ongo- ing groups (boards, councils, standing com- mittees), and special initiatives (task forces, product development teams, quality im- provement teams). Assessing these differ- ent groups provides the manager with a picture of the state of organizational per- formance and development. Different eval- uation techniques may be needed for these different groups, recognizing (a) their de- gree of permanency (whether they are of limited duration and will disband after ac- complishing a specific goal; long-term with complex goals that will take months or longer to accomplish; or ongoing, meant to accomplish multiple goals or a continuous task), (b) imposition on employees’ time (infrequent periodic meetings or concen-

trated time working together), (c) mem- bers’ modes of interaction (in-person and/or online), and (d) the group’s stage of development (early, mid, late) in relation to accomplishing their goals and members’ familiarity with each other and experience collaborating (group maturity).

For a permanent group, the manager might want to know about member atti- tudes about the group and the leader as well as the accomplishments of the group during a range of time (e.g., the activities of a council during the last year). Managers may want to know whether employee time spent on standing committees, councils, or boards is worthwhile and whether these groups are adding value to the organization.

Short-term groups can be evaluated at different stages of their group progress. During the early stage (before and shortly after the first meeting or two), questions for members, the leader, and observing super- visors can focus on whether the group has the talent it needs, members who are aware of each other’s backgrounds and expertise, motivated members, a clear goal, and clear expectations of what they are expected to do. When the group task is underway (ap- proximately at the midpoint), assessment can focus again on whether members are clear about their goals, the structure of the task(s), and responsibilities. Questions can also address how well members are inter- acting with each other— how well they are getting along, participating actively, sup- porting each other, being critical of each other, and contributing to, or standing in the way of, progress. As the group con- cludes its work and prepares to disband, or after the group has ended, evaluation can focus on the value of the work to the orga- nization, what the members learned about the subject matter, and whether the group members have gained capability to contrib- ute to groups in the future (i.e., their ability and desire to cooperate and work produc- tively in a group setting).

179Consulting Psychology Journal: Practice and Research September 2007

When Should Group Evaluations Be Conducted?

For ongoing groups, formal evaluations may be conducted annually as input to in- dividual performance appraisals and, con- comitantly, as an assessment of organiza- tional effectiveness. Organizational assess- ment might be incorporated into an annual corporate performance report along with financial data. For short-term groups, for- mal assessments can be conducted as the group concludes its work or immediately thereafter. This is when the members are likely to be open to discussing how they did as a group and receiving group feedback (Hackman & Wageman, 2005). In addition, this is the time to capture the leader’s and members’ opinions of the members’ indi- vidual contributions while they are still fresh in their minds. In long- and short-term groups, informal assessments should be conducted in relation to the stage of the group to assess environmental conditions, input, and process variables.

Who Participates in the Assessment Process?

Methods for group assessment may be designed centrally and applied organiza- tion-wide. This establishes common stan- dards and expectations for groups across the organization. Examples might be the use of standard procedures, such as surveys that measure members’ perceptions of group process and outcomes and ratings of individual members’ contributions and per- formance.

The data may be collected by the human resource department, department managers, group leaders, or group facilitators (profes- sional organization development consult- ants who work within the organization or who are external consultants). Attitude sur- veys and performance ratings from the group leader and members may be col- lected during group meetings or away from the group setting to guarantee confidential-

ity of the ratings. Data for evaluation pur- poses should be collected separately from data for diagnosis and development. Whether survey data are collected for eval- uation and/or development, the data should be collected in a way that promotes raters’ openness and honesty. Raters need to know and understand the purpose for the data and who will have access to it. Data for devel- opment may remain within the group or averaged across group members for discus- sion with managers about group progress and resources needed to improve group functioning. Data for evaluation may reflect individual as well as group performance and be compared to other data (e.g., ob- server ratings, objective indexes of group results) to make decisions about the group as a whole.

Group leaders and members can be ex- cellent sources for designing the assess- ment process, determining how data are collected, reviewing and interpreting the results and discussing implications for changing behaviors and interactions. The group leader and members themselves are key stakeholders for the group’s success. As partners in the assessment process, along with executives and managers, they should feel that they “own” the data and have a stake in using it to foster their per- formance. As such, they will take the data collection process seriously, share and in- terpret the results, and use the results as stimuli to suggest directions for behavior change.

What Is Assessed?

I distinguish between two general ap- proaches to assessment. The constructivist approach asks whether the right ingredients are present (time, talent, and task; Ericksen & Dyer, 2004), determine conditions (re- sources, pressures, and opportunities), and specifies interventions to help the group (e.g., change members, provide group training, hold a process discussion with member, and/or find more resources). The

180 Consulting Psychology Journal: Practice and Research September 2007

deconstructivist approach starts with the group as it is in the moment and tries to discern what is happening or has happened. It seeks to identify the barriers to more effective process and what factors are ham- pering or helping group progress, and what can be done to address these factors (e.g., process reflection about what the group needs to learn or has learned to improve its performance).

Group process will depend on the nature or purpose of the group. For instance, pro- cess measures might focus on collaboration (sharing information and joint decision making). Process measures might assess brainstorming used to generate alternatives, facilitated discussion and voting methods to evaluate alternatives, and the availability and use of specific techniques related to the task, such as fishbone analysis to examine root causes of problems in quality improve- ment teams. Data can also focus on group progress in terms of task and goal clarity, leadership, member participation, and the social context (Murphy & Cleveland, 1995). The leader may prepare feedback to individual members or to the group as a whole and determine ways to improve the group’s performance and create a high per- forming team. Similarly, a facilitator may use assessment results to determine what training or organizational methods the group might use to be more productive. These might include teaching the group continuous quality improvement/total qual- ity management (CQI/TQM) skills such as identifying customers, elements of work process, and problem frequency; using brainstorming; and applying methods for reaching consensus (e.g., multivoting).

Measures should reflect what managers want to know about the group. These mea- sures may vary from time to time depend- ing on the manager, the group, and its progress. For instance, for quality improve- ment teams, measures may assess how well members have learned to apply CQI skills and methods (Hackman & Wageman,

1995; Zbaracki, 1998). Assessment of CQI teams that are working with professional facilitators may focus on the facilitator’s role (e.g., “The facilitator . . .guides the team through problem solving methods, in- cluding consensus building; . . .helps the team leader and members to establish ground rules; . . .helps the team resolve conflicts; . . .provides training in CQI meth- ods as needed”; Wilkens & London, 2006). Other measures may be constructed to as- sess the effects of specific interventions, for instance, leader training in group coaching and facilitation (e.g., Do group members see a change in leaders’ behaviors?); con- flict resolution, negotiation, and collabora- tion (Do behaviors and interactions change over time? Do groups with the training perform better than comparison groups without the training?); and facilitation to promote a psychologically safe environ- ment (e.g., Do members feel they are able to bring up issues about how well they get along in the group?; Edmondson, 2002; Edmondson, Bohmer, & Pisano, 2001; West & Anderson, 1998).

Group Performance Dimensions

Performance dimensions to evaluate group members can be identified. These might include commitment to (attitudes about) the work and the group, active par- ticipation, degree of added value, handling conflict, negotiating, educating others, openness to others’ views, openness to new ideas, giving and seeking feedback and support, accepting roles, and meeting obli- gations. Performance dimensions to evalu- ate group leaders may include structuring the group’s task and agenda, maintaining order, treating members fairly and with re- spect, delegating tasks, and giving feed- back. Behaviors can be conceptualized for each of these dimensions and used to assess the group and/or leader.

Measures can also focus on the group as a whole, for instance, number and length of the meetings, overall attendance, goal clar-

181Consulting Psychology Journal: Practice and Research September 2007

ity, task structure, goal difficulty, clarity and frequency of communication among members, social and task orientated inter- actions, goal achievement (e.g., deadlines met), quality and quantity of output, and influence on other groups in the organiza- tion.

Models for Guiding Assessment Content

The models of group and organizational process held by the manager and consultant can and do guide group assessment. A number of models are available, each of which presents a perspective on what vari- ables are important to effective group pro- cess. Some are content oriented, specifying variables that are related to performance, such as composition, cohesiveness, and motivation (cf. Guzzo & Dickson’s, 1996, review). Others consider the sequence of these variables—for instance, ongoing, nonlinear cycles of input, mediation, output, and feedback (Ilgen, Hollenbeck, Johnson, & Jundt, 2005) along with concomitant cog- nitive and affective states (Marks, Mathieu, & Zaccaro, 2001). Ericksen and Dyer (2004) highlighted the importance of talent, task, and time as components of a high performing team. Hackman and Wageman (2005) pointed out the importance of rec- ognizing how stage of group development (early, mid, and late) determines what the group needs and is expected to do, as dis- cussed earlier. Another approach is a focus on interventions, such as the importance of the group members introducing themselves to each other at the outset of the group so that members develop a common under- standing of how each of them can contrib- ute to the group—a concept called interper- sonal congruence (Polzer, Milton, & Swann, 2002).

Wageman, Hackman, and Lehman (2005) state that a high performing team needs (a) to be a real team—meaning that the team is bounded such that everybody knows who is and isn’t on the team, the

team is stable over time, and members are interdependent; (b) a compelling direc- tion—meaning that team direction is clear, challenging, and consequential; (c) an en- abling structure—meaning that the team size, diversity, and skills make sense, the task is motivating, and the norms of con- duct are clear and accepted, and (d) a leader who is a supportive coach—meaning that the leader helps members focus on the task (setting direction, building commitment to group purpose), gives members positive and corrective feedback, facilitates their in- terpersonal relationships, and avoids un- helpful behaviors such as micromanaging. Wageman et al. (2005) developed the Team Diagnostic Survey to assess each of these concepts. The instrument also assesses the quality of team task processes by measur- ing the level of effort members collectively expend on the task, the quality of team task performance strategies, and the degree to which the team uses the full complement of member knowledge and skill.

Several models posit that groups can and should become continuous learning en- tities. Argyris and Schön (1996) describe a Model II, double-loop learning group that is open to clear and honest discussion of feedback and process. West (2002) shows how group reflexivity (taking time to dis- cuss process) can enhance team perfor- mance. Senge (1994) explains the value of generative learning in seeking, sharing, and applying new knowledge and skills. These models all suggest the use of reflection as a method for both assessing process and moving the group to a new stage of devel- opment. They also suggest questions for diagnosis. These assessment and interven- tion processes usually require a consultant and the willing involvement of the group leader and members. The leader can en- courage create a learning environment in a group, for instance, by rewarding learning and discovery along with performance, helping members develop and use learn- ing tools, such as reflection, and provid-

182 Consulting Psychology Journal: Practice and Research September 2007

ing feedback and coaching (Zaccaro, Hildebrand, & Shuffler, in press).

Sessa and London (2006) argued that groups can learn to adapt in reaction to pressures and opportunities, be generative, and/or transform (change members, goals, and methods in a radical way) when nec- essary. Groups that are pressured to change and that are sufficiently mature (open to new ideas from within and outside the group, ready to share experiences, and try new modes of interaction) will be genera- tive or transformative. Assessment meth- ods can evaluate pressures and readiness to learn early in a group’s life, measure learn- ing as the group progresses, and assess outcome as the group reaches its final stage.

Categories for Diagnosis and Assessment

Consider six categories of variables that could be included in a group assessment: environmental conditions, individual com- petencies, group competencies, process variables, individual outcomes, and group outcomes. What you ask about may depend on how long the group has been operating.

Environmental Conditions

These include the pressures and oppor- tunities that impinge on the group. Pres- sures may be a tight economy, competition, technology, deadlines, goals and expecta- tions, resource constraints, limitations on meeting facilities and modes of communi- cation, and so forth. Opportunities may be the inverse of these conditions: a favorable economy, lack of competition, new tech- nology or other proprietary ideas or prod- ucts, more resources, better facilities, and so forth. These conditions may change over time. Early assessment can call them to light and determine what conditions need a response or can be altered. Some conditions can be changed; others cannot. For in- stance, the economy and competition prob-

ably cannot be changed, but often resources can be reallocated, goals can be made more realistic, and facilities can be improved.

Individual Competencies

Assessment of individual competencies may be most important when managers and group leaders are selecting members for a group and during the early stage to deter- mine interventions while they can still be useful. Individual competencies include the leader’s and members’ abilities, experi- ence, and knowledge in the subject matter area and working with groups. Other indi- vidual competencies are individuals’ learn- ing orientation (the extent to which group members are proacative learners—that is, people with a strong desire to learn and improve; Dweck, 1986; VandeWalle, 1997), psychological collectivism (the ex- tent to which they prefer working in a group rather than alone; Jackson, Colquitt, Wesson, & Zapata-Phelan, 2006), and other individual skills that promote group pro- cess, such as self-management, communi- cation, decision making, and collaboration (willingness to share knowledge and make joint decisions) (Reilly & McGourty, 1998).

Group Competencies

Over time, a group develops competen- cies. Short-term groups have little time to build these competencies and refine them. Long-term groups that work on multiple tasks are likely to develop repeated patterns of interaction. Some will be effective; oth- ers will not, depending on the success of outcomes, members’ motivation, how well members get along with one another, and the leader’s ability to control, support, and coach the group. Functionality of interac- tions will be evident in the way the group members communicate, resolve conflicts, share knowledge, set goals, and plan, assign, allocate, and coordinate tasks and responsi- bilities (Cannon-Bowers, Tannenbaum,

183Consulting Psychology Journal: Practice and Research September 2007

Salas, & Volpe, 1995; Ellis, Bell, Ployhart, Hollenbeck, & Ilgen, 2005; Stevens & Campion, 1994; Swezey & Salas, 1992).

Process Variables

Elements of process includes behaviors, activities, progress, and barriers. What is assessed will depend on who is conducting the assessment and for what purpose. An executive who is monitoring the group (or multiple groups across the organization) does not necessarily need or want to know about what is happening during individual meetings. This would be micromanage- ment. However, a facilitator or consulting psychologist might want to know how well groups are operating, perhaps with an eye to introducing leader training. A facilitator who is working with a specific group may observe several meetings and interview or survey members. Barriers to effective pro- cess may include member disagreements, lack of patience, poor negotiation skills, the leader’s lack of control over meetings, foundering (no clear goals or ways of achieving goals), ignoring environmental conditions (boundary impermeability), and member insensitivity to each other.

The conditions that facilitate group pro- cess may also facilitate the collection and use of assessment data. Dysfunctional groups are not likely to be open to obser- vation, feedback, and training, making the leader’s and facilitator’s jobs all the more difficult and highlighting the importance of using different assessment techniques (ob- servations, surveys, consultation with the leader and members over time).

Individual Outcomes

Kirkpatrick’s (1998) criteria for learning assessment suggest that measures of mem- bers’ learning can be reactions, knowledge and skills acquired, behaviors changed, and results/outcomes. Jackson et al. (2006) in- dicated that group outcome measures may include (a) task performance (the profi-

ciency with which members perform their specific roles; Borman & Motowidlo, 1993), (b) citizenship behaviors (behaviors that are voluntary and not part of the indi- vidual’s job description or expectations and not formally rewarded by the group; e.g., Smith, Organ, & Near, 1983), (c) counter- productive behaviors (intentional behaviors that are contrary to the group’s interests; Sackett & DeVore, 2001), and (d) withdrawal behaviors (a passive avoidance of group work, reflecting dissatisfaction; Hulin, 1991).

Group Outcomes

Following Kirkpatrick (1998), measures of groups as learning systems may be atti- tudinal (average commitment of members to the group, and average satisfaction with the group output), knowledge and skills learned (measures aggregated across mem- bers of the group), behaviors changed (new patterns of interaction), and performance (the quality and/or quantity of group re- sults, such as events planned and imple- mented, units produced, services provided, decisions made, etc.). Over time, members learn and share perceptions and interaction patterns that contribute to the group’s ca- pabilities for the future, assuming the group has a future. Examples of these perceptions and interaction patterns include interper- sonal congruence (members viewing each other similarly; Polzer et al., 2002), a shared mental model (a common under- standing of how the group operates; Stout, Cannon-Bowers, Salas, & Milanovich, 1999), transactive memory (members’ col- lective knowledge of who in the group has expertise that can be relied on for different tasks when conditions change; Lewis, 2003, 2004; Wegner, 1986), collective ef- ficacy (the shared belief that the group is able to accomplish difficult goals; Myers, Feltz, & Short, 2004), and learning orien- tation (the average learning orientation across members; Bunderson & Sutcliffe, 2003). (Measures are readily available for

184 Consulting Psychology Journal: Practice and Research September 2007

each of these concepts from the references provided.)

Methods and Sources of Assessment

Data on group environment, input, pro- cess, and performance may be objective or subjective. Data may focus on resources (dollars, facilities, equipment available, etc.) and member characteristics (diversity of skills, experience, and knowledge). Ob- jective data may be available on the group’s operations (number of meetings, attendance, participation) and outcomes (achieving goals; quantity of output, if rel- evant; quality of output). Data may be sur- vey ratings or interview statements. Ob- server data can also be collected from peo- ple who have worked with the group, such as a facilitator or consultant, supervisors who commissioned the group or who su- pervise members of the group, other exec- utives, suppliers of information and re- sources, and customers/users of the group’s output.

Ratings From Members and the Leader

Group members who have been working together and know each other well should be able to assess each other’s performance and the leader’s performance and agree on their perceptions. Peer and upward ratings, as well as the leader’s evaluation of the group members, can be collected. Interrater agreement can be calculated. Behavioral expectation scales may be developed to rate the group on specific dimensions of perfor- mance or capability. For instance, a scale might measure “Ability to make Joint De- cisions” with examples of scales points be- ing, “This group can be expected to reach consensus quickly.”

Guidelines for Practice

Organization leaders can incorporate group assessment as part of performance appraisal and organization development.

Data can be collected across groups to de- termine employees’ capabilities to work within groups and to determine areas for training people to be more effective at group work. Data can be collected about individual groups to track their perfor- mance over time and to determine interven- tions that will facilitate their process and improve their outcomes. Also, data can be collected about employees’ individual con- tributions to groups, with input from the group’s leader, other members (peer rat- ings), the group facilitator, or other observ- ers of the group’s output (e.g., executives who know about the group and use its output). Organization development con- sultants can work with executives and man- agers to design the group appraisal process and use the results for individual and group development, evaluation, and reward. They can develop guidelines to help managers and group leaders design a group assess- ment process and use the results, for in- stance, how to feed back the results to individual member and the group as a whole and encourage group leaders to share and process the results (see London & Sessa, 2006b).

Reilly and McGourty (1998) offered several recommendations for the design and implementation of group performance appraisal. They suggested (a) identifying and defining behaviors that are important to the organization and that are known to re- late to effective team performance, (b) es- tablishing the relevance of measures to the group leader, members, and other stake- holders, including customers and execu- tives outside the group, (c) measuring each group member’s competencies and collect- ing group members’ perceptions of each other’s contributions and that of the overall team functioning and performance, (d) be- ing clear about the purpose(s) of assess- ment, and (e) using the assessment for group and individual development only, at least initially, describing desired group be- haviors, assuring rater confidentiality.

185Consulting Psychology Journal: Practice and Research September 2007

Executives and managers can determine the value of using a normative survey in- strument to assess group process and out- comes, such as the Team Diagnostic Survey (Wageman et al., 2005). The advantage of this instrument is that it is an online tool, and it gives groups feedback on design, leadership, internal dynamics, and condi- tions of team effectiveness. To test the in- strument and request use, Wageman et al. invited readers to visit the TDS website at https://research.wjh.harvard.edu/TDS/.

Another approach is to customize a sur- vey instrument or interview protocol to in- clude the constructs of interest. Many of the variables mentioned above have sample items in research publications. In other cases, measurement methods can be devel- oped locally and applied across the organi- zation or in a single group. The reliability of the instruments would need to be deter- mined (test–retest reliability, agreement among members, and internal consistency of the items).

As part of the implementation of a group appraisal, the purpose needs to be clear to those who provide and use the data. Indeed, when the group is formed, the group mem- bers should be aware that assessments would be conducted at various stages. Man- agers and group leaders should be sure that members know they will be assessed as individuals and as a group. The nature of the assessment can be described. This is a way to convey to the group members the behaviors and outcomes that are expected and rewarded.

The assessment should be timed in rela- tion to the stage of the group. Early assess- ments may focus on group members’ capa- bilities and motivation and may be part of the process of selecting and fine-tuning membership. Midway assessments may fo- cus on group member interactions and may be a part of process-consultation, facilita- tion, and member training in group tech- niques. Late assessments (after the group has achieved its goals, or as the group

reaches a key transition point moving from one task to another) may focus on perfor- mance and learning that may be part of evaluation for rewards and a determination of future capacity of members and the group.

Finally, over time, group assessment, combined with facilitation, feedback, and process improvement, can become a natural part of the organizational culture. Employ- ees who participate in groups will expect assessment in the same way they expect individual performance appraisal. In addi- tion, they will learn how to improve their group skills, and their groups will concen- trate on how to improve their group inter- action and outcomes. This will strengthen the value of group facilitation and increase self-management, helping group leaders be better coaches and helping consulting psy- chologists be better facilitators.

References

Argyris, C., & Schön, D. (1995). Organizational learning II: Theory, method, and practice (2nd ed.). New York: Prentice Hall.

Borman, W. C., & Motowidlo, S. J. (1993). Expanding the criterion domain to include elements of contextual performance. In N. Schmitt, W. C. Borman & Associates (Eds.), Personnel selection in organizations (pp. 71–98). San Francisco: Jossey-Bass.

Bunderson, S., & Sutcliffe, K. (2003). Manage- ment team learning orientation and business unit performance. Journal of Applied Psy- chology, 88, 552–560.

Cannon-Bowers, J. A., Tannenbaum, S. I., Salas, E., & Volpe, C. E. (1995). Defining competencies and establishing team training requirements. In R. A. Guzzo & E. Salas (Eds.), Team effectiveness and decision mak- ing in organizations (pp. 333–380). San Francisco: Jossey-Bass.

Cornelius Associates. (2004). Team roles anal- ysis: Self-assessment form for team building. Available at http://www.corneliusassociates .com

Devine, D. J. (2002). A review and integration of classification systems relevant to teams in organizations. Group Dynamics: Theory, Re- search, and Practice, 6, 291–310.

186 Consulting Psychology Journal: Practice and Research September 2007

Dweck, C. S. (1986). Motivational processes affecting learning. American Psycholo- gist, 41, 1040 –1048.

Edmondson, A. C. (2002). The local and varie- gated nature of learning in organizations: A group-level perspective. Organization Sci- ence, 13, 128 –146.

Edmondson, A. C., Bohmer, R. M., & Pisano, G. P. (2001). Disrupted routines: Team learning and new technology implementation in hospitals. Administrative Science Quar- terly, 46, 685–716.

Ellis, A. P. J., Bell, B. S., Ployhart, R. E., Hollenbeck, J. R., & Ilgen, D. R. (2005). An evaluation of generic teamwork skills train- ing with action teams: Effects on cognitive and skill-based outcomes. Personnel Psy- chology, 58, 641– 672.

Ericksen, J., & Dyer, L. (2004). Right from the start: Exploring the effects of early team events on subsequent project team develop- ment and performance. Administrative Sci- ence Quarterly, 49, 438 – 471.

Guzzo, R. A., & Dickson, M. W. (1996). Teams in organizations: Recent research on perfor- mance and effectiveness. Annual Review of Psychology, 47, 307–338.

Hackman, J. R. (2002). Leading teams: Setting the stage for great performances. Boston: Harvard Business School Press.

Hackman, J. R., & Wageman, R. (1995). Total quality management: Empirical, conceptual, and practical issues. Administrative Science Quarterly, 40, 309 –342.

Hackman, J. R., & Wageman, R. (2005). A theory of team coaching. Academy of Man- agement Review, 30, 269 –287.

Hulin, C. L. (1991). Adaptation, persistence, and commitment in organizations. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (Vol. 2, pp. 445–506). Palo Alto, CA: Con- sulting Psychologists Press.

Ilgen, D. R., Hollenbeck, J. R., Johnson, M., & Jundt, D. (2005). Teams in organizatioins: From input-process-output models to IMOI models. Annual Review of Psychology, 56, 517–543.

Jackson, C. L., Colquitt, J. A., Wesson, M. J., & Zapata-Phelan, C. P. (2006). Psychological collectivism: A measurement validation and linkage to group member performance. Jour- nal of Applied Psychology, 91, 884 – 899.

Kasl, E., Marsick, V. J., & Dechant, K. (1997). Team as learners: A research-based model of team learning. Journal of Applied Behav- ioral Science, 33, 227–246.

Kirkpatrick, D. L. (1998). Evaluating training programs: The four levels (2nd ed.). San Francisco: Jossey-Bass.

Kozlowski, S. W. J., Chao, G. R., & Morrison, R. F. (1998). Games raters play: Politics, strategies, and impression management in performance appraisal. In J. W. Smither (Ed.), Performance appraisal: State of the art in practice (pp. 163–207). San Francisco: Jossey-Bass.

Kozlowski, S. W. J., Gully, S. M., Nason, E. R., & Smith, E. M. (1999). Developing adaptive teams: A theory of compilation and perfor- mance across levels and time. In D. R. Ilgen & E. D. Pulakos (Eds.), The changing nature of performance: Implications for staffing, motivation, and development (pp. 240 –292). San Francisco: Jossey-Bass.

Kozlowski, S. W. J., & Ilgen, D. R. (2006) Enhancing the effectiveness of work groups and teams. Psychological Science in the Public Interest, 7, 77–124.

Lefton, R. E., & Buzzotta, V. R. (2005). The team scorecard. Retrieved October 11, 2006, from http://www.communicare.com

Lewis, K. (2003). Measuring transactive mem- ory systems in the field: Scale development and validation. Journal of Applied Psychol- ogy, 88, 587– 604.

Lewis, K. (2004). Knowledge and performance in knowledge-worker teams: A longitudinal study of transactive memory systems. Man- agement Science, 50, 1519 –1533.

Linkage Assessment Services. (2004). The HiPlay team assessment. Burlington, MA: Linkage, Inc.

London, M. (2001). The great debate: Should multisource feedback be used for administra- tion or development only? In D. W. Bracken, C. W. Timmreck, & A. H. Church (Eds.), The handbook of multisource feedback: The comprehensive resource for designing and implementing MSF processes (pp. 368 –385). San Francisco: Jossey-Bass.

London, M., & Sessa, V. I. (2006a). Continuous learning in organizations: A living systems analysis of individual, group, and organiza- tion learning. Research in Multi-Level Is- sues, 5, 123–172.

London, M., & Sessa, V. I. (2006b). Group feedback processes. Human Resource Devel- opment Review, 5, 303–329.

London, M., Smither, J. W., & Adsit, D. J. (1997). Accountability: The Achilles heel of multi-source feedback. Group and Organi- zation Management, 22, 162–184.

London, M., & Tornow, W. W. (1998). Intro- duction: 360-degree feedback—More than a

187Consulting Psychology Journal: Practice and Research September 2007

tool! In W. W. Tornow & M. London (Eds.), Maximizing the value of 360-degree feed- back: A process for successful individual and organizational development (pp. 1– 8). San Francisco: Jossey-Bass.

Marks, M. A., Mathieu, J. E., & Zaccaro, S. J. (2001). A temporally based framework and taxonomy of team processes. Academy of Management Review, 26, 356 –376.

Murphy, K. R., & Cleveland, J. N. (1995). Un- derstanding performance appraisal: Social, organizational, and goal-based perspectives. Thousand Oaks, CA: Sage

Myers, N. D., Feltz, D. L., & Short, S. E. (2004). Collective efficacy and team performance: A longitudinal study of collegiate football teams. Group Dynamics: Theory, Research, and Practice, 8, 126 –138.

Parker, G. R. (1998). 25 instruments for team building. Amherst, MA: HRD Press.

Petersen, T. (1994). Individual, collective, and systems rationality in work groups: Dilem- mas and solutions. Institute of Industrial Re- lations, University of California, Berkeley, Working Paper Series Number 58 –94.

Polzer, J. R., Milton, L. P., & Swann, W. B., Jr. (2002). Capitalizing on diversity: Interper- sonal congruence in small work groups. Ad- ministrative Science Quarterly, 47, 296 – 324.

Reilly, R. R., & McGourty, J. (1998). Perfor- mance appraisal in team settings. In J. W. Smither (Ed.), Performance appraisal: State of the art in practice (pp. 244 –277). San Francisco: Jossey-Bass.

Reliable Surveys Online. (2005). Team effec- tiveness. Available at http://reliablesurvey- s.com.

Sackett, P. R., & DeVore, C. J. (2001). Coun- terproductive behaviors at work. In N. Anderson, D. S. Ones, H. K. Sinangil, & C. Viswesvaran (Eds.), Handbook of industrial, work, and organizational psychology (Vol. 1, pp. 145–151). Thousand Oaks, CA: Sage.

Senge, P. (1994). The fifth discipline: The art and practice of the learning organizations. New York: Currency Doubleday.

Sessa, V. I., & London, M. (2006). Continuous learning in organizations: Individual, group, and organizational perspectives. Mahwah, NJ: Erlbaum.

Smith, C. A., Organ, D. W., & Near, J. P. (1983). Organizational citizenship behavior: Its nature and antecedents. Journal of Ap- plied Psychology, 68, 653– 663.

Stevens, M. J., & Campion, M. A. (1994). The knowledge, skill, and ability requirements for teamwork: Implications for human re- source management. Journal of Manage- ment, 20, 503–530.

Stout, R. J., Cannon-Bowers, J. A., Salas, E., & Milanovich, D. M. (1999). Planning, shared mental models, and coordinated perfor- mance: An empirical link is established. Hu- man Factors, 41, 71–71.

Swezey, R. W., & Salas, E. (1992). Guidelines for use in team-training development. In R. W. Swezey & E. Salas (Eds.), Teams: Their training and performance (pp. 219 – 245). Norwood, NJ: Ablex.

Tziner, A., Murphy, K. R., & Cleveland, J. N. (2005). Performance appraisal: Evolution and change. Group & Organization Manage- ment, 30, 4 –5.

VandeWalle, D. (1997). Development and val- idation of a work domain goal orientation instrument. Educational and Psychological Measurement, 57, 995–1015.

Wageman, R., Hackman, J. R., & Lehman, E. (2005). Team diagnostic survey: Develop- ment of an instrument. The Journal of Ap- plied Behavioral Science, 41, 373–398.

Wegner, D. M. (1986). Transactive memory: A contemporary analysis of the group mind. In B. Mullen & G. R. Goethals (Eds.), Theories of group behavior (pp. 185–208). New York: Springer-Verlag.

West, M. A. (2002). Sparkling fountains or stag- nant ponds: An integrative model of creativ- ity and innovation implementation in work groups. Applied Psychology: An Interna- tional Review, 51, 355– 424.

West, M. A., & Anderson, N. (1998). Measuring climate for work group innovation: Develop- ment and validation of the team climate in- ventory. Journal of Organizational Behav- ior, 19, 235–258.

Wilkens, R., & London, M. (2006). Relation- ships between climate, process, and perfor- mance in hospital continuous quality im- provement groups. Journal of Vocational Behavior, 69, 510 –523.

Zaccaro, S. J., Hildebrand, K., & Shuffler, M. (in press). The leader’s role in group learn- ing. In V. I. Sessa & M. London (Eds.). Group learning. Mahwah, NJ: Erlbaum.

Zbaracki, M. J. (1998). The rhetoric and reality of total quality management. Administrative Science Quarterly, 43, 602– 636.

188 Consulting Psychology Journal: Practice and Research September 2007