Powerpoint

Effective BY JAMES GRABOWSKI; ROBERT J. GREENE, PH.D.;

PETER RONZA; VALERIE JOHNSON; AN D LYNN LOHMAN

KEY TO EQUITABLE PERFORMANCE APPRAISALS ★ ★ ★ ★ ★ ★ ★ ★

Employees must view appraisals o f their performance as equitable if they are to accept the performance management system. But managers tend to vary in the level o f expectations they have for their employees. The perception that standards are uneven across managers can create significant dissatisfaction and the feeling that the playing field is not level. An additional concern is that uneven performance standards can cause a statistically significant adverse impact on protected classes related to pay and promotion.

Principles Underlying Effective Training Training managers is the most obvious strategy for gaining consistency, but most traditional approaches that focus on identifying various types o f rater error have been shown by research to be only moderately effective. The approach that has shown to be the most effective is using “frame of reference” training.

Frame o f reference training focuses on developing a consistent set of performance standards. Managers work in groups on a case that consists o f several completed appraisals that have commentary about job-related results and behaviors, but no ratings. During the training, individual managers independently arrive at their own ratings, factor by factor and overall. Their individual results are compiled and then presented to the group, which is tasked to come up with a common set o f ratings. Those whose ratings vary from the majority initially are invited to provide their rationale and through discussion the group arrives at consensus ratings. The dialogue among participants leads to a common frame o f reference as to what constitutes adequate performance, performance that significantly exceeds standards, and performance that does not fully meet standards. This consensus is used to develop rich definitions o f what performance at the various rating levels looks like. These standards then are used in the future to guide ratings.

W hen performance rating distributions across managers are first compared, it is often found that significant differences exist. A metropolitan utility that did a “dry run” of a new performance appraisal system found that the percentage of the 1,300 employees receiving “outstanding” ratings varied from one percent to 60 percent across departments. Even though it is likely that distributions will have some legitimate variation, due to the mix of the workforce in each unit, this large difference is also likely to be caused by a significandy different understanding of what rating levels are intended to mean. If these differences are not analyzed and addressed, the result will be inequitable ratings across departments and a resulting inequity in the administrative consequences. Employees are inherently suspicious that peers working for other managers are getting higher ratings for the same level o f performance, due to differences in the level o f expectations held by managers. This can erode satisfaction and can lead to legal action if pay or other outcomes are tied to appraisal ratings.

An A pplication o f Frame o f Reference Training A city with a population o f 50,000 in the western part of the Chicago metropolitan area implemented a frame of reference training program. The city of Elmhurst and Pontifex Consulting had worked together to develop a new classification and compensation system to ensure pay

J A N U A R Y 2 0 2 0 P U B L I C M A N A G E M E N T 1 7

administration was equitable and competitive. The city then created a new perform ance appraisal format th at would be used for all employees. M anagem ent was com m itted to ensure th at equivalent perform ance standards were being used by all managers, so th at appraisals were equitable across departm ents. Employees n o t covered by a collective bargaining contract had their pay actions tied to perform ance appraisal ratings, thereby making equity even more im portant.

Elm hurst m anagem ent decided on using a three- level rating system, using the following descriptors:

It is com mon to have five rating levels, which necessitates having an “O utstanding” and an “Unacceptable” rating. However, some believe that unacceptable performance should be dealt with during the year and result in performance improvement plans that should either correct the deficient performance or initiate administrative action if it does n o t improve. If an Outstanding rating is used, it can be described as: “Individual regularly made exceptional contributions that had a significant and positive impact on the performance o f the u n it/ organization. Results consistently exceeded

performance standards and objectives significantly over a sustained period. Employee has mastered all job-related skills and possesses a broad range of capabilities. Provides a model for excellence and helps others to do their jobs better.”

If a three-level approach is used, the organization should ensure that an employee performing consistently in an outstanding m anner is acknowledged in other ways, in order to communicate that extraordinary contributions are valued. In the private sector, cash awards may be used, b u t in public sector organizations, this is less common. The culture o f the organization will have an

impact on the approach used to reward those contributing at the highest level. And some jobs are controlled by set procedures, making it difficult for an employee to produce results that necessitate an “O utstanding” rating.

The Training Process A two-part training program was developed. Part one consisted in a series of group sessions with managers and employees in attendance at the same sessions. The principles of sound performance management were presented, and open discussion was encouraged. The decision to have all types o f employees in

★ RATING DESCRIPTION

Significantly Exceeds Standards S ig n ific a n tly exceeds p e rfo rm a n c e sta n d a rd s and e x p e c ta tio n s on a re g u la r basis. C o n trib u te s to th e e ffe c tiv e n e s s o f th e u n it and fe llo w e m p loyees. Is fu lly c o m p e te n t in all aspects o f th e jo b and has a ch ieved m a s te ry in som e. O fte n goes b e yo n d w h a t is re q u ire d to ensure w o rk is o f th e h ig h e s t q u a lity . Is h ig h ly p ro d u c tiv e .

Fully m eets p e rfo rm a n c e standards and e x p e c ta tio n s on a re gular basis. S om etim es makes c o n trib u tio n s th a t p o s itiv e ly im p a c t th e e ffe c tiv e n e s s o f th e u n it and o th e r em ployees. Fully

c o m p e te n t in all aspects o f th e jo b .

Does N ot Fully M eet Does n o t co n s is te n tly m e e t pe rfo rm a n ce standards and e xp e cta tio n s. Results and behaviors Standards som etim es are u n sa tisfa cto ry. M ay be able to p e rfo rm m o st aspects o f th e jo b b u t m ay lack

th e kn o w le d g e and skills to p e rfo rm others. P erform ance im p ro v e m e n t is needed.

1 8 P U B L I C M A N A G E M E N T J A N U A R Y 2 0 2 0

Managers should be informed that if their actual rating distributions

vary significantly from the guidelines, supporting evidence will be

needed to support them.

the sessions together was based on the desire to openly discuss the challenges associated with performance appraisal and to ensure everyone knew that they had all heard the same thing. Organizations often train managers and employees separately, leaving the door open to the suspicion that not everyone heard the same thing.

Once the fundamentals training was completed, the consultants and H R met with department heads and their key staff members to begin work on part two o f the training. The Police, Fire, Public Works, and Administrative department directors were each provided with an explanation o f the frame o f reference approach and were asked to develop a set o f dummy appraisals that could be used in the training. The appraisals were written with com mentary provided on job- related results and behaviors b u t no ratings. They consisted o f examples o f performance at all levels, which would require participants to decide how an overall rating would be arrived at. Once the models were completed for each department, they were distributed to the members o f the m anagement/ com mand structure (everyone

who would be doing appraisals). The managers rated each o f the model appraisals based on the commentary provided for each rating factor and for the overall rating. The results were used in the frame of reference dialogue sessions with all managers o f each department participating in the same session. The objective for these sessions was to establish a com mon understanding of how each of the rating levels should be interpreted and to ensure that the descriptions of what each level o f rating meant on the appraisal form were adequate to promote consistency across raters. The appropriate approach to combining individual factor ratings to produce an overall rating was then agreed upon.

In order to ensure the departments were aligned with each other, a review session with the city manager and department heads participating was then held. Definitions o f each level of performance were developed through

dialogue. It was agreed that forced distribution o f ratings should not be imposed, since a different mix o f employees performing at the various levels is expected across smaller units. Forced distributions are arbitrary and have not held up in legal proceedings. O n the other hand, expected distributions m aybe communicated as guidelines.

Managers should be informed that if their actual rating distributions vary significantly from the guidelines, supporting evidence will be needed to support them. A further step that can improve alignment is the use o f “calibration sessions” at each level. Supervisors meet with their manager as a group and align their ratings; managers meet with their director to align their ratings; and so on. During these alignment sessions, everyone has an opportunity to recommend refinements to the rating level descriptors to make them as clear and consistent as possible.

C o n c lu s io n

Each organization must concern itself with internal equity w hen performance standards are developed and then used to appraise performance. Individuals vary in their interpretation o f what constitutes adequate, superior, and poor performance, and in order to minimize the inconsistency across raters, steps must be taken to moderate these differences.

Gaining acceptance by employees that ratings are fair

and accurate will always be a challenge. Cognitive bias leads to people believing they are better performers than they are. This form of bias cannot be completely corrected. Recognizing that the bias exists and discussing it with employees can help to alleviate some unwarranted dissatisfaction with ratings that are believed to be too low. People can accept to some degree the difference of opinion between raters and themselves, but suspicions that inequity exists across raters will erode acceptance. W hen inconsistency across raters is apparent, this reality (or perception) elicits strong feelings of inequity, but organizations can take steps to minimize the amount of distortion it causes. Frame of reference training has been shown by research to be the most effective way to do that. Pi/1

JAMES GRABOWSKI is city manager, Elmhurst, Illinois (james.grabowski@ elmhurst.org).

ROBERT J. GREENE, PHD, is consulting principal, Pontifex, (RewardSystems® sbcglobal.net).

PETER RONZA is CEO, Pontifex, (pronza@ pontifex-hr.com).

VALERIE JOHNSON is human resources director, city o f Elmhurst, Illinois (valeriejohnson@ elmhurst.gov).

LYNN LOHMAN is human

of Elmhurst. Illinois (lynn. [email protected]).

J A N U A R Y 2 0 2 0 | P U B L I C M A N A G E M E N T 1 9

Copyright of Public Management (00333611) is the property of International City/County Management Association and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.