Powerpoint

November 2020 / ST R AT E G I C F I N A N C E / 21

One of the most com- mon challenges associated with performance ap - praisals in large organiza- tions is a lack of con - sistency in ratings given by different supervisors. When supervisors use different standards to rate employees, they may end up giving different ratings for the same level of performance. For exam- ple, consider Barbara, who works for Marc. Marc assesses Barbara’s overall performance on a 1-5 scale as a 4, because Marc be lieves her work “exceeded expectations” on the rating scale. But had Barbara been working for Michelle, Michelle might have felt that Barbara merely “met expectations” and rated her as a 3. On the other hand, had she been evaluated by Bob, Barbara would have been rated as “exceptional,” or a 5 on the rating scale, which could have made her eligi- ble for a higher bonus, a promotion, and training not available to employees with ratings of 3 or 4.

When different super- visors apply different stan- dards to assess employee performance, how can organizations really know which employees are per- forming exceptionally well and which are performing poorly? Employee dissatis- faction can result from such inconsistencies and can negatively affect an organization’s ability to develop human capital and to reward and retain high- performing employees.

THE CALIBRATION PROCESS

Higher-level managers are in a position to see the rat- ings that lower-level

RESEARCH

EMPLOYEE APPRAISALS AND THE CALIBRATION PROCESS Involving higher-level managers in employees’ annual performance appraisals can help avoid bias and favoritism while improving consistency and increasing employee satisfaction. BY WILL DEMERÉ, PH.D.; KAREN SEDATOLE, PH.D.; AND ALEXANDER WOODS, PH.D.

 A NNUAL PERFORMANCE APPRAISALS tend to receive a lot of negative press. Critics grumble that they aren’t timely, don’t provide sufficiently detailed feed back to improve per- formance, and can ultimately reduce employee motivation. Some of the most damaging com-

plaints come from employees who feel that their supervi- sor is unfair or biased. For example, supervisors may only recall employee performance on the most recent project, show favoritism toward some employees over others, exhibit some form of discrimination, or only focus on one aspect of the employee’s job.

supervisors assign to employees and can iden- tify inconsistencies. Thus, they can balance out the ratings of excessively lenient or tough supervi- sors so that consistent standards are applied and employees are assessed similarly regardless of who is actually rating them.

This is often called a calibration process. After lower-level supervisors rate their subordinates, the ratings are further evalu- ated by a calibration com- mittee composed of higher-level managers. The purpose of a calibration committee is to review employee ratings and adjust as needed to “cali- brate” the ratings. Surveys indicate that many organi- zations use calibration committees in their annual performance appraisal process to overcome supervisor rating biases.

INSIGHTS INTO CALIBRATION

We collaborated with a multinational organization to study its performance appraisal system and cali- bration process over a three-year period. The per- formance appraisal process started with supervisors determining initial em - ployee ratings. Similar cat- egories of employees were grouped into the same bonus pool, and a separate calibration committee reviewed the ratings for each group of employees. The calibration committees were generally composed of managers who were one level higher than the supervisors that assigned the initial ratings.

The committees came to a shared understanding of what constituted appro-

priate performance for each rating level and then reviewed the ratings for all employees in the pool. If the committees felt that specific ratings were too high or too low, they adjusted the initial ratings, resulting in a final, cali- brated rating for each employee in the pool. Supervisors were then free to share the final ratings with employees.

In the published study (“The Role of Calibration Committees in Subjective Performance Evaluation Systems,” Management Science, April 2019), we explored and discovered many interesting realities of the calibration process. For example, the commit- tees adjusted about one of every four ratings, reflect- ing instances in which they felt the ratings were too high or too low. This also means they accepted the supervisor’s assigned rating in 75% of the cases, recog- nizing that supervisors had the most direct knowledge of the employees’ perform- ance. When the committee did adjust ratings, it ad - justed the ratings down- ward about 80% of the time, suggesting an overall tendency of supervisors to give lenient ratings.

The committees made adjustments in a manner consistent with removing bias from the initial ratings and promoting greater con- sistency in ratings across supervisors. Specifically, supervisors who tended to issue higher-than-average ratings were more likely to have the ratings they assigned adjusted down- ward, while supervisors who tended to issue lower- than-average ratings were more likely to have the rat- ings they assigned adjusted

upward. Overall, we found that employee evaluations were more consistent after the calibration committees reviewed and adjusted the employee performance ratings.

We examined whether supervisors subsequently gave a higher rating to an employee whose previous rating had been adjusted upward and a lower rating to an employee whose pre- vious rating had been adjusted downward. This was generally the case. Supervisors did tend to respond, suggesting they learned from the calibra- tion process over time.

EMPLOYEE SATISFACTION

We also surveyed the orga- nization’s employees about the performance appraisal process. In general, higher- performing employees tended to think the system was fair, didn’t think favoritism was an issue, and were satisfied with the system. On the other hand, lower-performing employ- ees thought the system was unfair, believed favoritism was present, and were dis- satisfied with the system.

The organization inter- preted this finding as evi- dence that the system was working as intended because a main objective of the system was to retain and reward the highest- performing employees. This pattern provided evi- dence that the highest- performing workers were indeed the ones reaping the system’s benefits and were most likely to be sat- isfied, stay with the organ- ization, and continue to perform well.

Despite these benefits, the calibration process

wasn’t perfect. Because the committees focused more on adjusting ratings down- ward that they felt were too high, the final ratings tended to be more com- pressed around average performance levels, so employees were less differ- entiated from one another. This can make it more chal- lenging to identify high- performing employees for promotion and reward.

Overall, our study found the use of calibra- tion committees could effectively remove bias from supervisors’ initial subjective appraisals and make employee ratings more consistent across dif- ferent supervisors. The result was that employees, especially the higher- performing ones, were generally satisfied with the appraisal system and believed it was fair, which can positively influence future motivation and per- formance. In addition, the calibration process ap - peared to be a better way to reward knowledge workers dispersed throughout the world, whose performance is notoriously difficult to objectively measure. SF

Will Demeré, Ph.D., is an assistant professor at the Univer- sity of Missouri’s Trulaske College of Business. He can be reached at [email protected].

Karen Sedatole, Ph.D., is the interim dean and a professor at Emory University’s Goizueta Busi- ness School. She can be reached at [email protected].

Alexander Woods, Ph.D., is an associate professor at William & Mary’s Mason School of Busi- ness. He can be reached at [email protected].

RESEARCH

22 / ST R AT E G I C F I N A N C E / November 2020

Copyright of Strategic Finance is the property of Institute of Management Accountants and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.