presentation
annyzyl123
Week6-RaterissuesMichael.pptxRater Issues in Performance Management
Michael Rose
Psychology 601
Overview
Possible sources of performance information
Rater Motivation
Rater Training Programs
Case Study 6-4
Frame-of-Reference training
Article 1
Article 2
Article 3
Summary
Possible Sources of Performance Information (Raters)
Possible Sources
Supervisors
Peers
Subordinates
Self
Customers
Disagreements among raters
Not necessarily a problem
Behavioral indicators may vary across sources.
Important to define the target behavior clearly for all raters.
If disagreements are found, the importance of each source must be determined.
Rater Error Motivation
Raters may intentionally or unintentionally distort ratings.
Raters may be motivated to inflate or deflate ratings.
Motivation to provide accurate ratings.
Rater expects certain positive or negative consequences.
Probability of receiving rewards will be high if they provide accurate ratings.
Motivation to distort ratings.
Rater expects certain positive or negative consequences.
Probability of receiving rewards will be high if they distort ratings.
Motivations to Inflate or Deflate Ratings
Motivations for inflated ratings
Motivations for deflated Ratings
Maximize the merit raise/rewards
Encourage Employees
Avoid creating a written record
Avoid confrontation with employees
Promote undesired employees out of unit
Make the manager look good to his/her supervisor
Shock an employee
Teach a rebellious employee a lesson
Send a message to the employee that he/she should consider leaving
Build written record of employees poor performance
Preventing Conscious Distortion
Convince raters that they have more to gain by providing accurate ratings.
Increase accountability.
Have raters justify their ratings
Have raters justify their ratings face-to-face
Provide rater training
Rater Training programs
May cover the following topics:
Reasons for implementing the performance management system.
How to identify and rank job activities.
How to observe, record, and measure performance.
Information on the appraisal form and system mechanics.
How to minimize rating errors.
How to conduct an appraisal interview.
How to train, counsel, and coach.
Case Study 6-4
Provide a detailed discussion of the intentional and unintentional rating distortion factors that may come into play in this situation.
Evaluate the kinds of training programs that could minimize the factors you have described. What do you recommend and why?
Frame-of-Reference Training
Improves rater accuracy by familiarizing raters with the performance dimensions to be assessed.
Typically involves:
Discussion of the job description for the individual being rated.
Review of the definition for each dimension to be rated.
Discussion of examples of good, average, and poor performance.
Trainees rate fictitious employees.
Trainees informed of correct ratings for each dimension.
Article 1
Ratings of counterproductive performance: the effect of source and rater behavior.
Mann, S. L., Budworth, M., & Ismaila, A. S., (2012)
Purpose
To examine whether there is inter-rater agreement on counterproductive performance between self and peer-ratings.
To examine factors that moderate inter-rater agreement.
Factors examined include: self reported levels of counterproductive behaviors, conscientiousness, and integrity.
Hypotheses
Hypothesis 1: Peer-ratings of counterproductive performance are significantly higher than self-ratings of counterproductive performance.
Hypothesis 2 : Conscientiousness moderates the relationship between rating source and rater agreement such that individuals with similar levels of conscientiousness demonstrate agreement for self and peer-ratings of counterproductive behaviors.
Hypothesis 3: Values toward integrity moderates the relationship between rating sources and rater agreement such that individuals with similar levels of integrity demonstrate agreement for self and peer-ratings of counterproductive behavior.
Hypothesis 4: Individuals who exhibit similar levels of counterproductive performance, as rated by their peers, demonstrate agreement for self and peer-ratings of counterproductive behaviors.
Results
Hypothesis 1: Supported. Peer-ratings (m = 2.1) were significantly higher than self-ratings (m = 1.4).
Hypothesis 2: Not supported. Conscientiousness was not a significant moderator of the relationship between rating source and rater agreement.
Hypothesis 3: Not supported. Integrity was not a significant moderator of the relationship between rating sources and rater agreement.
Hypothesis 4: Supported. Individuals who exhibit similar levels of counterproductive performance, as rated by their peers, are more likely to agree on ratings of counterproductive performance. Estimated effect = 0.39 (p < 0.001).
Practical Implications
Individual differences between the rater and the individual being rated may have a significant impact in an organizational settings.
Provides support for 360 feedback on counterproductive performance, as sources were shown to provide unique feedback.
Understanding peer ratings is important due to the increased number of teams in the workplace.
Article 2
Rater personality and dimensions weighting in making overall performance judgments.
Ogunfowora, B., Bourage, J., (2010).
Purpose
Examined the effects of rater personality on the performance appraisal process.
Specifically, the influence of rater personality on the relative weights which raters placed on different performance dimensions was investigated.
Honesty-humility, openness
Hypotheses
Hypothesis 1: Rater honesty-humility will positively relate to weights which are placed on items associated with maintaining personal discipline.
Hypothesis 2: Rater openness will positively relate to weights which are placed on items associated with adaptive performance.
Results
Hypothesis 1: Not directly supported, as higher levels of honesty-humility did not (p > .05) relate to increased weights on personal discipline.
Though modesty was positively related to personal discipline (p < .01).
Hypothesis 2: Supported. Raters higher in openness weighed adaptive performance significantly higher than those lower in openness (p < . 01).
Practical Implications
Indicates that organizations must communicate a standard theory of performance to their employees.
Organizations must account for systematic differences in raters. (ex: Supervisors likely to be systematically different than other sources in openness. Expect systematically different ratings.)
Supports the use of frame-of-reference training.
Article 3
Rater training revisited: An updated meta-analytic review of frame of reference training.
Roch, S. G., Woehr, D. J., Mishra, V., & Kieszczynska, U. (2012).
Purpose
To demonstrate that not all measures of accuracy are equally improved by frame-of-reference training (FOR).
To investigate how much FOR training protocols differ.
Findings/Implications
FOR does not impact all measures of accuracy equally.
Best for training raters to recognize patterns of performance.
Therefore, improved the raters ability to rank order the employees who they were rating.
Provides support for FOR training as an effective rater training method.
Overview
Presented possible sources of performance information
Identified various rater motivations.
Made suggestions on how to overcome intentional or unintentional distortions.
Completed case study 6-4
Introduced Frame-of Reference training
Article 1: Difference between sources.
Article 2: Supported a standard theory of performance (FOR training).
Article 3: Identified situations which are most impacted by FOR training.
References
Aguinis, H. (2013). Performance Management. Indiana: Pearson
Mann, S. L., Budworth, M., & Ismaila, A. S. (2012). Ratings of counterproductive performance: the effect of source and rater behavior. International Journal of Productivity and Performance Management, 61, 142-156.
Ogunfowora, B., Bourdage, J., & Lee, K. (2010). Rater personality and performance dimension weighting in making overall performance judgments. Journal of Business and Psychology, 25, 465-476.
Roch, S. G., Woehr, D. J., Mishra, V., & Kieszczynska, U. (2012). Rater training revisited: An updated meta-analytic review of frame-of-reference training. Journal of Occupational and Organizational Psychology, 85, 370-395.
