presentation
annyzyl123
Week6-RaterIssuesinPMVanessa1.pptxRATER ISSUES IN PERFORMANCE MANAGEMENT
Vanessa Beckles
Psychology 601
Performance Assessment
1
Overview
Who Should Provide Performance Information
Rater Errors Motivation
Rater Motivation to Inflate and Deflate Ratings
Prevent Rater Distortion
Reasons for Rater Training Programs
Rater Error Training
Case Study
Criteria for Evaluating Errors
Article 1
Article 2
Article 3
Summary Conclusion
2
Who Should Provide Performance Information
Who Should Provide Performance Information
Supervisors
Peers
Subordinates
Self
Customers
Handling Disagreement across sources
Ratings may not be similar due to different levels of engagement with employee
3
Rater Errors Motivation
Performance ratings may be intentionally or unintentionally distorted or inaccurate
Raters behavior are influenced by
The motivation to provide accurate ratings
Rater expects positive or negative consequences
The probability of receiving these rewards and punishments will be high if accurate ratings are provided
The motivation to distort ratings
Rater expects any positive or negative consequences
The probability of experiencing such consequences if ratings are indeed distorted
4
Rater Motivation to Inflate and Deflate Ratings
Reasons supervisors may inflate ratings
Maximize the merit raise/rewards
Encourage employees
Avoid creating a written record
Avoid confrontation with employees or jeopardize relationship
Uncomfortable talking about weaknesses
Make the manager look good to his/her supervisor
Reasons to provide artificially deflated ratings
Shock employee
Teach a rebellious employee a lesson
Responsible for too many people to evaluate them accurately
Build a strongly documented, written record of poor performance
Failure to remember accurate performance behaviors
5
Prevent Rater Distortion
How to prevent conscious distortion of ratings
Provide incentives so raters have more to gain than to lose by giving accurate ratings
Accountability in the system of rating
Overly lenient appraisals are challenged
Raters justify their ratings
Raters justify their ratings in face-to-face meeting
Training Programs
Improve skills in evaluating performance
6
Reasons for Rater Training Programs
Training programs to address intentional and unintentional rater errors in appraisals should include:
Reasons for implementing the PM system
How to identify and rank job activities
How to observe, record, and measure performance
How to minimize rating errors
How to conduct an appraisal interview
How to train, counsel and coach
7
Rater Error Training
Primary Purpose
Conscious awareness of possible errors; discriminating among raters (differential elevation)
Content Training
Focus on leniency, halo, range of restriction
Liability of Approach
May use inappropriate response sets may be a “ rating effect” can reduce the rating effect but may also lower accuracy
Primary Value
Use to make decisions that require distinguishing accurately among employees
Rater Error Training (RET) most useful when the primary goal is to distinguish accurately among workers
London, M. Mone, E. M., & Scott, J.C. (2004)
8
Case Study: Our Civil Service
At the State Employment Service, a number of employment counselors were hired together during a special recruiting effort 12 years ago in 2000. They formed a cohort, went through training together, and received graduate hours in vocational counseling together.
About a year ago, Jane Midland, the first member of the cohort to get promoted, tested into a supervisory position at one of the Job Service Centers. Two of the eleven employees who report to her are members of the 2000 cohort. Barb Rick and George Malloy deeply respect her abilities and have a strong affection for her. In fact, Barb Rick has spent time at Jane’s home watching their children play together and helping with the remodel of Jane’s house. George, Jane, and Barb get together for lunch regularly. Recently, they have considered attending evening classes together to get a master’s degree in Human Resource Management.
Yesterday, Jane received a memo from management reminding her that it is time to complete the annual appraisal forms for her staff.
Discuss the factors that may cause Jane to intentionally and unintentionally distort her ratings of Barb and George.
Evaluate the kinds of training programs that could help minimize the factors you have described. What do you recommend and why?
Aguinis, (2013)
9
Criteria for Evaluating Errors
Psychometric
Indirect measures
Rater Error
Most common
Indirect measures
Rating Accuracy
Direct measures
Rare
Usually in a laboratory
Murphy, K. R. & Cleveland, J.N. (1995).
10
Article One
Explaining the Weak Relationship Between Job Performance and Ratings of Job Performance
Murphy, K. R. (2008)
11
Overview
Ways of Improving Performance Appraisals
Researchers and Practitioners regard performance ratings as the Rodney Dangerfield of HRM
“Rarely do they get much respect”
Some argue that they should be banned entirely
Survival of performance appraisals is primarily due to limited alternatives
Three General Models of the Relationship Between Job Performance and Performance Ratings
12
Improving Performance Appraisals
Researchers/Practitioners
Behavioral anchors
Identifying specific types of rating errors
Leniency error, halo error
Rater Training
Focus on improving quality of ratings versus avoiding specific errors
360 degree evaluations
Peer, supervisor, subordinates, and clients
Organizational Strategies
Forced Distribution Systems
Identify weakest performers
Called “rank and yank”
Group and Discussion Review systems
Require raters to compare, discuss, and justify evaluations
Both help calibrate raters and discourage unrealistically lenient or harsh ratings
Raters vulnerable to social influences
13
Three General Models
Numerous models in Organizations for performance-performance rating relationships
One Factor Models
Multi-Factor Models
Mediated Models
14
One Factor Models
Most Popular
Offer few explanations for improving raters’ limited ability to evaluate subordinates.
Multi-Factor Models
Useful starting point for improving appraisals
Weakness between rater and performance is not entirely the fault of the rater
Situational constraints a factor
Influence of nonperformance
Mediated Models
Engage raters as willing and motivated partners
15
Article Two
The Impact of Non-Performance Information on Ratings of Job Performance: A Policy-Capturing Approach
Spence, J. R., & Keeping, L. M. (2010)
16
Overview
Policy–capturing approach to performance appraisals
Background
Motives for Rating Distortion
Personality
17
Policy-Capturing Approach
“Scenario based research methodology that allows researchers to determine how individuals utilize various pieces of information to arrive at judgments”
Spence, J. R., & Keeping, L. M. (2010)
18
Background
People have difficulty rating other human beings
Challenge for managers to transition from being leaders to being judicial evaluators
Fear of repercussions
Three most commonly discussed reasons for non-performance rating distortion
Avoidance of negative consequences of ratings
Organizational norms
Opportunity to advance self-Interest
19
Motives for Rater Distortion
Avoidance of negative consequences
Raters may intentionally manipulate ratings to avoid uncomfortable situations
Connection with lenient appraisals
Avoid negative feedback
More lenient when giving face-to-face feedback
Potentially damaging interpersonal relationships
Alter performance ratings as preventive behavior for conflict
20
Motives for Rater Distortion
Organizational Norms
Reflect behavior that is acceptable in the workplace
Raters are influenced by the norms within an organization
Ratings may be altered by what managers perceive to be permissible behaviors
Relationship between rating accuracy and rating error
Situational factors such as organizational norms can influence raters motivations to rate accurately or inaccurately
21
Motives for Rater Distortion
Self-Interest
Managers distort ratings to make themselves look competent or receive incentives
Inflate employees ratings to gain resources or to gain favor with the leaders
Some managers use employee’s rating to self-enhance the perception of the department
Desire to manage impressions
22
Personality
Two areas based on the Five Factor Model of personality traits may provide some explanation for rater accuracy and inaccuracy
Conscientiousness
Less elevated ratings
Strongly controlled by long term performance
More focused on the big picture goal more accurate ratings
Agreeableness
Managers high in agreeableness may produce more lenient appraisals
Produce more elevated ratings
23
Article Three
Using a Frame-of-Reference Training to Understand the Implications of Rater Idiosyncrasy for Rating Accuracy
Uggerslev, K. L. & Sulsky, L. M. (2008)
24
Overview
What is Frame-of-Reference (FOR) Training?
Two levels of theories
Performance Theories and Rater Idiosyncrasy
Performance Theories Idiosyncrasy and Rating Accuracy
25
Frame-of-Reference Training
Primary Purpose
Develop shared performance schema with organization; discriminating among dimensions of performance (stereotype accuracy) and discriminating among ratees within organizations (differential accuracy)
Content of Training
Examples of normative (poor, average, good) behaviors for behavioral dimensions
Liability of Approach
Does not reduce rating effect
Primary Value
Used to make decisions that require comparing employees on different performance dimensions—e.g. job assignments and placements, and feedback for development and goal setting
London, M. Mone, E. M., & Scott, J. C. (2004)
26
Performance Theory and Rater Idiosyncrasy
Levels of Theory Idiosyncrasy
The differences between raters’ implicitly held theories, and the normative performance theory imparted during FOR training
Two forms of rater idiosyncrasy
Performance Dimensions
Performance-Related Behaviors
Both contain errors of omission and commissions
27
Performance Theory Idiosyncrasy and Rating Accuracy
Rater accuracy before FOR training:
Hypothesis 1: Prior to training, the more dimensions in performance evaluation the less accurate appraisals will be
Hypothesis 2: There should be better results for raters who had less dimensions (omission) than the organization for evaluation than the raters who had more dimensions (commission) than the organization
Hypothesis 3: Training will improve rating accuracy for all FOR trainers, such that trainees with relatively high performance theory idiosyncrasy will improve significantly more than trainees with lower idiosyncrasy.
28
Results
Hypothesis 1
Supported: Idiosyncratic raters have the most to gain and that there was negative relationship between idiosyncrasy and rating accuracy prior to raters’ receiving training
Hypothesis 2
Somewhat supported: Relationship between idiosyncrasy and rating accuracy may depend on both the degree of idiosyncrasy and the form of idiosyncrasy
Hypothesis 3
Somewhat supported: Most trainees were at least mildly idiosyncratic with each aspect of the performance theory
Idiosyncrasy“: extra or less dimensions than the organization for performance evaluating performance. Different that what is perceived to be normal evaluative or organization guidelines
29
Summary
Who Should Provide Performance Information: peers, customers, subordinates, and you
Discussed reasons for rater errors motivation
Several reasons was presented for rater motivation to inflate and deflate ratings
Presented ways to prevent rater distortion
Provided several reasons for rater training programs
One example of rater error training
Analyzed a case study
Covered different types of criteria for evaluating errors
Article 1
Article 2
Article 3
30
References
Aguinis, H. (2013). Performance management. Indiana: Pearson
London, M., Mone, E.M., & Scott, J.C. (2004). Performance management and assessment: Methods for improved rater accuracy and employee goal setting. Human Resource Management, 43, 319-336.
Murphy, K.R. (2008). Explaining the weak relationship between job performance and ratings of job performance. Industrial and Organizational Psychology, 1, 148-160.
Murphy, K.R., & Cleveland, J.N. (1995). Understanding Performance Appraisal: Social, Organizational, and Goal-Based Perspectives. Chapter 10: Error and accuracy measures. Thousand Oaks, CA: Sage.
Spence, J. R., & Keeping, L. M.,(2010). The impact of non-performance information on ratings of job performance: A policy-capturing approach. Journal of Organizational Behavior, 31, 587-608.
Uggerslev,K.L., & Sulsky, L. M., (2008). Using frame-of-reference training to understand the implications of rater idiosyncrasy for rating accuracy. Journal of Applied Psychology, 3, 711-719.
31
