Quality Management Programs
Chapter 5
Measuring Results
and
Behaviors
5-1
Copyright © 2019 Chicago Business Press
Key Questions
Where should each individual focus their efforts?
What are the expected standards?
How do we know how well the results were achieved?
5-2
Copyright © 2019 Chicago Business Press
Measuring Results: Overview
Accountabilities
Objectives
Performance Standards
5-3
Copyright © 2019 Chicago Business Press
Accountabilities
Broad areas of a job for which an employee is responsible for producing results
5-4
Copyright © 2019 Chicago Business Press
Determining Accountabilities
Collect information about the job (Job Description)
Determine importance of task or cluster of tasks
Percentage of employee’s time spent performing tasks
Impact on the unit’s mission if performed inadequately
Consequences of error
5-5
Copyright © 2019 Chicago Business Press
Objectives
Statements of important and measurable outcomes, that, when accomplished, will help ensure success for the accountability
5-6
Copyright © 2019 Chicago Business Press
Determining Objectives
Purpose is to identify outcomes:
Limited in number
Highly important
Achieving them will have a dramatic impact on overall organization success
5-7
Copyright © 2019 Chicago Business Press
Characteristics of Good Objectives
5-8
Copyright © 2019 Chicago Business Press
What are the benefits of using SMART objectives?
Choose a short-term personal or professional goal and frame the objective using the SMART technique
5-9
Copyright © 2019 Chicago Business Press
SMART Objectives
Performance Standards
Yardstick used to evaluate how well employees have achieved objectives
Information about what to look for to determine the level of performance that has been achieved
5-10
Copyright © 2019 Chicago Business Press
Performance Standards
Aspects of performance objectives:
Quality
How well the objective is achieved
Quantity
How much, how many, how often, and at what cost?
Time
Due dates, schedule, cycle times, and how quickly?
5-11
Copyright © 2019 Chicago Business Press
Standards Must Include
An Action
The Desired Result
A Due Date
A Quality or Quantity Indicator
5-12
Copyright © 2019 Chicago Business Press
Good Performance Standards
5-13
Copyright © 2019 Chicago Business Press
Measuring Behaviors: Overview
Identify Competencies
Identify Indicators
Choose Measurement System
5-14
Copyright © 2019 Chicago Business Press
Identify Competencies
Measurable clusters of KSAs
Knowledge
Skills
Abilities
Critical in determining how results will be achieved
5-15
Copyright © 2019 Chicago Business Press
Company Spotlight
Xerox Capital Services (XCS) used leadership competencies to create leadership development program
Steps followed:
Senior managers and leaders identified the 12 most critical competencies for success at XCS
12-week curriculum developed with readings and course material
5-16
Copyright © 2019 Chicago Business Press
Types of Competencies
Differentiating
Distinguish between superior and average performance
Threshold
Needed to perform to minimum standard
5-17
Copyright © 2019 Chicago Business Press
Identify Indicators
Observable behaviors
Used to measure the extent to which competencies are present or not
5-18
Copyright © 2019 Chicago Business Press
Necessary Components for Describing Competencies
Definition of competency
Description of specific behaviors
When competency is demonstrated
When competency is not demonstrated
Suggestions for developing the competency
5-19
Copyright © 2019 Chicago Business Press
Choose a Measurement System
Comparative system
Compares employees with one another
Absolute system
Compares employees with pre-specified performance standards
5-20
Copyright © 2019 Chicago Business Press
Choose a Measurement System
5-21
Copyright © 2019 Chicago Business Press
Types of Comparative Systems
Simple rank order
Alternation rank order
Paired comparisons
Relative percentile
Forced distribution
5-22
Copyright © 2019 Chicago Business Press
Simple Rank Order
Advantages:
Simple and easy to do
Results are clear
Disadvantages:
Judges performance based on one dimension only
May be difficult to rank similar performance levels
5-23
Copyright © 2019 Chicago Business Press
Alternation Rank Order
Advantages:
Simple and easy to do
Results are clear
Uses two anchors (best and worst)
Disadvantages:
Judges performance based on one dimension only
May be difficult to rank similar performance levels
Does not specify threshold for acceptable performance
5-24
Copyright © 2019 Chicago Business Press
Paired Comparisons
Advantages:
Thorough
Final rankings are more accurate
Disadvantages:
Very time consuming
May encounter problem of comparing “apples and oranges”
5-25
Copyright © 2019 Chicago Business Press
Relative Percentile
Advantages:
Simple and easy to use
Evaluates specific competencies or overall performance
Disadvantages:
May be difficult to consider all ratees at the same time
Time consuming if using several scales for different competencies
5-26
Copyright © 2019 Chicago Business Press
Relative Percentile
5-27
Copyright © 2019 Chicago Business Press
Example of Relative Percentile Method Scale
Don
Desiree
Heather
Forced Distribution
Advantages:
Categorizes employees into specific performance groups
Facilitates reward assessment
Competition may be good for organizational performance
Disadvantages:
Assumes performance scores are normally distributed
May discourage contextual performance and teamwork
5-28
Copyright © 2019 Chicago Business Press
Company Spotlight
General Electric (GE) has moved from forced distribution ratings to a more performance management approach
Changes include:
An app-based system to provide ongoing coaching
Evaluating employees relative to desired traits
5-29
Copyright © 2019 Chicago Business Press
Why might forced distributions demotivate employees?
How have forced distributions evolved over the years?
5-30
Copyright © 2019 Chicago Business Press
Forced Distribution
Advantages of Comparative Systems
Easy to explain
Straightforward: Identifies top as well as underperformers
Better control for biases and errors found in absolute systems: Leniency, Severity, and Central tendency
Beneficial for jobs that are very autonomous
Attracts individuals high in cognitive abilities
5-31
Copyright © 2019 Chicago Business Press
Disadvantages of Comparative Systems
Rankings may not be specific enough for:
Useful feedback
Protection from legal challenge
No information on relative distance between employees
Specific issues with forced distribution method
5-32
Copyright © 2019 Chicago Business Press
Company Spotlight
Deloitte abandoned forced distribution ratings in favor of a system that provides feedback on a project or quarterly basis
Changes include:
Documented, weekly “check-ins” with employees
Supervisor training on how to evaluate performance
5-33
Copyright © 2019 Chicago Business Press
Nature of Performance Distribution
Forced distribution implies that performance is normally distributed
Recent research shows that, instead, performance is distributed following a heavy-tail
That is,
There are more star performers
Differences between top and average performers are much greater
5-34
Copyright © 2019 Chicago Business Press
Nature of Performance Distribution
What does the “myth of the bell curve” refer to?
Under what situations are traditional assumptions of performance more likely to hold?
5-35
Copyright © 2019 Chicago Business Press
Nature of Performance Distribution
5-36
Copyright © 2019 Chicago Business Press
Nature of Performance Distribution
Producing Star Performers
5-37
Copyright © 2019 Chicago Business Press
Identify and eliminate situational constraints
Allow star performers to rotate across teams
Invest sufficient resources in star performers
Attend to stars developmental network
Shield from financial challenges
Give them preferential treatment, but explain why to all workers
Invest disproportionate amount of resources in stars
Absolute Systems
Essays
Behavior checklists
Critical incidents
Graphic rating scales
5-38
Copyright © 2019 Chicago Business Press
Behavior Checklists
Advantages:
Easy to use and understand
Provides quantitative information
Widespread use
More objective than other systems
Disadvantages:
May feel impersonal and disconnected
Scale points used are often arbitrary
Difficult to get detailed and useful feedback
5-39
Copyright © 2019 Chicago Business Press
Behavior Checklists
5-40
Copyright © 2019 Chicago Business Press
Example of Behavior Checklist Item
Essays
Advantages:
Simplest absolute method
Individualized for each employee
Can be done anytime
Potential for detailed feedback
Disadvantages:
Unstructured and may lack detail
Depends on supervisor’s writing skill
Comparisons virtually impossible
Lack of quantitative information; difficult to use in personnel decisions
5-41
Copyright © 2019 Chicago Business Press
Critical Incidents
Advantages:
Focus on actual job behavior
Provides specific examples
Employees identify with rating
Disadvantages:
Collecting critical incidents can be very time consuming
Quantification is difficult
5-42
Copyright © 2019 Chicago Business Press
Critical Incidents
5-43
Copyright © 2019 Chicago Business Press
Example of Critical Incidents for Adaptability/Flexibility
Graphic Rating Scales
Advantages:
Meanings, interpretations, and dimensions being rated are clear
Useful and accurate
Most popular tool
Disadvantages:
Time consuming and resource-laden to develop
Lacks individualized feedback and recommendations
5-44
Copyright © 2019 Chicago Business Press
Graphic Rating Scales: BARS Improvement
Behaviorally Anchored Rating Scales (BARS)
Uses critical incidents as anchors
Involves multiple groups of employees in development
Identify important job elements
Describe critical incidents at various levels of performance
Check for inter-rater reliability
5-45
Copyright © 2019 Chicago Business Press
BARS Improvement
5-46
Copyright © 2019 Chicago Business Press
Example of BARS Improvement scale
Advantages and Disadvantages of Absolute Systems
Advantages:
Can be used in large and small organizations
Evaluations more widely accepted by employees
Disadvantages:
Higher risk of leniency, severity, and central tendency biases
Generally more time consuming than comparative systems
5-47
Copyright © 2019 Chicago Business Press
Role of Context
Plays important role in determining how performance is measured
Examples
Competitive versus Collaborative
Value of long-term relationships with customers
Industry Trends
Leadership
5-48
Copyright © 2019 Chicago Business Press
Measuring Performance
Several types of methods
Differ in terms of:
Practicality (time and effort)
Usefulness (quantifiable)
5-49
Copyright © 2019 Chicago Business Press
Quick Review
Measuring Results
Determine Accountabilities
Determine Objectives
Determine Performance Standards
Measuring Behaviors
Comparative Systems
Nature of the Performance Distribution
Absolute Systems
Role of Context
5-50
Copyright © 2019 Chicago Business Press
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher. Printed in the United States of America.
Copyright © 2019 Chicago Business Press
1-51
Copyright © 2019 Chicago Business Press
51