business idea

10-IRE347-EvaluationandCostBenefit_PreClass.pdf

Home >Business & Finance homework help >Economics homework help >business idea

IRE347 TRAINING & DEVELOPMENT Anny Fong M.Eng, MBA, LL.M, PMP

Aug 8, 2022

Agenda

¨ Recap from previous lecture ¨ Training Evaluation ¨ Cost/Benefit Analysis ¨ Case Discussion ¨ A look into the (near) future ¨ Group work

Questions from exit cards ¨ What are some signs of relapse? ¨ If we are looking to assess who is trainable (so that they can

attend training), would this lead to discrimination in the workplace?

¨ When parents go on parental leave, should they be retrained when they are back?

¨ How can trainees achieve skill mastery that remains constant over time?

¨ What happens when trainees are challenged with multitasking and fear change?

¨ What should employees do if they have challenges with the content after training has completed?

¨ What are some ways to communicate and solve problems re: organizational politics so that training can be successfully leveraged?

Quick recap of previous class

¨ Transfer of training – what is it? ¨ Transfer of training process ¨ Training inputs that impact the transfer of

training ¤ Trainee characteristics ¤ Training design ¤ Work environment

¨ Transfer of training strategies ¤ Before/during/after

¨ Transfer of training interventions ¨ Post-training supplements

First impressions of Training Evaluation and Cost/Benefit Analysis

¨ Kahoot!

¨ To participate, join at ¤ www.kahoot.it or Kahoot! App ¤ Enter the game PIN when it’s time

https://create.kahoot.it/details/51d23006-e938-412e-95e0-60ed9251eeff

http://www.kahoot.it/

Training Evaluation – At A Glance 6

Goals • Assess value of

training • To employees • To

organization

Methods • Continuum of

techniques and measures

• Appropriateness depends on context

Key Trade-off • Balance

informational needs with difficulty/cost of obtaining information

Image sources: The Noun Project: goal by Birckhead Creative, tools by anggun, give and take by Alfredo @ IconsAlfredo.com

Why should we care about training evaluation?

Improve training

Identify who would benefit the most

Determine benefits and cost-effectiveness

Determine if goals were met/problems were solved

Justify and reinforce value/credibility of training

Theory vs. Practice 8

¨ Of the organizations that do evaluate training, mainly simpler evaluation strategies are used

¨ When evaluation outcomes are more complex, fewer organizations conduct training evaluations

Theory Practice

Barriers to Training Evaluation 9

Practical •Knowledge of models, research

design, data analysis, etc. (could be intimidating)

•Time and effort required •Financial costs

Political •Evaluation can be threatening •Accountability •Investment implications

Image sources: The Noun Project: Practical by WEBTECHCHOPS LLP, political by Adrien Coquet

Types of Training Evaluation 10

Data collection

• Trainee perceptions • Work environment

Purpose

• Formative • Summative

Descriptive • Describe the trainee upon training completion

Causal • Did the training cause post-training behaviours?

Models of Training Evaluation 11

¨ Specifies what is being measured and their interrelationships

Kirkpatrick COMA Decision-

based evaluation

Kirkpatrick Hierarchical Model 12

What it is ¨ Each succeeding level

is more important (though more difficult to obtain)

¨ Each level has a causal link to next level

L5: ROI

L4: Results

L3: Behaviour

L2: Learning

L1: Reaction

M o

re im

p o

rt a

Ea sie

r to o

b ta

in What research says 🤔 Little correlation

between levels 🤔 Lacks precision

COMA Model 13

What it is ¨ Designed for formative evaluation ¨ Focus is on variables important for

transfer

In comparison ✓ Variables are causally related to

transfer of training ✓ Greater precision than

Kirkpatrick’s Model 🤔 Focuses exclusively on transfer 🤔 Not suited for summative

evaluation 🤔 Different questionnaires needed

for different training programs

ognitive

rganizational

otivational

ttitudes

Have trainees mastered the skill?

Does the environment support applying the skills?

Does the trainee want to apply the skills?

Do trainees feel capable of applying the skills?

Decision-Based Evaluation (DBE)

What it is ¨ Specifies that evaluations should

be guided by several key questions

In comparison ✓ Flexible, useful in summative and

formative evaluations 🤔 Relatively new model; still needs

more testing 🤔 Need a certain level of skill to

develop questions, collect data, and analyze data

• What do we want to find out?Target

• What are the variables?Focus

• What methods to use?Methods

Examples of Data Collection 15

What? Why? Approach Example

Reactions • Opinions • Attitudes

• Assess likes/dislikes • Assess perceived

usefulness

• Questionnaire • Focus groups • Interviews

How satisfied were you with the content of the training?

Learning • Declarative • Procedural

• Assess knowledge transfer of training

• Tests (MC, situation-based, etc.)

Which of the following steps are required to authenticate a caller?

Behaviour • Assess display of newly learned skills

• Self-reports • Observations • Production

indicators

What is the error rate after training?

Examples of Data Collection 16

What? Why? Approach Example

Motivation • Training • Transfer

• Desire to learn new skill

• Desire to apply new skill

• Questionnaire How important is it to check the filter before starting the maintenance routine?

Self-efficacy • Assess level of confidence in performance

• Questionnaire How confident are you in explaining the new product to the customer?

Perceived / anticipated support

• Assess trainees’ perception of support available to them

• Questionnaire My manager creates opportunities to apply the new skill (1 = always, 5 = never)

Examples of Data Collection 17

What? Why? Approach Example

Organizational perceptions

• Assess transfer climate

• Standardized questionnaires

• Transfer climate questionnaire (Rouiller & Goldstein)

Organizational results

• Assess impact of training on organization

• Organizational records (e.g. P&L)

• Error rates • Efficiency ratio

(operating costs) • # of items sold

A Few Approaches to Data Collection 18

¨ Not an exhaustive list

¨ Appropriate method depends on what you’re trying to measure ¤ Descriptive ¤ Causal

¨ Consider the ease of obtaining data as well

Costs and Benefits 19

¨ Cost-benefit analysis is a key part of training evaluation

¨ Since training is an investment, we need to be able to demonstrate the return on investment

¨ A few ways we’ll examine ¤ Net benefit ¤ Benefit-cost ratio (BCR) ¤ Return on investment (ROI)

Cost Categories 20

Direct

• Linked to a particular training program • Trainer's salary • Equipment • Course

materials

Indirect

• Support training activities but are not directly linked to a particular program • Administrative

support • Marketing

Developmental

• Cost to develop a training program • Needs analysis • Training

material • Evaluation

Overhead

• Incurred by training program, but not associated with a particular training program • General

operation of training function

Trainee compensation

• Salaries and benefits of employee attending training program • Salaries,

benefits • Replacement

costs while training

Benefits 21

Net benefit

• Benefit of training minus cost of training

Benefit-Cost Ratio (BCR)

• Benefit of training divided by cost of program

Return on Investment (ROI)

• (Benefit-cost) divided by cost

Example ¤ Benefit of training is $3,000 ¤ Training costs $1,800

$3,000 - $1,800 = $1,200 net benefit

$3,000/$1,800 = 1.67 BCR

($3,000-$1,000) /$1,000 = 2 We get a return of $2 for every $1 we spend on training

Longer-Term (?) Benefits 22

Example ¤ Benefit of training is $3,000/month (e.g. cost

avoidance) ¤ Training costs $15,000

¨ How long does it take to re-coup the costs?

($15,000 cost) / ($3,000 per month in savings) = 5 months

The payback period is 5 months.

How Do We Quantify Some of the Benefits? 23

¨ Productivity ¤ “We can now product 12 widgets/hour instead of

8 widgets/hour” → labour costs

¨ Quality ¤ “We now only need to re-work 1 in 10 widgets,

instead of 1 in 4 widgets” → labour costs, client complaints, regulatory fines

¨ Profit ¤ “We now can sell 1,000 more products” → revenue/FTE

¨ Etc.

Considerations for Cost-Benefit Analysis 24

¨ Not an exact science ¤ Assumptions and judgement are made when

estimating the monetary benefits ¤ Process only works if managers and clients

accept the assumptions ¨ Data must be accurate and process must be

believable ¨ Leverage internal and external experts

¤ Consider industry practices as well ¨ Sometimes there are “soft” benefits as well

¤ Customer satisfaction ¤ Competitive strategy

“DATAIN” Case

Breakout Discussion 1. For each option,

¤ What are the costs of each option? ¤ What are the benefits of each option? ¤ Which option has a faster payback? ¤ What are your assumptions with each

option? ¤ Which option do you recommend and why?

2. What are some practical challenges to realizing the benefits?

“DATAIN” Case One way to look at it

Summary 29

¨ A key trade-off in training evaluation is the balance between informational needs and difficulty/cost of obtaining information

¨ Results from training evaluation could inform how to improve training, who it would best benefit, illustrate benefits, determine if goals of training were met, and can justify the value of training

¨ There are practical and political barriers to training evaluation ¨ Models of training evaluation ¨ Methods of data collection ¨ Cost benefit analysis

¤ Types of costs ¤ Types of benefits ¤ Different ways to represent benefits

Exit cards! 30

¨ Before you leave today, please fill in the exit card ¤ Instead of an ‘attendance’ sign in ¤ Opportunity to reflect on today’s class

¨ Quercus > Quizzes > Exit Card – Class 10

Looking Forward

Aug 10 •Training

programs

Aug 15 •Review

Aug 17, 2pm EDT •Final

assessment released

Aug 19, 2pm EDT •Final

assessment due

Image sources: The Noun Project: Crystal ball by Sumit Saengthong, collection by Markus, books by Jakub Čaja, exam by Saeful Muslim, Shooting Star by IconMark