Training ROI and the Organization's Strategic Connection

profileSleepy
LectureNotesEvaluatingTrainingFourLevels.ppt
Home>Human Resource Management homework help>Training ROI and the Organization's Strategic Connection

Evaluating Training

*

Training Evaluation

  • Evaluation and analysis is the last step in the ADDIE training model. It allows an opportunity to go back to the beginning and assess the results of the training cycle. Early in the process, learning objectives were set and actions to accomplish these objectives were determined. Evaluation allows the organization to assess if the training accomplishments met those objectives and reflect the goals established in the strategic plan.

  • Reluctance to evaluate:

Managers are unwilling to devote time to evaluation.

Lack of know-how or no importance.

  • Why evaluate?

Link to organizational strategy.

Effectiveness of training.

Return on investment.

*

*

Organizations invest millions of dollars in training programs in an effort to gain the competitive advantage generated by well-trained employees. A 2004 American Society for Training and Development (ASTD) benchmarking survey showed that small companies (those with 1-499 employees) spent $1,194 per employee for training and organizations with 2000 or more employees spent $800 per employee. No one would argue that training is not expensive. Organizations want to see a positive return for training dollars spent. As a result, training managers are increasingly being asked to justify expenses and demonstrate how training dollars increase the organization’s bottom line return.

Even so, managers are sometimes unwilling to spend the time needed to evaluate training, or they simply don’t know how to do it or don’t see the importance of evaluation. Training departments with limited budgets often assume their training has been effective and then put their dollars into new training programs instead of evaluation. Some of the reluctance to evaluate is understandable because in many cases, much of the benefit of training is intangible and can be difficult or impossible to measure. What dollar value can be placed on improved employee attitudes? Priceless, certainly! Management, though, wants real numbers; priceless won’t suffice as an answer.

Training Evaluation

Was the training effective?

  • Training effectiveness: The benefits the organization and trainees receive from training.

*

*

Training Evaluation

  • Formative evaluation: Evaluation of training that takes place during program design.

May result in content change.

May involve pilot test.

May adjust to meet needs of the trainees.

Formative evaluation starts long before the training program is completed. It begins during the program design. Evaluation in the early stages enables trainers to ensure that when the training is implemented, it is well organized and runs smoothly. Formative evaluation is also done to assess whether the trainees will learn the content intended and that they will be satisfied with the training program.

*

*

Training Evaluation

  • Summative evaluation: Evaluation conducted at the end of training.

Used to determine the extent to which trainees have changed as a result of the training program.

Used to measure return on investment.

Summative evaluation occurs at the end of the program. It is used to determine the extent to which trainees have changed as a result of the training. It measures the change in trainees knowledge, skills, attitudes and behaviors. It is also used to assess the organization’s return on training investment.

*

*

Instructional Design: ADDIE Model

*

*

Evaluation Process

  • Conduct a needs analysis.
  • Develop measurable learning outcomes and plan for transfer of training.
  • Develop outcome measures.
  • Choose an evaluation strategy.
  • Plan and execute the evaluation.

*

*

Kirkpatrick’s Four-Level Model of Evaluation

  • Level 1: Reaction
  • Level 2: Learning
  • Level 3: Behavior
  • Level 4: Results

  • The following slides describe each of the four levels in detail.

*

*

Level 1: Reaction

  • Reaction: How did participants react to the program?

Smile sheets.

Informal comments from participants.

Focus group sessions with participants.

  • This level measures reaction only; learners identify if they were satisfied with the training. It does not indicate if learners acquired any knew knowledge or skills, nor does it indicate that any new learning will be carried back to the workplace.

  • If learners react poorly to the training and indicate dissatisfaction at this evaluation level, trainers must determine if the negative results are due to poor program design or unskilled delivery.

*

*

Level 2: Learning

  • Learning: To what extent did participants improve knowledge and skills and change attitudes as a result of the training?

Pre- and post-tests scores.

On-the-job assessment.

Supervisor reports.

*

*

Level 3: Behavior

  • Behavior: Do learners use their newly acquired skills and knowledge on the job?

Transfer of training.

On-the-job observation.

Self-evaluation.

Supervisor and peer evaluation.

  • Kirkpatrick’s third evaluation level explores the consequences of the learner’s behavior. Has the learner transferred the learning back to changed performance in the workplace? Can the learner actually put the newly acquired skills to use on the job?

  • This is referred to as transfer of training. No matter how good the training program was, if participants cannot (or will not) use the new skills and knowledge on the job, the training has little value to the employer.

*

*

Level 4: Results

  • Results: What organizational benefits resulted from the training?

Difficult and costly to collect.

Impossible to isolate the results of training.

Measuring return on investment.

Financial reports.

Quality inspections.

Interviews.

*

*

Level 4: Results

  • Kirkpatrick’s level four evaluates the final results of the training. It asks the question – What effect has the training achieved? Effects can include such things as morale, teamwork, and most certainly, the monetary effect on the organization’s bottom line. Management wants to know if they received value for the training dollars spent and what their return on investment was.

  • Collecting and analyzing evaluation at this level can be difficult and time-consuming. Part of the difficulty comes from the challenge of isolating the training variable from other factors in the organization that may also affect learners’ behaviors. When employee behavior changes, it is difficult to know if the change is the result of training or the result of some other environmental factor.

  • Level four evaluations are done through financial reports, quality inspections and interviews with management personnel.

*

*

Level 4: Results

  • The difficulty and cost of conducting evaluations increases as you move up the levels. Organizations and trainers must carefully consider which levels of evaluations are appropriate for which training programs. Most commonly, level one evaluations are conducted for all training. Level two--learning evaluations--are generally conducted for skills training programs. Level three evaluations–behavior—for strategic programs and level four--results evaluations—are appropriate only for broad-based, high-budget training programs.

  • Unfortunately, the easy evaluation instruments used at level one don’t give results that have much value to the organization. The value of the information obtained from the process increases as evaluation moves to higher levels. Level four–results–is the most difficult to assess and yet reveals the most valuable information.

*

*

Levels of Evaluation vs. Value

*

Practice

Think about workplace training you have received. How would you design evaluation instruments for that training using the four levels of evaluation?

*

*