HRMN 406: Training Program

Chapter 8

Training Program Assessment

How do you know? Because! But how do you really know? Assume?

Chapter Eight Learning Objectives

Readers will achieve the following learning objectives after reading Chapter Eight.

1 Define and explain the purpose and role of training program assessments.

2 Identify and describe the major types of assessment goals. 3 Define, describe, and explain the importance of trainee assessment. 4 Identify, describe, and explain the major types of assessment strategies. 5 Describe and explain the purpose of informal assessments with

examples. 6 Describe and explain the purpose of formal assessments with

examples. 7 Define, describe, and explain the function of formative assessments. 8 Define, describe, and explain the function of summative assessments. 9 Identify and describe the primary uses of assessments.

10 Identify and describe examples of assessment methods. 11 Define, discuss, and compare the various types of training programs

and when to use assessment methods.

Learning Objective #1: Define and Explain the Purpose and Role of Training Program Assessments

A training program assessment represents a systematic process for collecting and analyzing information to make decisions about the benefits of training in general, identify issues about a specific training program, and to plan a new training program. Training program assessment provides decision-makers with a “big picture” perspective on the training offered by the organization

C o p y r i g h t 2 0 1 9 . R o u t l e d g e .

A l l r i g h t s r e s e r v e d . M a y n o t b e r e p r o d u c e d i n a n y f o r m w i t h o u t p e r m i s s i o n f r o m t h e p u b l i s h e r , e x c e p t f a i r u s e s p e r m i t t e d u n d e r U . S . o r a p p l i c a b l e c o p y r i g h t l a w .

EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 10/3/2022 6:09 PM via UNIVERSITY OF MARYLAND GLOBAL CAMPUS AN: 1881039 ; Stanley C. Ross.; Training and Development in Organizations : An Essential Guide For Trainers Account: s4264928.main.eds

106 Training Program Assessment

to employees. This “big picture” perspective enables decision-makers to review the training within the broader context of how training supports the organization in achieving the organization’s goals.

To better understand the importance of program assessment, we need to discuss several critical issues that underscore the value of assessment to the organization along with the benefits associated with conducting program assessments. We know that values shape behavior. As such, for assessment to be a vital subject for organizational actions linked to training, the training program assessment must become an important value among the core values associated with the organization’s culture. Furthermore, another important value, derived from the discipline of quality management, is the focus on continuous and ongoing improvement. This is fundamental to designing and implementing training programs because the role of training is to promote changes that modify employee behavior to support the achievement of the organization’s goals.

Assessment helps decision-makers learn what aspects of training are ben- eficial and where there is a need to institute modifications to the training plan model and the training plan model implementation process. The training plan model represents a framework or blue print to guide the training program designers. The training plan model is part of the systematic process for creat- ing individual training programs and implementing the training program. The training plan model and model implementation process represent collec- tive decisions made by all the important stakeholders. As such, these decisions require some type of assessment to ensure that the results of the decisions benefit the organization and trainees.

Program assessment is an important part of the organization’s change pro- cess. Assessment imposes the need for accountability among all organizational participants. Accountability requires standards of behavior that represent the performance indicators needed to assess successful training. Without includ- ing standards, the information collected is of limited value because the infor- mation needs to be analyzed by comparing the findings with the expectations. Expectations are defined by the standards set. For example, if trainees score above 70 on an end-of-training exam, training is assessed as being successful. There are levels of success that represent more discriminating forms of assess- ment. Using descriptive statistics we can say that 95% of trainees earned 70 or higher; 85% f trainees earned 80 or higher, and so forth. This level of analysis provides more discerning insights into the impact of a training program.

The assessment of training needs to generate objective data because train- ing goals are often directly linked to the organization’s goals. Trainees need to learn a subject and apply what was learned during training on the job. The organization needs to determine if learning has occurred and if the learning is transferable if the application of what was taught is to have a meaningful impact on the organization. This means the organization must move beyond program assessment that only assesses whether trainees liked the training and the training process, because a trainee’s positive attitude does not equate to

EBSCOhost - printed on 10/3/2022 6:09 PM via UNIVERSITY OF MARYLAND GLOBAL CAMPUS. All use subject to https://www.ebsco.com/terms-of-use

Training Program Assessment 107

learning. Quantitative data provides decision-makers with objective perfor- mance indicators that reveal insights which are not available using qualitative information, especially if the information is to shed light on achieving the organization’s goals.

Learning Objective #2: Identify and Describe the Major Types of Assessment Goals

Assessment involves the evaluation of three types of goals along with the organization’s efforts at achieving each type of goal. These goal types include:

• Organizational goals • Training goals • Individual training program goals

Noting the linkage between these goals during the assessment process, or if there is a disconnect between the goals, is important to ensure learning, because the goals need to be mutually supportive.

Organizational goals are strategic. The entire organization focuses on achieving these goals. Training represents one method that organizations use to achieve organizational goals. Assessing the specific impact of training on achieving organizational goals presents a challenge because organizations typ- ically have several initiatives that aid in achieving the organizational goals. The collective wisdom of the decision-makers and achieving the training goals jointly represent the optimal, indirect ways to validate the benefits of training towards achieving the organization’s goals. Two examples help illustrate the point. An organization can have the goal of increasing sales per salesperson and the goal of lowering inventory costs on a percentage change basis per inventory control specialist. Training goals can emphasize designing specific training programs oriented towards achieving these organizational goals.

Training goals represent goals linked to the organization’s own goals and they are meant to help achieve the organization’s goals. .Examples of training goals for sales people can include:

• Learn the basic methods of salesmanship. • Learn strategies for qualifying potential customers. • Learn strategies for closing a sale. • Learn strategies for overcoming resistance.

Training goals for employees participating in training to improve inventory management practices and reduce inventory costs can include:

• Learn all the major methods used to monitor the inventory. • Learn the important inventory management strategies.

EBSCOhost - printed on 10/3/2022 6:09 PM via UNIVERSITY OF MARYLAND GLOBAL CAMPUS. All use subject to https://www.ebsco.com/terms-of-use

108 Training Program Assessment

• Learn the basic techniques associated with operating the programmed, automated re-order function of the technology used in the inventory management software.

Training program goals represent the goals for trainees to achieve that enable the trainees to aid the organization in achieving the overall training goals and indirectly the goals of the organization itself.

Examples of training program goals include:

• Trainees accomplished all the learning objectives. • Learning objectives represent objectives with performance targets; learn

how much trainees achieved these goals (e.g., 10% of trainees received a score of 90 or higher on the comprehension exam).

• Learn whether trainees applied what was learned to the job and the extent to which learning was applied.

• Assess the decisions regarding the various segments of the training pro- gram design.

• Learn about the effectiveness of each instructional method used during training.

• Assess the extent to which trainees’ self-perceptions of their self-efficacy abilities strengthened as a result of the training program.

Learning Objective #3: Define, Describe, and Explain the Importance of Trainee Assessment

For the organization, the goal of training is to positively impact the trainee’s work performance. For the trainee the goals of training are two-fold. First, to build their self-efficacy abilities to exert greater control over their work responsibilities. Increased control enhances the trainee’s efforts at making work more enjoyable with increased responsibilities that result in such tangi- ble benefits as positive work reviews, promotions, pay raises, etc. The second goal is to strengthen the trainee’s ability to shape career plans in a long-term direction in order to achieve the trainee’s vision about their career.

Assessment is necessary in order for the organization and trainee to learn if training is influencing the process for achieving the goals of both the organi- zation and the trainee. Assessment produces information regarding trainees’ learning in general and the extent to which trainees have achieved the learn- ing objectives. Learning objectives represent one set of standards for assessing if learning occurs. It is important to assess the level of success because having levels shows that there are degrees of mastery of a subject. The higher the level achieved, the greater the mastery of the subject. For example, if trainees received a score of 60 on a scale of 0 to 100, then some mastery is evident. But if trainees average a score of 95 on the same scale, this demonstrates almost total mastery of the subject taught during the training.

EBSCOhost - printed on 10/3/2022 6:09 PM via UNIVERSITY OF MARYLAND GLOBAL CAMPUS. All use subject to https://www.ebsco.com/terms-of-use

Training Program Assessment 109

Mastery of a subject does not sufficiently help the organization or the individual to know if the trainees will move forward in the process of achiev- ing the organization’s and their own goals. Knowledge does not necessarily equate with job success. Assessment needs to include learning about whether the trainee’s behaviors post-training represent a shift in their work efforts as a consequence of the training. Mastery is linked with self-confidence. If the trainee considers themselves as having mastered a subject and values the usefulness of the subject in achieving their personal career goals, then the trainee’s behavior shifts. Attitude is an important influence in the behavior of a trainee, but even more important is the WIIFM (What’s in it for me?) prin- ciple. Trainees’ practical, pragmatic approach is the dominant influence over attitude. Trainees can have positive attitudes in general and enjoy the training without altering their work behavior after training. Likewise, trainees can have a less than positive attitude about training, but nevertheless the trainee’s post-training behavior shifts.

A change in behavior is still not sufficient feedback to judge fully the effectiveness of training. Results are paramount to recognizing the value of a training program. A change in performance that leads to results moves both the organization and the individual forward on the path to achieving the organization’s goals and the trainee’s personal goals. Assessing the individual’s performance is necessary to learn if the change in behavior is also associated with changes in work performance.

Learning Objective #4: Identify, Describe, and Explain the Major Types of Assessment Strategies

An assessment strategy is not an assessment method. An assessment strategy represents a specific plan for conducting an assessment of trainee learning and includes assessment of the training program as well. Strategy influences the choice of assessment methods. Assessment methods aid in implementing the strategy. Assessment is integral to the design of any training program because decision-makers and training program designers need to learn if training ben- efits the organization and the trainees. Identifying benefits, ferreting out pro- gram design flaws, as well as justifying the types of training offered represent the primary reasons why assessment needs to be an important component of the training process.

There are a wide-range of possible assessment strategies to select from. Some can be combined to make a comprehensive assessment strategy. Designers can choose to use the integrative assessment strategy. The integra- tive strategy is meant to achieve two goals: assessment of the training program and assessment of trainee learning. To assess the training program, there is a pre-test(s) prior to starting training and a post-test(s) that occurs after the program ends. If any changes are identified in the post-test the presumption is that the result is due to training. Decision-makers and designers can set

EBSCOhost - printed on 10/3/2022 6:09 PM via UNIVERSITY OF MARYLAND GLOBAL CAMPUS. All use subject to https://www.ebsco.com/terms-of-use

110 Training Program Assessment

standards for the post-test(s) to learn if the program has achieved the expecta- tions. Assessment during training focuses on trainee learning during the train- ing process. Learning can be assessed after training too, but this commonly involves using testing methods to evaluate the training program.

Next is the feel-good assessment strategy. This strategy is often used to learn about a trainee’s attitude and impressions of the training program along with learning if the training was beneficial. The assumption underlying the use of this strategy is that the trainees know best if they learned, and that a positive attitude equates with learning. Researchers have found that personal bias often skews the findings one way or another. This type of assessment strategy works best when learning about trainee attitudes and impressions is the primary goal.

The concurrent assessment strategy guides assessment during training only. Typically, this strategy is most useful when the content of the training is of secondary importance to the organization and the trainee. Limited time, limited budgets, and a secondary-type subject only warrant the use of a less ambitious assessment strategy.

The behavioral assessment strategy focuses on post-training assessment, with the emphasis on trainee behavioral changes as a consequence of train- ing. The goal of this strategy is to learn about the impact of training on trainee behavior after training. The basic premise is that if learning occurred as a result of training then behavioral changes should result. However, while behavior might change, job performance might not. If job performance is the focus, the job performance assessment strategy is the preferred strategy to use. Behavior needs to change, but the changed behavior should then lead to changes in job performance. This strategy represents another way to learn about the changes in job performance as a consequence of training.

The non-traditional assessment strategy has two variations. For atypical training programs, a non-traditional approach for assessing the benefits is more likely to offer useful feedback to the organization and trainee. Training that focuses on developing soft skills is an example of a subject that applies in this case. Standard types of training programs would emphasize the use of tra- ditional assessment methods. Two other examples of subjects in this category include management training programs and orientation programs.

Respondent bias can be a problem. A non-traditional assessment strategy that emphasizes the collection of unbiased information is valuable to all par- ties involved with learning about training program impact. Bias distorts the feedback in ways that can negatively influence future programming decisions.

Finally, there is the performance assessment strategy. The performance assessment strategy focuses on learning if trainees have achieved program performance standards. Whether a standardized test or a non-standardized test is used, a set of performance standards are identified and trainee learn- ing is expected to lead to trainees performing at or above the performance standard(s). For example, a standard could be a minimum score of 70 or

EBSCOhost - printed on 10/3/2022 6:09 PM via UNIVERSITY OF MARYLAND GLOBAL CAMPUS. All use subject to https://www.ebsco.com/terms-of-use

Training Program Assessment 111

higher on a test. A standard is modifiable and designers determine the impor- tance of trainees demonstrating a particular level of subject knowledge. The use of a standard also benefits trainees who need to repeat training and then be assessed again, as the standard represents a goal to achieve.

Learning Objective #5: Describe and Explain the Purpose of Informal Assessments with Examples

Assessment serves as a means for learning if the training program has accom- plished the training goals and the trainees achieved the learning objectives. Assessment is one component of the training program; a training program represents a process, designed to facilitate trainee learning. Assessment can occur formally or informally. Formal assessment represents the use of pre- planned methods to assess learning and the effectiveness of the training process in facilitating trainee learning. Informal assessment represents an unplanned approach for assessing trainee learning and possibly the training process itself. However, there is an important difference.

Through observing trainee behavior, trainee comments, or in other ways, trainers become aware that there may be problems with trainee learning. Trainers need to quickly learn if their perceptions and any other forms of indi- rect sources of feedback are valid and then determine if changes to the training process need to occur. Informal assessment is often a spontaneous action taken by trainers. If important issues arise, the trainer cannot wait to conduct a thor- ough assessment of the training program. Immediate feedback is necessary to make adaptive changes to the training program while in progress.

Often the changes to the training program are minor, such as adding more time when using a particular instructional method or reducing the time allo- cated when using a different instructional method. Adding time means either reducing the time allocated for another instructional method(s) or eliminating an instructional method(s) altogether. Decisions need to be made and imple- mented quickly.

There are circumstances where major changes to the training program are necessary. Major changes are unusual but sometimes necessary during training. Early warning signs commonly appear, at which point trainers use informal assessment as the means to confirm if the signs of trouble indicate the need for a major modification in the training. Short training programs that run for less than half a day require spontaneous modifications to the train- ing process because there is little time for the trainer(s) to reflect. Immediate action is necessary to ensure the success of the training program.

Informal assessments gather valuable information about trainee attitudes, trainee feelings about the training, and the extent to which learning is occur- ring. Additional information is gleaned about the training process to identify gaps. This information helps during the process associated with re-designing the training program. Learning about trainee reactions to the trainer(s) is

EBSCOhost - printed on 10/3/2022 6:09 PM via UNIVERSITY OF MARYLAND GLOBAL CAMPUS. All use subject to https://www.ebsco.com/terms-of-use

112 Training Program Assessment

important as well so that the trainer(s) can respond if the feedback is not linked to personality but more to the delivery of instruction or other aspects of the training process.

There are a variety of informal assessment methods. These can include non-planned Q&A sessions, use of a verbal assessment rubric conducted spontaneously but appearing as part of the training program, asking questions of individual trainees or hearing unsolicited comments from trainees during breaks, observations of trainee behaviors during the application of an instruc- tional method(s), and monitoring trainee non-verbal communication. There is little time for the trainers to make major changes, but experienced trainers know the importance of adapting to turn a bad situation into a success.

Learning Objective #6: Describe and Explain the Purpose of Formal Assessments with Examples

Formal assessment represents a planned, systematic process for collecting and analyzing information about a training program. The purpose of assessment is to evaluate the impact of training, identify aspects of the training program that were effective, and identify the parts of training that were less effective. Assessment provides decision-makers with evaluative information that helps in making decisions related to training.

A training program’s benefits are determined by the success of trainees achieving the learning objectives, whether the training goals are achieved, and the impact training has on achieving the organization’s goals. Each of these end states is linked in a conditional chain of means–end relationships. Though non-training factors also influence the achievement of organizational goals, it is important to learn if training has positively impacted the organization’s goals.

Kirkpatrick’s four-level model of evaluation is a useful paradigm for help- ing to understand and follow a process to follow when conducting a training program assessment (Editorial Team, 2017). Stage One involves the assess- ment of a trainee’s reaction to training. In Stage One, evaluators want to learn about the trainee’s attitude and perceptions about the training program. Stage Two involves assessing how much trainees learned during training. In Stage Three the focus is on assessing trainee behavior post-training to deter- mine what behavior changed as a consequence of training. Stage Four of the assessment process involves learning how the organization has benefited from training. As discussed previously, benefits refer to the extent to which train- ing contributes to achieving the training goals and the organization’s goals.

A formal assessment refers to assessment that is an integral part of the training program. As such, designers identify both the times when assess- ment should occur and the specific types of assessment methods to use. As a component of the training process, assessment represents a process within a process. As a first step, designers begin creating the assessment process by identifying standards to judge the effectiveness of training. Following this

EBSCOhost - printed on 10/3/2022 6:09 PM via UNIVERSITY OF MARYLAND GLOBAL CAMPUS. All use subject to https://www.ebsco.com/terms-of-use

Training Program Assessment 113

step is the selection of the assessment methods to use to assess trainee perfor- mance compared against the expectations set (performance standards). Next, designers need to identify when to evaluate the information using the per- formance standards as a guide. The final stage is to identify when and how to report the findings. The report needs to describe the training impact on the trainee achieving the learning objectives, impact on the training goals, and the impact of the training on the organization’s efforts to achieve the organi- zation’s goals. The report also needs to cover the effectiveness of the training program, what worked, and where challenges occurred.

In formal training programs, designers identify the stage(s) in which assess- ment is to occur. Assessment can begin during the pre-training stage. Pre- training assessment follows a simple experimental design. The stages are pre-training assessment, training, and then post-training assessment to learn how training has impacted trainee learning. Assessment can and often does occur during training. Any type of subjects involving technical training require more frequent assessment and the use of objective assessment methods.

Technical training involves learning in blocks of knowledge, where each block builds on a prior block of knowledge. Trainers need to learn if the trainees have learned sufficiently to move on to the next block of knowledge. This is why assessment methods that generate objective feedback are impor- tant. Objective feedback provides the information to determine whether to move forward to the next block of technical knowledge.

The next assessment stage is the end-of-training stage. Evaluators want to assess trainee learning and the effectiveness of the training program. The final assessment stage is post-training assessment. Evaluators want to learn about the transfer effect. The transfer effect refers to changes in trainee behavior as a result of completing training and whether training had an impact on job performance.

In sum, formal assessment of a training program represents a systematic process that provides the following benefits.

• Diagnostic feedback about the training program. • The assessment of the usefulness of the training program. • Guidance in making decisions regarding modifications to the train-

ing program. • Identification of effective and ineffective instructional methods used during

training. • Knowledge about the extent to which trainees actually used what was

learned in performing their job differently.

Learning Objective #7: Define, Describe, and Explain the Function of the Term Formative Assessment

In a systematic approach to designing an assessment process, the general rule of thumb is to incorporate assessment as part of the training program design

EBSCOhost - printed on 10/3/2022 6:09 PM via UNIVERSITY OF MARYLAND GLOBAL CAMPUS. All use subject to https://www.ebsco.com/terms-of-use

114 Training Program Assessment

process. By ensuring that assessment is part of the training program, trainers, decision-makers, and trainees are assured of getting feedback about trainee attitudes regarding the training program and the extent to which trainees achieve the learning objectives.

There are four stages to the assessment design process. The first stage is pre-training assessment, which has already been covered. The second assess- ment stage occurs during training. This stage is labeled “formative assess- ment.” Formative assessment can include planned and unplanned methods used to elicit information from the trainees and through trainer observations. The goal is to identify gaps in the training program that impede learning and as a consequence the trainee’s ability to achieve all of the learning objectives. Assessment during training can include assessing levels of learning if these are included in the learning objectives.

Where issues arise during the training process, trainers can respond to them by modifying the training program. The goal in adapting the training program to respond to unforeseen issues is to ensure that trainees achieve at least some of the learning objectives. Another benefit associated with con- ducting assessment(s) during training is that the trainers and the trainees keep focused and follow the prescribed training process. Each stakeholder knows that assessment represents a method and a process used to evaluate the impact of the training program.

Table 8.1 identifies and describes formal and informal methods in assessing learning as well as the use of direct and indirect methods to assess learning. A formal method represents the planned use of an assessment method(s) at a specific time during training. Designers determine in advance the sched- ule for when an assessment occurs. Often the assessment schedule revolves around testing to learn if trainees have successfully achieved a pre-determined number of learning objectives.

Informal assessment methods are unplanned and used only when trainers perceive there are issues arising during training that need investigating prior to the use of a formal assessment method. Informal assessment is typically spontaneous; trainers perceive the need to learn if their perceptions are accu- rate and, if validated, they pursue corrective measures.

Table 8.1 Types of Assessment Methods and the Difference between Formal and Informal Methods to Assess Learning

Formal Informal

Direct Planned assessment of actual trainee learning using measurable methods

Unplanned assessment of trainee learning through face-to-face interactions with the trainees

Indirect Planned assessment of trainee learning using second-hand sources

Unplanned assessment of trainee learning using indirect assessment methods

EBSCOhost - printed on 10/3/2022 6:09 PM via UNIVERSITY OF MARYLAND GLOBAL CAMPUS. All use subject to https://www.ebsco.com/terms-of-use

Training Program Assessment 115

There are direct and indirect methods that trainers can use to assess the training process and trainee learning. Direct methods typically generate objective types of information. Indirect methods generate information about the training process and trainee learning through other means, and the infor- mation is commonly subjective. Types of indirect methods include: sponta- neous or planned Q&A session, observation, conversations during breaks in training, or use of the organization’s “grapevine.”

Learning Objective #8: Define, Describe, and Explain the Function of Summative Assessment

Summative assessment refers to conducting any type of assessment dur- ing the third stage of the assessment process. Recall that stage one is the pre-assessment stage; stage two is referred to as formative assessment, and a formative assessment involves assessing concurrent with training. Summative assessment is composed of two sub-stages. Assessing at the end of training is one sub-stage. The second sub-stage occurs post-training.

Summative assessment is a final, planned approach to collecting informa- tion about the effectiveness of the training program. Effectiveness is defined by the extent to which trainees have achieved the learning objectives. The learning objectives represent standards that focus learning and also focus the summative assessment process which is to learn about the effectiveness of the training program.

However, achieving learning objectives does not conclusively determine the overall effectiveness of a training program. The fourth stage of Kirkpatrick’s four-level model of evaluation emphasizes that the effectiveness of a training program is determined by the impact of training on the trainee.

Impact refers to the extent to which learning brings about change in the trainee. Learning can impact trainees in two ways: behavioral changes, and changes in work performance or different work results. Trainee behavior can change, which is one of the intentions of training, but changes in behavior are expected to lead to improved work performance. Improved work perfor- mance should directly or indirectly impact the organizational goals.

The use of formal assessment methods is expected to lead to unbiased, objective feedback that helps decision-makers, designers, and trainers in assessing the value of a training program based on the impact training has on work performance. Impact on work performance is judged according to pre-determined standards of performance after the completion of the train- ing program. Impact on behavior is likewise evaluated by comparing actual behavior with pre-determined types of expected behavior.

Summative assessment can include the use of direct assessment methods or indirect assessment methods. There are no pre-determined guidelines that dictate the use of both or one or the other. Designers and decision-makers make the decision about whether to use direct, indirect or both forms of

EBSCOhost - printed on 10/3/2022 6:09 PM via UNIVERSITY OF MARYLAND GLOBAL CAMPUS. All use subject to https://www.ebsco.com/terms-of-use

116 Training Program Assessment

assessment methods. Direct methods include standardized tests and observa- tion. Indirect measures can include self-reports and feedback from a third party. The circumstances determine which type is used.

Learning Objective #9: Identify and Describe Uses of Assessment

There are a variety of ways to use assessments. One important use of assess- ments is to assess the training strategy, which is the overall plan that guides all decisions pertaining to the choice of training subjects and the general model that influences the design of all training programs.

Assessment of the training strategy includes the assessment of the training goals because the purpose of the strategy is to achieve these goals. Achieving the training goals is essential in helping to achieve the organization’s goals. Decision-makers and training program designers need to assess the relevance of the training goals to learn if the training goals support the organization’s goals and if training is achieving the goals.

Assessment is fundamentally a diagnostic process and assessment methods are diagnostic methods used to gather specific types of information. The choice of diagnostic assessment methods is guided by the purpose of the assessment at that time. To diagnose is to learn if a training program is effective, where there are problems in the training process, any issues with the training design model, and if there are issues with the training goals and/or training strategy.

Assessment helps in learning if trainees have achieved the learning objec- tives of a specific training program and – where objectives have standards of performance – the extent to which trainees achieve these standards. This is important to learn because the training program design requires that achiev- ing the training program learning objectives supports achieving the train- ing program goals. An example of a standard is a learning objective where trainees need to identify and describe five or more motivation theories. Five represents an objective measure of performance. Objectivity is important because objectivity is associated measuring the results.

Assessment is also useful in evaluating the actual decisions made regarding the choice of subjects to cover in training, the value of these subjects to the organization, the training process followed (guided by the training program model), the instructional methods used, and even the choice of assessment methods selected. In summary, decision-makers need to examine all aspects of organizational training to ensure that training supports the overall efforts of the organization.

Learning Objective #10: Identify and Describe Examples of Assessment Methods

There are many different types of assessment methods training program designers can select. The determining criteria in selecting assessment methods

EBSCOhost - printed on 10/3/2022 6:09 PM via UNIVERSITY OF MARYLAND GLOBAL CAMPUS. All use subject to https://www.ebsco.com/terms-of-use

Training Program Assessment 117

include time and cost constraints, and ease of use and relevance to the train- ers, decision-makers, and trainees. Assessment methods are organized into two categories: formal and informal methods. Formal methods often generate objective data and are included in the design prior to training. Formal meth- ods can be normed or validated to ensure the relevance of each question. Informal assessment methods do not follow a pre-planned schedule. The use of informal assessment methods follows an arbitrary decision-making process that primarily involves the trainers, who intuitively sense the need to conduct an assessment to confirm or disconfirm their impressions about any aspect of the training that is of concern.

Table 8.2 presents a variety of examples of assessment methods by assess- ment type, and within each type examples of formal and informal assessment methods are given. In some instances, such as a presentation, an assessment method can be both formal and informal, differing only if the method adheres to a structure or format designed as part of the training program design process.

There are standardized assessment tests that the organization can buy or choose to develop internally. These typically generate objective forms of feedback. Practical tests represent another type of assessment method. Practical tests com- monly refer to an applied focus of the assessment method. For example, assem- bling something, calculating a problem, or offering ideas on resolving different types of situations. Oral tests include conducting some type of presentation –

Table 8.2 Categories of Assessment Methods Based on Whether Formal or Informal Types of Assessment Methods with Examples of Each Type

Categories Formal Methods Informal Methods

Standardized Tests • Multiple choice • Essay • Situational judgment

Practical Tests • Role play • Reports • Situational judgment • Essay • Team project

• Self-reports • Essay

Oral Tests • Presentation with a scoring rubric • Spontaneous presentation

Observation • Simulations • Behavioral modeling

• Carried out during the performance of program activities

Peer Assessment • Survey Supervisory

Assessment • Survey • Observation

• Observation

Indirect Tests • Scoring rubrics • Interviews: either focus on behavior or

focus on situation assessment; or use a multi-model of both forms of interview

• Reconstruction of the day • Team project

• Q&A • Opinion survey • Situational judgment • Team project

EBSCOhost - printed on 10/3/2022 6:09 PM via UNIVERSITY OF MARYLAND GLOBAL CAMPUS. All use subject to https://www.ebsco.com/terms-of-use

118 Training Program Assessment

formal or informal – that demonstrates successful learning. Observation forms of tests principally focus on monitoring trainee behavior while using related types of instructional methods. An observation can be both formal (i.e., with an observation scoring rubric) or informal (i.e., spontaneous impressions). Formal observation assessments sometimes involve a non-trainer trained in identifying specific trainee behaviors relevant to the training program.

Peer and supervisor assessments mostly involve the use of formal methods to generate objective information. A peer assessment can occur during training among the trainees and when instructional methods involve planned interac- tions among trainees, where trainees interact with each other. Peer assessment can refer to assessment by work peers if they are involved in the training and are individuals that the trainee works with in the performance of their regular work responsibilities. Supervisor assessment methods are formal and are intended to focus on trainee behavior at work that has been influenced during the training and/or trainee work behavior that the training has influenced. Finally, there are indirect types of assessment methods, commonly informal, that represent any type of evaluative activity, such as observing trainee actions during the conduct of a meeting with another person or persons.

Learning Objective #11: Define, Discuss, and Compare the Types of Training Programs and When to Use Assessment Methods

Any training program can incorporate assessment as part of the training pro- cess. However, designers of training programs confront various constraints, and these constraints frequently determine how to include assessment during the training process along with the choice of assessment methods. Where training focuses on developing an individual’s personal abilities, the use of informal assessment methods is the most common practice, if used at all. The individual knows what was learned. In this situation an attitude survey is most useful assessment method to generate trainee feedback about the training program.

When a training program emphasizes technical training themes, impor- tant changes to mainstream work processes, meeting legal requirements, the transferability of learning, change in critical attitudes, or in evaluating the effectiveness of a training program, then using formal assessment methods is necessary. Formal assessment methods provide decision-makers, training pro- gram designers, and trainers with objective information that provides insights about a program’s effectiveness. Effectiveness is measured by objective feed- back gained through the use of formal assessment methods.

For example, technical training programs are frequently designed where the subject matter is sequenced in a set order (i.e., stage) and each succeed- ing stage of the training process requires knowing the preceding stage suf- ficiently to move forward. If learning is insufficient, trainers need to know this quickly to make modifications to the training process. Without modi- fications, less gets learned when moving forward in the training process and

EBSCOhost - printed on 10/3/2022 6:09 PM via UNIVERSITY OF MARYLAND GLOBAL CAMPUS. All use subject to https://www.ebsco.com/terms-of-use

Training Program Assessment 119

trainees become frustrated and resistant to learning because of the inability to successfully keep moving forward in the training process.

Training associated with changes in important work processes is critical to the ongoing success of the organization. Work processes follow a stage-by- stage model and often critical work processes are long and complex. Knowing if the trainees have learned and understand the specifics of the modified work process is essential to the successful application of the new work process.

Legal requirements that mandate that an organization provide certain types of training means that the organization likely needs to report the results of training. The use of formal assessment methods generates objective informa- tion useful in reporting training program results to government agencies, as well as to decision-makers in the organization.

Transferability of learning is sometimes critical in achieving the goals of the organization. Knowing if trainee behavior has changed and if job perfor- mance has improved is essential in learning if training has been effective. Most training programs operate on the assumption that some transferability will occur. Though transferability is important, designers need to prioritize when using formal assessment methods to assess the transferability effect. Designers need to prioritize assessment-based input from decision-makers about which training programs actually require assessment to measure the transferabil- ity effect.

Changing employee attitudes is important to an organization because atti- tudes can positively or negatively impact an individual’s job performance, and this can impact the organization’s performance. Designers need to involve decision-makers in deciding about the need to use formal assessment methods when learning about trainee work attitudes.

Finally, designers and decision-makers need to make decisions about the usefulness of a training program by assessing the training program in order to learn what works and what changes are necessary. Formal assessment methods provide the objective feedback, and the use of an attitude survey helps to find out how trainees reacted to the training. With this information, decision- makers and designers can make decisions about the viability of a training program, as well as the benefits and problems with various components of the training program.

Questions

1 Training program assessment is important. Explain. 2 Identify and briefly describe the major types of assessment goals. 3 Identify and briefly describe the major types of assessment strategies. 4 Explain the differences between formal and informal assessments. 5 Briefly describe the differences between formative and summative assess-

ment methods. Which is more important to an organization? Explain. 6 Provide two examples of each of the following: formative assessment

methods and summative assessment methods.

EBSCOhost - printed on 10/3/2022 6:09 PM via UNIVERSITY OF MARYLAND GLOBAL CAMPUS. All use subject to https://www.ebsco.com/terms-of-use

120 Training Program Assessment

Bibliography

Australian Skills Quality Authority. (2015). Guide to developing assessment tools (online). Australian Government, April 1. Available at: https://www.asqa.gov.au/sites/g/files/ net3521/f/Guide_to_developing_assessment_tools.pdf?v=1508135481, 1–11.

Cate, O.T. & Scheele, F. (2007). Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Academic Medicine. (82)6: 542–547.

Dawson, R. (1995). Fill this in before you go: Under-utilized evaluation sheets. Journal of European Industrial Training. 19(2): 3-7.

Editorial Team. (2017). Kirkpatrick’s four-level training evaluation model. Mindtools (online). Available at: https://www.mindtools.com/pages/article/kirkpatrick.htm: 1–7.

Educational Testing Services. (2012). Assessment Methods (online). Available at: https:// www.ets.org/s/workforce_readiness/pdf/21333_big_5.pdf: 1–8.

Edutopia. (2008). What are some types of assessment? July 15 (online). Available at: https://www.edutopia.org/assessment-guide-description: 1–3.

Jones, C.A. (2005). Assessment for Learning. Learning and Skills Development Agency: 1–43. Lewis, P. & Thornhill, A. (1994). The evaluation of training: An organizational culture

approach. Journal of European Industry Training. 18(8): 25–32. Mann, S. & Robertson, I.T. (1996). What should training evaluations evaluate? Journal of

European Industrial Training. 20(9): 14–20. Mavin, S., Lee, L. & Robson, F. (2010). The evaluation of learning and development

in the workplace: A review of the literature. Higher Education Finding Council for England. 1–30.

Pappas, C. (2016). 7 tips to create effective eLearning assessments to measure online training. eLearning Industry (online). January. Available at: https://elearningindustry. com/7-tips-create-effective-elearning-assessments-measure-online-training: 1–10.

Saad, S., Carter, G. W., Rothenberg, M. & Israelson, E. (2000). Testing and assessment: An employer’s guide to good assessment. U.S. Department of Labor, Employment and Training Administration. Available at: https://www.onetcenter.org/dl_files/emp TestAsse.pdf.

Scalese, R.J., Obeso, V.T. & Isenberg, S.B. (2008). Simulation technology for skills training and competency assessment in medical education. Journal of General Internal Medicine. 23(1): 1–14.

Schilling, K. & Applegate, R. (2013). Best methods for evaluating educational impact: A comparison of the efficacy of commonly used measures of library instruction. Journal of Medical Library Association. 100(4): 258–269.

Stassen, M.L.A., Doherty, K. & Poe, M. (2016). Program-based review and assessment: Tools and techniques for program improvement. Office of Academic Planning & Assessment: University of Massachusetts Amherst. Available at: http://www.umass.edu/ oapa/sites/default/files/pdf/handbooks/program_assessment_handbook.pdf: 1–61.

Training Today. (2016). Ensure your training is effective (online) Available at: http:// trainingtoday.blr.com/article/ensure-your-training-is-effective/: 1–6.

EBSCOhost - printed on 10/3/2022 6:09 PM via UNIVERSITY OF MARYLAND GLOBAL CAMPUS. All use subject to https://www.ebsco.com/terms-of-use