Granth W2D1 652
CHAPTER FOUR
Relating Training to Business Performance: The Case for a Business Evaluation Strategy
William J. Tarnacki II
Eileen R. Banchoff
For many years organizations have been professing that the key to a truly sustainable competitive advantage is an engaged and talented workforce. “Managers are fond of the maxim `Employees are our most important asset.’ Yet beneath the rhetoric, too many executives still regard—and manage—employees as costs. That’s dangerous because, for many companies, people are the only source of long-term competitive advantage” (Bassi & McMurrer, 2007, p. 115). This professed realization has pushed organizations to establish training programs (and even corporate universities) that provide opportunities for employees to develop skills and competencies related to their existing (or sometimes future) roles in the organization.
These training programs have evolved tremendously over time, becoming much more sophisticated and oriented toward creating a well-rounded workforce. Unfortunately, these training and development (T+D) efforts have not kept pace with the changing demands of business. In fact, T+D departments have evolved to be separate entities from the operations of the business, basically managing a repository of training options versus partnering with business and operational leaders to customize solutions based on evolving business needs. Recent attempts to broaden T+D efforts to encompass performance improvement (PI) are a much needed, long overdue, uphill climb. Unfortunately again, today’s business leaders are looking to their human resources (HR), PI, and T+D colleagues to operate at a much higher level and to develop a new language around the expectations and the demands of the business.
If our field of practice is changing (albeit slowly), it stands to reason that the traditional evaluation methods (see Table 4.1) we use to measure transfer from our training programs (skills and knowledge) are also too narrow to measure business results. These evaluation methods are being taught and even trained in the context of another narrow model—the ADDIE instructional design model (analyze, design, develop, implement, and evaluate). Evaluation strategies and tools, based on T+D and ADDIE, limit our ability to understand the overall business model and associated metrics in order to offer robust, impactful, meaningful evaluation results that help manage the business.
Table 4.1 Traditional Evaluation Types
Types of Evaluation Which Evaluates . . .
Formative appropriateness of all components of an instructional solution through each stage of the instructional systems design (ISD) process
Summative learning and reaction from an instructional solution during final development and implementation, including all stakeholders, media, environment, and organization as components of that instructional solution
Confirmative effectiveness of an instructional solution after implementation through analysis of changes in behavior, achievement, and results
Meta evaluation methods applied to the ISD process
Level 1 participant reaction to an instructional intervention
Level 2 participant learning as a result of the instruction
Level 3 changes in participant behavior on the job due to the instruction
Level 4 results to the business as a result of the instruction
Level 5/ROI financial value of the instruction, in terms of positive cash flow from the investment minus the cost of the program implementation
To help build the case for a more business-focused evaluation strategy, this chapter will trace the real-life professional journey of Joseph Williams (fictional name) as he matures from a young, academically prepared ADDIE advocate to a sophisticated human resources strategist and business partner. Mapping Joseph’s experiential journey demonstrates how his perspectives changed to align his training results with business results without completely revamping the long-trusted tools and methods he learned in graduate school.
STUDY OF PERSEPCTIVES
The Academic Perspective
It was May 1997 and Joseph Williams had just graduated with a master’s degree in instructional technology from a very reputable urban university. Basking in the glory of his new degree, Joseph was excited about applying all the wonderful concepts and practices he had just spent two plus years learning. He left the graduation ceremonies eager to employ ADDIE and its corresponding evaluation techniques to become the hero he knew some organizations were desperately waiting for.
While completing his coursework in the evenings, Joseph spent his days as a foreman and quality specialist on the shop floor of a small manufacturing organization. In this academic preparation period, he worked diligently to incorporate several new skills and techniques into his day-to-day manufacturing activities. But he soon ran into the brick wall of lack of interest and engagement and found that this organization was just too small and had too few resources to “hear his voice.” In order to begin applying his new wealth of knowledge and expertise, he decided he had to move on to a bigger and more strategic enterprise.
The Novice Perspective
Soon after graduation (in 1997), Joseph jumped at an opportunity to work in the training and development department of a large automotive company’s finance division. From his first day in the new white-collar environment, Joseph was sure he could make a difference by analyzing, designing, and delivering training programs that would have an immediate, positive impact on this global finance business.
Alas, it did not take long for Joseph to realize that the full application of the instructional design model was not part of the expectations of his new job. Instead, management directed him to use his ISD background to oversee projects that addressed needs they had identified during an analysis performed five years earlier! He also soon discovered that his deliverables were predetermined (seminar courses in a standard format) and that the implementers (the project trainers) would have minimal involvement in the design, development, and evaluation processes. Joseph held the title of “Instructional Designer,” but he soon acquiesced to being just a project manager.
The “siloed” nature of the firm’s T+D structure was perplexing to Joseph, but there were several other issues that also seemed even more counter-intuitive to this fledging ISDer.
First, Joseph could not understand why design and development activities were focused on “old” information, especially when the organization was in the process of a major restructuring. For some time now, many department customers were indicating that much of the proposed content was out-of-date due to recent technological and organizational improvements.
Second, he did not know why T+D operated as its own function, was not integrated with the operational areas of the business, and was only minimally assimilated with the broader human resources function.
Third, Joseph struggled with the reality that there were stand-alone needs analysis, design, development, and delivery functions, but evaluation was not its own entity. And worse yet, evaluation was not conducted with any depth besides basic Levels 1 and 2 or summative evaluation (even formative evaluation was minimally applied).
Table 4.2 illustrates Joseph’s foundational perspectives—how different types of evaluation were taught and aligned during Joseph’s (and most professionals’) graduate preparation.
Table 4.2 Application of Traditional Evaluation Types
images