For Eng.Kelvin Only

Chapter 4: Research-Based Guidelines for Designing Distance Learning—What We Know So Far

Overview

Eduardo Salas Renee E. DeRouin Lisa N. Littrell

As Horton (2000, p. 6) aptly pointed out, ―Web-based training is part of the biggest change in the way our species conducts training since the invention of the chalkboard or perhaps the alphabet. The development of computers and electronic communications media has removed barriers of space and time. We can obtain and deliver knowledge anytime anywhere.‖ The advent of the World Wide Web has provided us with greater flexibility in the way we design and implement training than ever before. However, web- based training is only one small part of today‘s training technology. For instance, in recent years, satellite, video, audio, audiographic computer, and multimedia technology have been applied to organizational development programs, allowing organizations to provide training for employees simultaneously around the globe (Leonard, 1996). The use of these various technologies for training make up what are referred to as ―distance learning‖ methods.

Corporate and government investment in distance learning has skyrocketed over the last few years. For instance, in 1999, organizations spent $3 billion on technology- delivered training. In 2001, this expenditure was estimated to jump to $8.2 billion, an increase in investment of approximately 170 percent in two years (Moe & Blodget, 2000). With regard to e-learning, one type of distance learning, organizations reported spending approximately $2.2 billion on the development and implementation of this training method in 2000. Those expenditures were expected to rise to $14.5 billion in 2004 (―The Payoffs of e-Learning,‖ 2003).

The popularity of distance learning stems from its convenience. Distance learning allows employees to access training modules on demand from any location around the world. Moreover, it has been linked with substantial cost savings. Cisco Systems reported savings of 40 to 60 percent after implementing distance learning methods (Gill, 2000), Dow Chemical of $34 million (Brayton, 2001), IBM of $400 million (Mullich, 2004), and Hewlett-Packard of eleven months of training and $5.5 million (Horton, 2000). These benefits make distance learning an obvious solution to the challenge of training a global workforce in need of continuous skill updating. However, despite its advantages, research suggests that distance learning is not always more effective than traditional forms of instruction, such as classroom instruction (Alavi, Yoo, & Vogel, 1997; Russell,

1999; Webster & Hackley, 1997). In fact, Gale (2003) suggests that distance learning can be detrimental to organizational development if improperly implemented.

One reason for this may be that distance learning programs are not always developed according to standardized, experimentally tested procedures but instead are often simply adaptations of other training tools used by the company (for example, text materials transported into Microsoft PowerPoint and Word and placed on the web; Barton, 2001) or modifications of training programs offered by distance learning vendors (for example, click2learn.com, Smartforce, Teach.com, Learningbyte International, Learnframe, DigitalThink, Inc.; see www.distance-learning-list.com for an inventory of distance learning providers). As a result, organizations would benefit from a theoretically derived and empirically validated set of guidelines for the design and delivery of distance learning systems (Salas & Cannon-Bowers, 2001).

The purpose of this chapter is to provide applied researchers and training practitioners with literature-based guidelines for the design and delivery of distance learning systems. In developing these guidelines, we draw from the science of training literature (for example, Salas & Cannon-Bowers, 2001; Tannenbaum & Yukl, 1992) and from the theoretical and research literature on distance learning within educational (where most experimental work is performed) and organizational settings. However, because the focus of this chapter is on the implementation of organizational distance learning systems, our emphasis is on the findings from workplace distance learning studies.

In order to provide readers with an understanding of what distance learning in organizations actually involves, we begin with a definition and description of distance learning. Then, we offer thirteen guidelines for the design of distance learning programs within workplace organizations and provide literature support for each of these recommendations. We conclude with directions for future needs of distance learning research and practice and with an overall summary of how these recommendations might be used to improve the effectiveness of distance learning systems.

What Is Distance Learning?

Distance learning is ―the desired outcome of distance education‖ (Kaplan-Leiserson, 2002, para. 78), although the terms ―distance learning‖ and ―distance education‖ are often used interchangeably. Distance education has been defined by the American Society for Training and Development (ASTD) as an ―educational situation in which the instructor and students are separated by time, location, or both. Education or training courses are delivered to remote locations via synchronous or asynchronous means of instruction, including written correspondence, text, graphics, audio- and videotape, CD- ROM, online learning, audio- and videoconferencing, interactive TV, and facsimile‖ (Kaplan-Leiserson, 2002, para. 77). Many other terms are used in reference to distance learning technology, including distributed, online, distance, Internet- and web-based training (Salas, Kosarzycki, Burke, Fiore, & Stone, 2002). This has resulted in a rather fragmented understanding of distance learning, which as Kosarzycki, Salas, DeRouin,

and Fiore (2003) point out, might be due to the rapid pace at which this technology has developed.

Distance learning is a broader concept than e-learning, which is primarily considered to be associated with electronic and computer-based learning methods (OneTouch Systems, Inc., 2003). As a result, distance learning encompasses both electronic and non-electronic forms of instruction (Kaplan-Leiserson, 2002). Because of its flexibility, distance learning offers several benefits over traditional forms of instruction, including the ability to train employees quickly and efficiently around the globe, use the most knowledgeable instructors for high-quality training, provide updates to training material as necessary, increase the responsibility trainees have for learning, and add trainees or instructors without much supplementary cost (Burgess & Russell, 2003). In addition, distance learning drastically reduces the indirect costs of training (for example, costs associated with travel to training facilities, hotel accommodations for trainees, lost work time for employees attending training) that have estimated to account for 80 percent of organizational training costs (Kozlowski, Toney, Mullins, Weissbein, Brown, & Bell, 2001).

How Can Distance Learning Be Improved?

As many organizations report, distance learning can be a viable method for training workplace skills (Brown, 2001; Gopher, Weil, & Baraket, 1994; O‘Hara, 1990). However, this requires that it be designed in a way that facilitates effective learning. In this section, therefore, we discuss thirteen research-based guidelines for the design of successful distance learning programs (see Table 4.1 for a summary of these guidelines).

Table 4.1: Research-Based Guidelines for Distance Learning.

Guideline Description References

Only provide distance learning when you are sure it meets the organization‘s needs.

Before distance learning is chosen as the strategy for delivering training, it is necessary to consider the organization development problems that it will address and the manner in which it can help solve these problems.

Gale, 2003; Kiser, 2002; ―The Payoffs of e- Learning,‖ 2003; ―Ten Tactics to Make e- Learning ‗Stick‘,‖ 2003.

Take into consideration human cognitive processes when designing distance learning programs.

Distance learning should be designed so that learning modules are consistent with the tenets of cognitive learning theory, a theory that explains how the mind processes environmental stimuli.

Clark & Mayer, 2003.

Enhance the learning experience

Using multiple forms of media in distance learning is advantageous,

Clark & Mayer, 2003; Mayer, 1989; Mayer &

Table 4.1: Research-Based Guidelines for Distance Learning.

Guideline Description References

by including both graphics and text in the presentation of learning topics.

because past research has demonstrated that individuals learn the most when they are actively engaged in the learning process; active learning is facilitated when both graphics and text are used in distance learning, because individuals are forced to make relevant connections between the words and the pictures.

Anderson, 1991; Mayer & Gallini, 1990; Mayer, Steinhoff, Bower, & Mars, 1995.

Include learning games.

Learning games are typically computer games that have been adapted to train specific workplace skills; these games can be beneficial to learning in that they: (1) increase the appeal of online training; (2) make the idea of ―tests‖ less frightening; (3) facilitate discovery learning; and (4) offer trainees substantial amounts of practice in workplace skills.

DeVeaux, 2001; Horton, 2000, 2002.

Keep learners engaged.

One of the primary reasons why learners drop out of distance learning programs is the failure of many online Cannoncourses to keep learners engaged; this problem, however, can often be remedied by a simple change in the focus of the distance learning program.

Horton, 2000; Moshinskie, 2001; Salas & Bowers, 2000; Skipper, 2000.

Offer a blended approach.

Offering a combination of training techniques will likely satisfy both employees who prefer to work online and employees who prefer classroom instruction.

Goodridge, 2001; Masie, 2002; Mullich, 2004; Phillips, Phillips, & Zuniga, 2000; ASTD & The Masie Center, 2001.

Allow for interaction between trainees and for communication between trainees and facilitators.

Distance learning programs can allow for interaction between trainees and for communication between trainees and facilitators through asynchronous and synchronous communication tools and by building factors into the program such as virtual communities and interactions with expert sources and facilitators.

Davis, 2003; Green, 2003; Salopek, 2002; Shepherd, 2003; ―Ten Tactics to Make e- Learning ‗Stick‘,‖ 2003.

Table 4.1: Research-Based Guidelines for Distance Learning.

Guideline Description References

Offer computer- based, distance learning methods to computer-savvy trainees or train learners on computer basics before offering computer-based training.

Distance learning that employs computer-based instruction will likely be most suitable for trainees with high levels of prior experience with computers; however, one way to bring trainees with lower levels of computer experience up to the level of trainees with more extensive experience is to provide instruction on basic computer skills before trainees begin computer- based, distance learning.

Brown, 2001; Schelin & Smarte, 2002.

Provide distance learning for hard- skill training but supplement it with other forms of instruction for soft- skill training and for training on such abstract topics as workplace ethics.

Distance learning appears to be particularly appropriate for the training of explicit, factual-based knowledge, for soft-skill training, and for training on such abstract topics as workplace ethics; however, distance learning might best be administered by supplementing it with other training approaches (lecture, role play).

Mullich, 2004; Welsh, Wanberg, Brown, & Simmering, 2003.

Offer trainees control over certain aspects of instruction.

Trainees in distance learning programs might benefit from control over both the context of training examples/practice problems and the amount of instruction; however, it might be better to withhold some types of control from trainees, such as control over pacing, sequencing, and provision of optional content, because research has not shown these types of control to be consistently positive.

Freitag & Sullivan, 1995; Gray, 1989; Judd, Bunderson, & Bessent, 1970; Lai, 2001; Ross, Morrison, & O‘Dell, 1989; Ross & Rakow, 1981; Steinberg, Baskin, & Matthews, 1985.

When offering trainees control over instruction, make sure that trainee preparation, system design, and workplace conditions facilitate successful use of Cannonthat control.

In order to ensure that increased learner control will lead to better training outcomes, certain conditions should be met before trainees are granted this control, such as providing instructions on how to use learner control and why, allowing trainees to ―skip‖ rather than ―add‖ extra instruction, and promoting use of learner control through supervisor

Baldwin & Ford, 1988; DeRouin, Fritzsche, & Salas, 2004; Gay, 1986; Hicken, Sullivan, & Klein, 1992; Rouiller & Goldstein, 1993; Salas & Bowers, 2000, 2001; Steinberg, Baskin, & Matthews, 1985; Tannenbaum & Yukl,

Table 4.1: Research-Based Guidelines for Distance Learning.

Guideline Description References

support of this instructional feature. 1992.

Guide trainees through the distance learning program.

Tools, such as advanced organizers and cognitive maps, may help trainees to have a better understanding of the core elements of training and, simultaneously, to traverse a distance learning program more easily.

Bell & Kozlowski, 2002; Cannon-Bowers, Rhodenizer, Salas, & Bowers, 1998; Mayer, 2004.

Make the program user-friendly.

The course content of distance learning must be divided into small, manageable sections and each web page within the program should be limited to no more than two hundred words.

Mullich, 2004; Zeidman, 2003.

Guideline 1: Only provide distance learning when you are sure it meets the

organization’s needs.

Before distance learning is adopted as a training approach, it is important that organizations examine whether distance learning is an appropriate training strategy for the particular training need. Often and unintentionally, organizational management wants distance learning to work in a situation for which it is either inappropriate or infeasible. However, distance learning is not ideal for every training endeavor within an organization. Consequently, before distance learning is chosen as the strategy for delivering training, it is necessary to consider the organizational development problems that it will address and the manner in which it can help solve these problems. In addition, the costs of distance learning, its feasibility given the organizational infrastructure, and its usefulness must be discussed before a distance learning program reaches the initial design phase (―Ten Tactics to Make e-Learning ‗Stick‘,‖ 2003).

Furthermore, according to the same source, distance learning courses should be designed with a specific job in mind. Many organizations make the mistake of buying a distance learning program before they consider the objectives of the program. However, the phrase ―one size fits all‖ does not apply to distance learning, and organizations are often forced to make their distance learning needs fit the purchased program. One company, the Royal Bank of Scotland, was able to avoid this problem by conducting a thorough analysis of existing distance learning programs before making the switch from classroom to online training (Kiser, 2002). The Royal Bank decided to create its own ―homegrown‖ web-based/CD-ROM program with the help of a custom e-learning developer, Epic Group. According to Lars Hyland, the Epic Group account manager, ―The system didn‘t offer the full range of functionality written on the advertisements of a lot of LMS [Learning Management System] vendors, but it met 80 to 90 percent of the bank‘s actual needs‖ (Kiser, 2002, p. 34). Because the Royal Bank did not choose to

adopt an off-the-shelf program whose customization would have cost even more time and money, the company was able to develop a program that appropriately matched the organization‘s training needs and that could later be rebuilt for added functionality.

Guideline 2: Take into consideration human cognitive processes when designing

distance learning programs.

Distance learning should be designed so that learning modules are consistent with the tenets of cognitive learning theory. Cognitive learning theory explains how the mind reacts to sensory information in the environment. In order for sensory information to be processed and stored in long-term memory, several events have to occur. First, the critical information from the environment needs to be recognized. Second, working memory has to be cleared so that room is available for rehearsal (an essential component of learning) of this critical information. Third, the new auditory and visual information has to be consolidated and then integrated with the information already present in long-term memory through rehearsal in working memory. Fourth, the knowledge and skills gained from the integration must be retrieved from long-term memory and placed into working memory when needed. Fifth, meta-cognitive skills (that is, skills in how to manage one‘s cognitive processes) must guide and direct all of the above events (Clark & Mayer, 2003).

The design of distance learning programs should take into account each of these cognitive events. For instance, in order for critical information in learning modules to be recognized, instructors should ensure that this information is presented via vibrant colors or lists of learning objectives. To reduce the load on working memory, distance learning designers should omit irrelevant pictures and sounds and be concise in their wording of text. In order to consolidate the information from the eyes and ears and to integrate it with existing information, pictures on the screen should be combined with text and sounds and active practice should be encouraged. To improve retrieval and transfer, the practice exercises and examples in the distance learning program should incorporate material from the actual job. And last, in order to enhance meta-cognitive monitoring, the distance learning program should include self-checks on learning progress (Clark & Mayer, 2003).

Guideline 3: Enhance the learning experience by including both graphics and text in

the presentation of learning topics.

Using multiple forms of media in distance learning is advantageous because past research has demonstrated that individuals learn more from a combination of graphics and text than from text alone (Mayer, 1989; Mayer & Anderson, 1991; Mayer & Gallini, 1990). Active learning is facilitated when both graphics and text are used in distance learning, because individuals are forced to make relevant connections between the words and the pictures (Clark & Mayer, 2003). However, the illustrations that accompany the text must enhance the material presented in the text rather than distract the learner. Graphics can be used to teach content types, organize topics, demonstrate relationships, and present case studies (Clark & Mayer, 2003).

Typical distance learning programs that use both graphics and text present the textual information and graphical illustrations separately (Clark & Mayer, 2003). That is, the individual must first read the text and then scroll down to view the graphics. Learning would be enhanced if the text and graphics were presented in a more integrative fashion. Instead, the text should be placed as closely as possible to the graphical illustrations used to enhance textual meaning. Research supports this assertion in that several studies have revealed that learners receiving integrated information presentations outperform learners receiving material in which the text is separated from the graphics (Mayer, Steinhoff, Bower, & Mars, 1995; Moreno & Mayer, 1999).

Guideline 4: Include learning games.

Another way in which distance learning can be improved is through the use of learning games. Learning games are typically computer games (arcade games, quiz-show games, crossword puzzles) that have been adapted to train specific workplace skills. However, learning games can also involve tools as simple as email, chat, and Internet forums through which groups of employees collaborate on training topics (for example, the galactic wormhole game in which players ―travel‖ back and forth in time and discuss how relevant, work-related issues would change depending on the time period considered) (Jasinski & Thiagarajan, 2000). According to Horton (2002), games can be beneficial to learning in that they: (1) increase the appeal of online training; (2) make the idea of ―tests‖ less frightening; (3) facilitate discovery learning (that is, allow trainees to uncover the patterns and relationships in information themselves); and (4) offer trainees substantial amounts of practice in workplace skills. Learning games also allow trainees to actively participate in distance training without taking their focus away from the training content (Horton, 2000).

One place where learning games have been applied extensively is in military training. Because many of today‘s military recruits have extensive experience with video games, the military now uses digital war games, such as Joint Force Employment, Marine Doom, Spearhead II, and Falcon 4.0, as instructional tools for military personnel. These games serve to capture recruits‘ attention and, at the same time, provide them with skills that can be transferred to the battlefield (DeVeaux, 2001).

Through the use of learning games, learners are likely to become more engaged in the learning process and more motivated to succeed than they would in online courses that rely solely on text as a means of conveying information. However, several guidelines should be followed in the choice or design of a learning game for organizational distance learning (see Horton, 2002). First, success in the game should indicate learning of the desired material. If success in the game is not related to mastery of the required knowledge, skills, and attitudes (KSAs), then the game is not a useful method of presenting and testing information. Second, the game should always be a challenge to the learner. As the learner progresses through the game, the game should increase in difficulty. This technique ensures that the learner will continue to work hard during the learning game and that he or she will not lose interest partway through its completion. Third, the game should provide the learner with continuous and useful

feedback. This feedback will allow learners to gauge their performance and to make changes to their strategies and/or methods as the game is played. Fourth, the game should not be too difficult for the learner to understand or play. If the game requires rules and instructions that are too complicated for learners to comprehend, they are likely to give up on the game or to quit the training program altogether. Fifth, the game should be flexible and adaptable to training needs. This characteristic of the learning game allows it to be altered when new technology becomes available, when training content has to be changed, or when the game has to be played by individuals or entire classrooms at once (Horton, 2002).

Guideline 5: Keep learners engaged.

Motorola reports that a significant gap exists in the number of employees who register for their e-learning courses and the number of employees who actually complete them (Moshinskie, 2001). In addition, estimates suggest that as many as 80 percent of employees who sign up for online training programs drop out before the programs end (Flood, 2002). This attrition rate is troubling because one of the basic benefits of distance learning is greater accessibility to instructional material. If this increased accessibility actually reduces the number of people who attend employee training, then the usefulness of distance learning for employee development may be called into question.

One of the primary reasons why learners drop out of distance learning programs is the failure of many online courses to keep learners engaged (Skipper, 2000). This problem, however, can often be remedied by a simple change in the focus of the distance learning program. In his review of the literature and discussions with distance learning experts, Moshinskie (2001) found that distance learners are most likely to be engaged when they are able to see the benefits of training to on-the-job performance. As a result, learner engagement in e-learning can be enhanced simply by tying training material to requisite job skills. One way in which this might be accomplished is to describe how the training objectives are related to workplace KSAs (Salas & Cannon-Bowers, 2000). This information can be provided in the instructions trainees are given before training begins.

In addition to linking training objectives to job skills in the instructions, Horton (2000) recommends that distance learning tasks actually require the development of some form of intellectual property that learners can use after completing the training course. For instance, the course could provide employees with training on how to write action plans or business proposals that could later be utilized within the company. In addition, the training could involve the creation of a computer program that could afterwards be applied back on the job. The development of these various forms of intellectual property will help trainees see the relevance of training to workplace KSAs. As a result, they will find themselves more engaged in the learning process and will be less likely to quit before training is completed.

Guideline 6: Offer a blended approach.

Although distance learning is often one of the most efficient ways to deliver training, it is sometimes not the most preferred. A report by ASTD found that 62 percent of employees surveyed preferred classroom training to e-learning (ASTD & The Masie Center, 2001). In addition, in their review of the literature on distance learning, Phillips, Phillips, and Zuniga (2000) found that most employees prefer face-to-face instruction over distance learning methods.

One way to remedy this problem is to offer a blended approach to learning. In blended approaches, distance learning (typically computer-based, e-learning) is combined with classroom instruction to create experiences that capitalize on the best of both worlds: the cost effectiveness and on-demand availability of distance learning and the face-to- face interaction and socialization features of classroom training (Goodridge, 2001; Masie, 2002). Because distance learning is best used to convey explicit knowledge, such as product information or principles regarding customer satisfaction, it is typically used for the delivery of content information to trainees. Face-to-face instruction, in contrast, is typically used for the delivery of workplace skills due to its ability to convey tacit knowledge, such as judgment, personal awareness, interpersonal skills, and growth (Mullich, 2004).

One organization that has had success with the blended learning approach is IBM. Studies have revealed that, when compared to traditional classroom instruction, IBM‘s blended learning approach reduced training costs by one-third and increased learning outcomes by 500 percent (Mullich, 2004). Although the effectiveness of the blended learning approach over and above distance learning alone has yet to be determined, it appears that offering a combination of training techniques will likely satisfy both employees who prefer to work online and employees who prefer classroom instruction. However, as Hofmann (2001) points out, ―Determining the right blend [of classroom and distance learning techniques] isn‘t easy or to be taken lightly‖ (p. 18).

Guideline 7: Allow for interaction between trainees and for communication between

trainees and facilitators.

In a survey of online learning conducted by the 2001 Campaign for Learning, results indicated that only 3 percent of respondents preferred learning alone (Linne & Plers, 2002). In addition, research has shown that adults learn better in situations in which they are members of a community of learners (Davis, 2003). As a result, many organizations have incorporated synchronous and asynchronous communication tools into their distance learning programs. Synchronous communication tools typically involve the use of chat rooms or threaded discussions that allow trainees to ask and answer questions about training topics in ―real time.‖ In contrast, asynchronous communication tools, such as message boards, permit trainees to post questions or answers that can be accessed later by other trainees (Selix, 2001).

In a study comparing the use of these two communication tools within a distributed learning environment, Alavi, Marakas, and Yoo (2002) found that email, an asynchronous communication tool, was a significantly more effective method of interaction between trainees than a more advanced asynchronous and synchronous messaging system. In particular, the authors found that the use of email for communication between trainees led to better learning outcomes than the more advanced system. One reason for this might be that the advanced system was so complex that trainees spent more time trying to understand the communication tool than they spent working on the training task; as a result, the less complex communication tool (email) was more effective in transmitting information from one trainee to another than the sophisticated asynchronous/synchronous messaging tool.

In addition to asynchronous and synchronous communication tools, distance learning programs can also be made more interactive by building factors into the program, such as virtual communities and interactions with expert sources and facilitators (Shepherd, 2003). At Mattel, for example, best practices in business are shared through a web- based operations university in a way similar to that of a virtual community (Green, 2003). When one Mattel factory develops an innovative idea, that idea is posted to the web so that factories around the world can be alerted to and apply this new idea in their own facilities. In addition, it is essential that distance learning programs incorporate an online tutor or facilitator into the design to provide technical support (―Ten Tactics to Make e- Learning ‗Stick‘,‖ 2003). Jennifer Hofmann, president of InSync Training Synergy, recommends the use of two facilitators for synchronous learning events in which audio and video are employed, one facilitator for controlling visuals and troubleshooting and one for presenting instructional content (Salopek, 2002). According to Hofmann, ―[This technique] helps to change the voices up to keep learners engaged‖ (Salopek, 2002, p. 18). These various methods for facilitating interaction among trainees have yet to be supported by empirical research, although they are likely to facilitate a sense of community among learners, which can lead to improved learning outcomes (Davis, 2003).

Guideline 8: Offer computer-based, distance learning methods to computer-savvy

trainees or train learners on computer basics before offering computer-based training.

Although distance learning has been found to be an effective training method, it is not necessarily the best training method for everyone. Certain types of trainees (for example, trainees with computer experience) might perform better in distance learning environments than others do. One study (Brown, 2001) found that computer experience was significantly related to post-training test scores. Specifically, trainees with more computer experience performed better on the training post-test than did trainees with less computer experience. It is likely that distance learning that employs computer- based instruction will, therefore, be most suitable for trainees with high levels of prior experience with computers.

One way to bring trainees with lower levels of computer experience up to the level of trainees with more extensive experience is to provide instruction on basic computer

skills before trainees begin computer-based, distance learning. This instruction may be presented in the form of a tutorial that trainees complete before training. In order to determine who needs to complete the tutorial, a brief quiz on computer knowledge may be administered before training. Trainees who pass the quiz may continue on to the main topics of the distance learning program, whereas trainees who do not pass may be presented with the training on computer skills.

Jabil Circuit, an electronics manufacturing company, found that the effects of low computer experience on distance learning outcomes could be ameliorated by including computer training and e-learning in their classroom courses (Schelin & Smarte, 2002). According to Michael McGinnis, the training manager at Jabil Circuit, ―The way we introduced e-learning to our campus was to make it part of our classes. This enabled computer-illiterate personnel to learn how to operate the computer, and it had great results in the training . . .‖ (Schelin & Smarte, 2002, p. 22). In addition, Jabil Circuit only included less-sophisticated technologies (for example, Microsoft PowerPoint) in their distance learning platform when it was first introduced. This allowed less computer- savvy employees to focus on the content of the instruction rather than on its various tools and functions.

Guideline 9: Provide distance learning for hard-skill training but supplement it with

other forms of instruction for soft-skill training and for training on such abstract topics

as workplace ethics.

As mentioned briefly in the discussion of blended learning approaches, distance learning appears to be particularly appropriate for the training of explicit, factual-based knowledge (Mullich, 2004). However, its use in the training of soft skills has been the subject of some debate (Welsh, Wanberg, Brown, & Simmering, 2003). For instance, distance learning may be inappropriate for training in interpersonal skills and teamwork. These skills often involve the use of nonverbal and verbal communication and may, therefore, require face-to-face interaction between trainees and between trainees and instructors. In addition, distance learning may be an ineffective training method for such topics as workplace ethics. Workplace ethics often involves a set of unwritten rules that vary depending on the circumstances involved. As a result, they are not typically hard- and-fast rules that can be delivered primarily via text format.

Although organizations and educational institutions, such as the Bank of America (Dobbs, 2000) and Loyola Marymount University (see http://careers.lmu.edu), have implemented distance learning approaches for the training of interpersonal and interviewing skills, and organizations such as Dupont have used distance learning techniques to teach workplace ethics (―Ethics Before It Was Fashionable,‖ 2004), the use of distance learning for these topics might best be administered in a blended approach (as described above). In such cases, supplementing distance learning with other training approaches, for example, lecture or role play, might best provide for the training of soft skills and topics that are difficult to specify in concrete terms.

Guideline 10: Offer trainees control over certain aspects of instruction.

Research findings from the educational psychology literature suggest that learners benefit from increased control during the learning experience. For instance, learners have been found to have greater satisfaction with multimedia learning programs when they are given control over the context of examples. Ross, Morrison, and O‘Dell (1989) found that learners who were allowed to choose how their examples were framed (for example, in terms of educational, business, or sports contexts) were more satisfied with the learning experience and viewed more practice problems than learners who were not given control over the contexts of their examples.

Similar results have been found for learners given control over the amount of instruction. When learners in a computerized training program are allowed to specify whether they prefer a brief or a comprehensive training program before beginning training and are subsequently matched to this preference, they have been found to enjoy the training more and to perform better on training post-tests than trainees in programs unmatched to their preferences (Freitag & Sullivan, 1995).

These findings suggest that trainees in distance learning programs might benefit from control over both the context of training examples/practice problems and the amount of instruction (as indicated by preferences for a brief or comprehensive training program evaluated prior to the start of the training program). However, it might be better to withhold some types of control from trainees. The research findings for control over the pacing, sequencing, and provision of optional content (that is, the choice of whether or not to view more examples/practice exercises during training) has not been shown to be consistently positive. In fact, such control has been found to lead to significantly reduced training performance (Gray, 1989; Judd, Bunderson, & Bessent, 1970; Lai, 2001; Ross & Rakow, 1981; Steinberg, Baskin, & Matthews, 1985). As a result, organizations might choose to withhold giving trainees control over these instructional features until after they have demonstrated mastery of training topics.

Guideline 11: When offering trainees control over instruction, make sure that trainee

preparation, system design, and workplace conditions facilitate successful use of that

control.

In order to ensure that increased learner control will lead to better training outcomes, certain conditions should be met before trainees are granted this control (DeRouin, Fritzsche, & Salas, 2004; for a complete summary of these conditions, see the learner control guidelines in Table 4.2). First, steps should be taken to prepare trainees to take control. These steps may include providing instructions before training that describe the types and amount of control that trainees will be given during training and, more importantly, why they are given this control (Gay, 1986; Steinberg, Baskin, & Matthews, 1985). An example of how training instructions might fulfill these dual purposes is as follows: ―In this training course, you are free to select the context of training examples. In other words, because you are the best judge of your own learning style, this program offers you a choice of the types of examples you would like to view. Your choices

include business-related, military, and educational contexts. You can choose the example context that you believe will best help you to understand the topic in each example. This helps you to be in the driver‘s seat of your own learning experience‖ (DeRouin, Fritzche, & Salas, 2004, p. 152). By providing trainees with an understanding of the learner control they have and why this control is beneficial to their learning experience, they are more likely to accept and successfully utilize learner control within distance learning programs.

Table 4.2: Research-Based Guidelines for Offering Learner Control in Workplace e-Learning.

Guideline Description References

Preparing Trainees for Learner-Led Instruction

Understanding learner control is half the battle.

Give trainees instructions that allow them to understand the control they have and how that control can contribute to improved learning outcomes.

Gay, 1986; Hicken, Sullivan, & Klein, 1992; Steinberg, Baskin, & Matthews, 1985.

Give it time. Provide trainees with enough time in training to learn how to use learner control strategies.

Cronbach & Snow, 1981; Reeves, 1993.

Calibrate expectations.

Help trainees understand that adult training, especially learner-controlled training, is challenging.

Freitag & Sullivan, 1995; Gray, 1987.

Designing Learner-Controlled Training

Offer help. Trainees should be given tools during training that help them to diagnose their skill development.

Meta- cognitive/self- regulatory training: Osman & Hannafin, 1992; Schraw, 1998; self-tests/feedback: Brown & Ford, 2002; advisement/adaptive guidance: Bell & Kozlowski, 2002; Shyu & Brown, 1992; Tennyson, 1980.

What‘s good for one trainee may not be good for another.

Certain trainees may benefit more from learner control than others (for example, trainees with high ability, prior experience, and motivation).

Colquitt, LePine, & Noe, 2000; Gay, 1986; Kanfer & Ackerman, 1989; Lai, 2001; Tsai &

Table 4.2: Research-Based Guidelines for Offering Learner Control in Workplace e-Learning.

Guideline Description References

Tai, 2003.

More isn‘t necessarily better.

The amount of control given needs to be matched to the amount necessary for effective training; with too much control, trainees‘ cognitive resources may become tied up in decision making rather than training content.

Freitag & Sullivan, 1995; Gray, 1987.

―Skipping‖ is better than ―adding.‖

Allowing trainees to skip extra instruction rather than to add extra instruction during training may increase the amount of time spent on the optional portions of a program and still offer trainees control over the amount of instruction.

Hicken, Sullivan, & Klein, 1992.

Keep it real. Trainees may benefit from control over the context of their examples, such as nursing, sports, and so forth.

Ross, Morrison, & O‘Dell, 1989.

Footprints help (―You are here‖).

The training program may need to provide trainees with tools that allow them to pilot themselves through the program and utilize the control they have.

Cognitive maps: El-Tigi & Branch, 1997; Large, 1996; footprints, ―return‖ arrows and ―landmark‖ links: Nielsen, 1990.

Keep each instructional segment self- contained.

Trainees in learner-controlled training should not be required to remember too much material when transferring from one instructional segment to another.

Kearsley, 1988.

Share design control.

Trainees can be given some control over the program‘s design (for example, to open multiple windows at once and to control their size and location on the screen).

Kearsley, 1988; Park, 1991.

Be consistent. The design of the training program should be relatively consistent so that trainees are able to better focus on learner control decisions.

El-Tigi & Branch, 1997.

Create smooth transitions.

In learner-led instruction, transitions between instructional segments are important so that trainees understand how the segments are functionally related.

Park, 1991.

Table 4.2: Research-Based Guidelines for Offering Learner Control in Workplace e-Learning.

Guideline Description References

Creating Workplace

Promote it. Organizations can promote learner- controlled training through supervisor support of learner-led instruction.

Baldwin & Ford, 1988; Rouiller & Goldstein, 1993; Salas & Cannon- Bowers, 2000, 2001; Tannenbaum & Yukl, 1992.

Organizational climate matters.

Some organizational climates may be more supportive of learner-controlled training than others. Organizations should first assess employees‘ receptivity to learner-led instruction prior to its implementation.

Rouiller & Goldstein, 1993; Theory Y cultures: McGregor, 1957; need for future research.

Make it matter. Use of learner control strategies can be linked to organizational incentives to increase trainee motivation.

Colquitt, LePine, & Noe, 2000; Vroom, 1964; need for future research.

Source: Adapted from ―Optimizing e-learning: Research-based guidelines for learner- controlled training,‖ by R. E. DeRouin, B. A. Fritzsche, & E. Salas, 2004, Human Resource Management, 43, pp.150–151. Copyright 2004 by Wiley Publishers, Inc.

Second, the distance learning system should be designed so that trainees can effectively make use of the control they have. For instance, the learner control literature suggests that designing an instructional program so that learners are allowed to ―skip‖ instruction is more beneficial than designing a program so that they are allowed to ―add‖ extra instruction. Specifically, Hicken, Sullivan, and Klein (1992) found that learners who were given a comprehensive version of a program and were allowed to skip over certain topics viewed more examples and were more satisfied with the computerized learning program than learners who were given an abbreviated version of a program and were allowed to add extra topics. By allowing trainees to skip rather than add instructional topics, they are less likely to pass over important information in the program; moreover, trainees are likely to feel more responsibility for their learning because they chose which topics to view and which to skip. These conditions may lead trainees to perform better on the overall instructional program and to focus more on the training content.

Third, workplace conditions must facilitate learner control. For example, the science of training literature suggests that supervisor support has a profound impact on training effectiveness (Baldwin & Ford, 1988; Rouiller & Goldstein, 1993; Salas & Cannon- Bowers, 2000, 2001; Tannenbaum & Yukl, 1992). Because supervisor support reflects

how much the organization itself values training, trainees are more likely to invest themselves in the training experience if their supervisors consider training important.

Supervisor support of learner control is likely to have similar effects on its acceptance. If supervisors do not support learner control in distance learning, trainees are not likely to make use of this learning option. Therefore, it is important that supervisors promote the use of learner control in distance learning systems and its potential value in the learning experience.

Guideline 12: Guide trainees through the distance learning program.

Although the literature suggests that learners benefit from increased control over certain aspects of training, it is important that trainees still be given enough structure in the learning program that they are able to learn the skills necessary for effective performance on the job. The concept of ―discovery learning‖ has been heralded by educational psychologists as the instructional wave of the future (Shulman & Keisler, 1966). Its roots in constructivism, a theory of learning in which learners are considered active participants (Steffe & Gale, 1995), discovery learning refers to the idea that students learn best by guiding themselves through the learning process, making mistakes that eventually lead to a better cognitive understanding of the subject matter. However, Mayer (2004) in a recent article in American Psychologist, argued that discovery learning as a purely exploratory, undirected means of learning is less effective than discovery learning in which structure and guidance are provided for learners. Mayer cited three areas of research supporting his proposition: discovery of problem- solving rules (1960s), discovery of conservation strategies (1970s), and discovery of programming concepts (1980s). In his examination of each of these three research areas, Mayer found that guided discovery by an instructor was more successful than pure discovery in which students themselves chose what and how to learn. As a result, he concluded that, as an instructional method, discovery learning is primarily effective when learners are directed through the learning process.

Distance learning is one form of training that is particularly amenable to discovery learning (Allen, 2002). Its ability to link learning modules to outside websites and to provide information in virtually any form (for example, graphics, computer animation, simulation) makes it an ideal environment for trainee-directed instruction. However, Mayer‘s (2004) review of the literature on discovery learning suggests that distance learning may be more effectively designed and implemented with guidance designating which topics to study during training. The science of training suggests that the use of tools, such as advanced organizers, may help trainees to have a better understanding of the core elements of training and, simultaneously, to traverse a distance learning program more easily (Cannon-Bowers, Rhodenizer, Salas, & Bowers, 1998). In addition, adaptive guidance in the form of recommendations about the material on which to concentrate may be presented to trainees throughout the training program to help them remain focused on appropriate topics (Bell & Kozlowski, 2002).

Guideline 13: Make the program user-friendly.

A survey conducted by researchers at James Madison University revealed that 36 percent of respondents cited poor design and mismatch of learning styles as the primary reasons for abandoning e-learning programs (Mullich, 2004). Thus, many elements must be incorporated into distance learning programs to make them more user-friendly. First, distance learning courses should be designed by experts. These individuals not only must possess an understanding of course design, but they also must understand instructional design principles (Zeidman, 2003). Second, the course content of distance learning must be divided into smaller, more manageable sections. This can be accomplished by dividing the course into lessons spanning no longer than fifteen pages (Zeidman, 2003). Furthermore, each web page within the program should be limited to no more than two hundred words. These strategies help to ensure that learners are not overwhelmed by the sheer quantity of instructional material.

The quality of the course design can also be improved by including instructional pages at the beginning of each learning module (Zeidman, 2003). The first few pages of each course should include instructions regarding the use of the distance learning program. This procedure enables the user to practice navigating the interface before beginning course material. In addition, each page throughout the distance learning course should include links to a help page (Zeidman, 2003). This practice will make it easier for learners to solve problems as they progress through the course.

Future Needs in the Design and Delivery of Distance Learning Systems

In the creation of our distance learning guidelines, we found that much of the research literature focuses on how best to deliver distance learning to trainees and on the cost benefits of distance learning over traditional forms of instruction. However, little research is devoted to such topics as evaluation, what works in distance learning, standardization of the design and architecture of distance learning, and the application of distance learning to synthetic learning environments. Consequently, several needs exist for the future of distance learning research and practice.

First, applied researchers should conduct more robust evaluations of distance learning systems. The only way for organizations to truly know whether or not distance learning is effective is to experimentally evaluate the learning outcomes of trainees. Researchers, therefore, should move away from focusing primarily on the delivery of distance learning to focusing on whether trainees learn in distance learning systems. A few recent attempts have been made to evaluate whether or not distance learning is an effective learning approach in workplace organizations and educational institutions (Alavi, Marakas, & Yoo, 2002; Brown, 2001; Gopher, Weil, & Baraket, 1994; O‘Hara, 1990).

However, these research studies are few and far between. One reason may be that businesses traditionally do not allow researchers into their organizations to

systematically evaluate the effectiveness of their distance learning systems. However, if we are to gain a better understanding of the consequences and benefits of distance learning to workplace training, businesses must open their doors to applied researchers.

Second, researchers and practitioners must develop a catalog of what works in distance learning. Although several practitioners have created ―tips‖ and ―tactics‖ for offering distance learning in organizations (―Ten Tactics to Make e-Learning ‗Stick‘,‖ 2003), it is important that researchers and practitioners work together to produce an inventory of best practices in distance learning. Moreover, it is important that these best practices be linked to learning outcomes.

One attempt to bridge the gap between the research and practice of distance learning was Welsh, Wanberg, Brown, and Simmering‘s (2003) discussion of empirical and interview-based findings for distance learning. By combining a review of the distance learning literature with subject-matter expert interviews, they were better able to determine the most effective practices of distance learning initiatives and what practices are actually employed in workplace organizations. More alliances like this must be formed between researchers and practitioners in order to synthesize the results of empirical research with applied practice of distance learning systems.

Third, practitioners need to standardize the design and architecture of distance learning so that distance learning is both created and presented in a consistent format. This format should be based on the findings of the distance learning literature so that what has been found to be effective in distance learning research and practice is applied to its actual design and structure. The design and architecture, however, should be flexible and adaptable enough that organizations can incorporate their own learning techniques into the distance learning framework and can modify this framework to meet their specific organizational needs.

Fourth, researchers must study the application of synthetic learning environments to distance learning. Synthetic learning environments refer to the use of simulation and virtual reality in the training of complex tasks (Aidman, Galanis, Manton, Vozzo, & Bonner, 2002). Distance learning may involve the use of synthetic learning environments through the integration of simulation and virtual reality techniques into distance learning systems. The overall purpose of this integration is to make the learning experience more dynamic for trainees by having them interact with the program through hands-on tasks rather than text. As a result, trainees will likely learn more from the distance learning program than they would through more passive participation with the training material.

Conclusion

In their interview with subject-matter experts of distance learning programs, Welsh, Wanberg, Brown, and Simmering (2003) identified four themes about how distance learning will be applied in the future. These themes suggest that, in the next few years, synchronous learning tools will be employed more, blended learning approaches will

become more widespread, distance learning technology will be improved and will become more easily accessible, and several aspects of distance learning (for example, the presentation of information, performance support, training, and collaboration with peers) will be more appropriately integrated. Although these four themes imply that distance learning programs will improve over the next few years, researchers and practitioners would still benefit from a list of research-based guidelines for the effective design of distance learning programs.

Therefore, in this chapter, we offered thirteen guidelines for the design and delivery of distance learning programs. These guidelines were based on relevant empirical and theoretical research on distance learning from both educational and organizational settings and from the science of training literature. The guidelines may be viewed by researchers and practitioners as both tools for practice and as avenues for future research.

Because distance learning will be increasingly adopted and applied by organizations in the next few years (―The Payoffs of e-Learning,‖ 2003), it is important that organizations determine the most effective ways of designing distance learning programs. We hope that these guidelines will serve as a tool for the development of workplace distance learning courses as well as a stimulus for future research on the factors leading to distance learning success.

References

Aidman, E., Galanis, G., Manton, J., Vozzo, A., & Bonner, M. (2002). Evaluating human systems in military training. Australian Journal of Psychology, 54, 168–173.

Alavi, M., Marakas, G. M., & Yoo, Y. (2002). A comparative study of distributed learning environments on learning outcomes. Information Systems Research, 13, 404– 415.

Alavi, M., Yoo, Y., & Vogel, D. R. (1997). Using information technology to add value to management education. Academy of Management Journal, 40, 1310–1333.

Allen, M. (2002, January). Discovery learning: Repurposing an old paradigm. e- Learning, 3, 19–20.

ASTD & The Masie Center. (2001). e-Learning: “If we build it, will they come?” Alexandria, VA: ASTD.

Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for future research. Personnel Psychology, 41, 63–105.

Barton, K. (2001, January). Survey: Learning decisions benchmarking survey: The nuts and bolts of making e-learning. Learning Decisions Interactive Newsletter. [Retrieved March 2, 2004, from www.learning_decisions.com/login/site/default.cfm]

Bell, B. S., & Kozlowski, S.W.J. (2002). Adaptive guidance: Enhancing self-regulation, knowledge, and performance in technology-based training. Personnel Psychology, 55, 267–306.

Brayton, C. (2001, October 1). The learning curve. The fragmented e-learning industry rallies around a new business case: The value chain. Internet World. [Retrieved March 2, 2004, from www.internet_world.com/magazine.php?inc=100101/10.01.01ebusiness1.html]

Brown, K. G. (2001). Using computers to deliver training: Which employees learn and why? Personnel Psychology, 54, 271–296.

Brown, K. G., & Ford, J. K. (2002). Using computer technology in training: Building an infrastructure for active learning. In K. Kraiger (Ed.), Creating, implementing, and managing effective training and development (pp. 192–233). San Francisco: Jossey- Bass.

Burgess, J.R.D., & Russell, J.E.A. (2003). The effectiveness of distance learning initiatives in organizations. Journal of Vocational Behavior, 63, 289–303.

Cannon-Bowers, J. A., Rhodenizer, L., Salas, E., & Bowers, C. A. (1998). A framework for understanding pre-practice conditions and their impact on learning. Personnel Psychology, 51, 291–320.

Clark, R. C., & Mayer, R. E. (Eds.). (2003). e-Learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning. San Francisco: Pfeiffer.

Colquitt, J. A., LePine, J. A., & Noe, R. A. (2000). Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. Journal of Applied Psychology, 85, 678–707.

Cronbach, L. J., & Snow, R. E. (1981). Aptitudes and instructional methods: A handbook for research on interactions. New York: Irvington.

Davis, J. (2003). The magic of blending e-learning, classic training. Bank Technology News, 16, 58–59.

DeRouin, R. E., Fritzsche, B. A., & Salas E. (2004). Optimizing e-learning: Research- based guidelines for learner-controlled training. Human Resource Management Journal, 43, 147–162.

DeVeaux, P. (2001, April). War games: Digital game-based learning transforms military training. e-Learning, 2, 30–32.

Dobbs, K. (2000). What the online world needs now: Quality. Training, 37, 84–94.

El-Tigi, M., & Branch, R. M. (1997). Designing for interaction, learner control, and feedback during web-based learning. Educational Technology, 37, 23–29.

Ethics before it was fashionable. (2004, February). Corporate Legal Times, 14, 23.

Flood, J. (2002). Read all about it: Online learning facing 80% attrition rates. Turkish Online Journal of Distance Education, 3.

Freitag, E. T., & Sullivan, H. J. (1995). Matching learner preference to amount of instruction: An alternative form of learner control. Educational Technology, Research and Development, 43, 5–14.

Gale, S. F. (2003). Making e-learning more than ―pixie dust.‖ Workforce Management, 82, 112–122.

Gay, G. (1986). Interaction of learner control and prior understanding in computer- assisted video instruction. Journal of Educational Psychology, 78, 225–227.

Gill, M. (2000). e-Learning technology and strategy for organizations. In K. Fry (Ed.), The business of e-learning: Bringing your organization in the knowledge economy. Sydney, Australia: University of Technology.

Goodridge, E. (2001, April 23). e-Learning blends in with classrooms. Information Week, 834, 97.

Gopher, D., Weil, M., & Baraket, T. (1994). Transfer of skill from a computer game trainer to flight. Human Factors, 36, 387–405.

Gray, S. H. (1987). The effect of sequence control on computer assisted learning. Journal of Computer-Based Instruction, 14, 54–56.

Gray, S. H. (1989). The effect of locus of control and sequence control on computerized information retrieval and retention. Journal of Educational Computing Research, 5, 459–471.

Green, H. (2003, November 24). Online extra: Taking Mattel from ―atoms to electrons‖ [Electronic Version]. BusinessWeek.

Hicken, S., Sullivan, H., & Klein, J. (1992). Learner control modes and incentive variations in computer-delivered instruction. Educational Technology, Research and Development, 40, 15–26.

Hofmann, J. (2001, April). Blended learning case study. Learning Circuits. [Retrieved February 9, 2004, from www.learningcircuits.org/2001/_apr2001/Hofmann.htm]

Horton, W. (2002). Games that teach: Simple computer games for adults who want to learn. In A. Rossett (Ed.), The ASTD e-learning handbook. New York: McGraw-Hill.

Horton, W. K. (2000). Designing web-based training: How to teach anyone anything anywhere anytime. Hoboken, NJ: John Wiley & Sons.

Jasinski, M., & Thiagarajan, S. (2000). Virtual games for real learn-_ing: Learning online with serious fun. Educational Technology, 40, 61–63.

Judd, W. A., Bunderson, C. V., & Bessent, E. W. (1970). An investigation of the effects of learner control in computer-assisted instruction prerequisite mathematics (MATHS) (Report No. TR-5). Austin, TX: Computer-Assisted Instruction Laboratory. (ERIC Document Reproduction Service No. ED053532)

Kanfer, R., & Ackerman, P. L. (1989). Motivation and cognitive abilities: An integrative/aptitude-treatment interaction approach to skill acquisition. Journal of Applied Psychology-Monograph, 74, 657–690.

Kaplan-Leiserson, E. (2002). e-Learning glossary. [Retrieved March 2, 2004, from www.learningcircuits.org/glossary.html]

Kearsley, G. (1988). Authoring considerations for hypertext. Educational Technology, 28, 21–24.

Kiser, K. (2002, July). Taking it to the bank: How the Royal Bank of Scotland group led one of the United Kingdom‘s first and largest online learning initiatives—and has been able to cash in on the results. e-Learning, 3, 32–36.

Kosarzycki, M. P., Salas, E., DeRouin, R., & Fiore, S. M. (2003). Distance learning in organizations: A review and assessment of future needs. In D. Stone (Ed.), Advances in human performance and cognitive engineering research: Vol. 3. Human resources technology (pp. 69–98). Greenwich, CT: JAI.

Kozlowski, S.W.J., Toney, R. J., Mullins, M. E., Weissbein, D. A., Brown, K. G., & Bell, B. S. (2001). Developing adaptability: A theory for the design of integrated-embedded training systems. In E. Salas (Ed.), Advances in human performance and cognitive engineering research (Vol. 1, pp. 59–123). Amsterdam: JAI/Elsevier Science.

Lai, S. (2001). Controlling the display of animation for better understanding. Journal of Research on Computing in Education, 33. [Retrieved February 25, 2004, from www.iste.org/jrte/33/5/lai.cfm]

Large, A. (1996). Hypertext instructional programs and learner control: A research review [Electronic version]. Education for Information, 14, 95–106.

Leonard, B. (1996). Distance learning: Work and training overlap. HR Magazine, 41, 41–47.

Linne, J., & Plers, L. (2002, November). The DNA of an e-tutor. ITTraining, 26–28.

Masie, E. (2002). Blended learning: The magic is in the mix. In A. Rossett (Ed.), The ASTD e-learning handbook. New York: McGraw-Hill.

Mayer, R. E. (1989). Systematic thinking fostered by illustrations in scientific text. Journal of Educational Psychology, 81, 240–246.

Mayer, R. E. (2004, January). Should there be a three-strikes rule against discovery learning? The case for guided methods of instruction. American Psychologist, 59, 14–19.

Mayer, R. E., & Anderson, R. B. (1991). Animations need narrations: An experimental test of a dual-processing system in working memory. Journal of Educational Psychology, 90, 312–320.

Mayer, R. E., & Gallini, J. K. (1990). When is an illustration worth ten thousand words? Journal of Educational Psychology, 88, 64–73.

Mayer, R. E., Steinhoff, K., Bower, G., & Mars, R. (1995). A generative theory of textbook design: Using annotated illustrations to foster meaningful learning of science text. Educational Technology, Research and Development, 43, 31–43.

McGregor, D. M. (1957). The human side of enterprise. Management Review, 46, 22– 28.

Moe, M. T., & Blodget, H. (2000). The knowledge web. Merrill Lynch & Co. Merrill Lynch—eLearning: The knowledge web part 4 corporate e-learning—feeding hungry minds. [Retrieved February 25, 2004, from www.usc.edu/dept/education/globaled/hentschke/documents/_KW4.pdf]

Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91, 358–368.

Moshinskie, J. (2001). How to keep e-learners from e-scaping. Performance Improvement, 40, 28–35.

Mullich, J. (2004). A second act for e-learning. Workforce Management, 83, 51–55.

Nielsen, J. (1990). The art of navigating through hypertext. Communications of the ACM, 33, 296–310.

O‘Hara, J. M. (1990). The retention of skills acquired through simulator-based training. Ergonomics, 33, 1143–1153.

OneTouch Systems, Inc. (2003). FAQ: About e-learning. OneTouch Systems Corporate Website. [Retrieved on February 8, 2004, from www.one_touch.com/aboutel.html]

Osman, M. E., & Hannafin, M. J. (1992). Metacognition research and theory: Analysis and implications for instructional design. Educational Technology, Research and Development, 40, 83–99.

Park, O. (1991). Hypermedia: Functional features and research issues. Educational Technology, 31, 24–31.

The payoffs of e-learning go far beyond the financial. (2003). HR Focus, 80, 7–8.

Phillips, J., Phillips, P. P., & Zuniga, L. (2000). Evaluating the effectiveness and the return on investment of e-learning. What Works Online: 2000, 2nd Quarter. American Society for Training and Development. [Retrieved March 2, 2004, from www.astd.org/ASTD/Resources/_eval_roi_community/return.htm]

Reeves, T. C. (1993). Pseudoscience in computer-based instruction: The case of learner control research. Journal of Computer-Based Instruction, 20, 39–46.

Ross, S. M., Morrison, G. R., & O‘Dell, J. K. (1989). Uses and effects of learner control of context and instructional support in computer-based instruction. Educational Technology, Research and Development, 37, 29–39.

Ross, S. M., & Rakow, E. A. (1981). Learner control versus program control as adaptive strategies for selection of instructional support on math rules. Journal of Educational Psychology, 73, 745–753.

Rouiller, J. Z., & Goldstein, I. L. (1993). The relationship between organizational transfer climate and positive transfer of training. Human Resource Development Quarterly, 4, 377–390.

Russell, T. L. (1999). The no significant difference phenomenon. Raleigh: North Carolina State University.

Salas, E., & Cannon-Bowers, J. A. (2000). Design training systematically. In E. A. Locke (Ed.), The Blackwell handbook of principles of organizational behavior. Oxford, UK: Blackwell Publishers Ltd.

Salas, E., & Cannon-Bowers, J. A. (2001). The science of training: A decade of progress. Annual Review of Psychology, 52, 471–499.

Salas, E., Kosarzycki, M. P., Burke, C. S., Fiore, S. M., & Stone, D. L. (2002). Emerging themes in distance learning research and practice: Some food for thought. International Journal of Management Reviews, 4, 135–153.

Salopek, J. J. (2002, February). Virtually face-to-face: Synchronous e-learning defies bandwidth barriers. e-Learning, 3, 16–19.

Schelin, E., & Smarte, G. (2002, April). Recognizing the champions: Real-life tales of e-learning triumph and innovation. e-Learning, 3, 12–24.

Schraw, G. (1998). Promoting general cognitive awareness. Instructional Science, 26, 113–125.

Selix, G. (2001). Improving blended learning. [Electronic version]. e-Learning, 2, 48.

Shepherd, C. (2003, September). e-Learning‘s neither a threat nor a panacea. ITTraining, 34.

Shulman, L. S., & Keisler, E. R. (Eds.). (1966). Learning by discovery. Chicago: Rand- McNally.

Shyu, H., & Brown, S. W. (1992). Learner control versus program control in interactive videodisc instruction: What are the effects in procedural learning? [Electronic version]. International Journal of Instructional Media, 19, 85–96.

Skipper, J. (2000, April). Distance learning barriers: ASTD study. DL News, IBM Corporation, p. 1.

Steffe, L. P., & Gale, J. (Eds.). (1995). Constructivism in education. Mahwah, NJ: Lawrence Erlbaum.

Steinberg, E. R., Baskin, A. B., & Matthews, T. D. (1985). Computer-presented organizational/memory aids as instruction for solving Pico-fomi problems. Journal of Computer-Based Instruction, 12, 44–49.

Tannenbaum, S. I., & Yukl, G. (1992). Training and development in work organizations. Annual Review of Psychology, 43, 399–441.

Ten tactics to make e-learning ‗stick‘ in your dept. or center. (2003, June). IOMA‘s Report on Customer Relationship Management, 4–6.

Tennyson, R. D. (1980). Instructional control strategies and content structure as design variables in concept acquisition using computer-based instruction. Journal of Educational Psychology, 72, 525–532.

Tsai, W., & Tai, W. (2003). Perceived importance as a mediator of the relationship between training assignment and training motivation. Personnel Review, 32, 151–163.

Vroom, V. H. (1964). Work and motivation. Hoboken, NJ: John Wiley & Sons.

Webster, J., & Hackley, P. (1997). Teaching effectiveness in technology-mediated distance learning. Academy of Management Journal, 40, 1282–1309.

Welsh, E. T., Wanberg, C. R., Brown, E. G., & Simmering, M. J. (2003). e-Learning: Emerging uses, empirical results and future directions. International Journal of Training and Development, 7, 245–258.

Zeidman, B. (2003). Guidelines for effective e-learning. Chief Learning Officer, 2, 24– 28.