MODULE 650

JOURNAL OF RESEARCH ON TECHNOLOGY IN EDUCATION, J9(4), 331-357

Examining the Development of a Hybrid Degree Program: Using Student

and Instructor Data to Inform Decision-Making

Audrey Amrein-Beardsley, Teresa S. Foulger, and Meredith Toth Arizona Stale University

Abstract This paper investigates the qtiestions and considerations that should be discussed by administrators, faculty, and support staff when designing, developing and offering a hybrid (part online, part face-to-face) degree program. Using two Web questionnaires, data were gathered from nine instructors and approximately 450 students to evaluate student and instructor perceptions and opinions of hybrid instruction and activities. In comparison to prior research, the results of this study offer larger and more significant policy and programmatic implications for degrees based on the hybrid format, including instructional technology training and support for students and instructors, creation of common class procedures and expectations, and development of consistent schedules that maximize benefit and flexibility for stttdents and instructors. (Keywords: hybrid, online, degree program, communities of practice, teacher education, organizational change.)

INTRODUCTION While online learning has become the focus of much research and debate

regarding its efficacy in meeting or exceeding student learning outcomes (Neuhauser, 2002; Russell, 1999; Skylar, Higgins, Boone, Jones, Pierce, & Gelfer, 2005; Summers, Waigandt, & Whittaker, 2005), hybrid courses have been largely treated as a subset of distance education and are seldom examined as a unique method of course delivery. Due to the development of readily available technologies, the potential of hybrid instruction as a model that combines these new technological applications with more traditional approaches to education has been recognized (Anastasiades & Retalis, 2001). While literature exists evaluating online courses (Benbunan-Fich & Hiltz, 2003; DeTure, 2004; Overbaugh & Lin, 2006), online degree programs (Benson, 2003; Snell & Penn, 2005; Wilke & Vinton, 2006), and hybrid courses (Donnelly, 2006; Leh, 2002; Riffell & Sibley, 2005), little has been published specific to the design opportunities made available by hybrid degree programs.

Recent studies by the National Center for Education Statistics (Waits & Lewis, 2003) and The Sloan Consortium (Allen & Seaman, 2006) show a growing appeal and acceptance of online learning. However, little is understood about effective program design when multiple courses are linked in a formal degree program.

Drawn by the appeal of a model that combines the flexibility of online learning with the benefits of in-class meetings and activities, a teacher education college in a university in the southwest United States chose to investigate

Journal of Research on Technology in Education 331

the hybrid model as a new delivery method for its teacher preparation undergraduate degree program. Utilizing a survey research, mixed-methods approach, this study was largely exploratory in nature and sought to answer the following research question: What policy and programmatic issues should be discussed by administrators, faculty, and support staff when designing, developing and offering a hybrid degree program?

Through an analysis of student and instructor perceptions of hybrid course design and instruction coupled with administrative directives, the researchers sought to understand the concerns of each group. This study documents the knowledge brokered between students, instructors and administrators, and provides information to stakeholders that will inform degree program decisions and promote common practices across classes.

LITERATURE REVIEW

Compared to other areas of education research, the field of online learning is still relatively new, and consistent definitions or methods of categorization have yet to be established. Classifications of online learning vary in a number of ways, such as the technologies employed (Garrison, 1985), teaching and learning methods (Misko, 1994), pedagogical approaches (Dziuban, Hartman & Moskal, 2004), and where the design lies on the continuum from fully face- to-face to fully online (Allen & Seaman, 2005; Twigg, 2003). Some scholars do not draw such clear distinctions and instead describe as "hybrid" any course that combines traditional face-to-face instruction with online technologies (Swenson & Evans, 2003).

For the purposes of this study, the researchers use the hybrid terminology already in use by our university administration. This definition aligns with that of the Sloan Consortium (Allen &: Seaman, 2006) as a delivery method that blends face-to-face and online instruction. More particularly, it aligns with Twigg's hybrid model, which offers a more specific definition referring to the "replacement" of traditional class time with out-of-class activities such as Web-based resources, interactive tutorials and exercises, computerized quizzes, technology-based materials, and technology-based instruction (Twigg, 1999).

To facilitate the transition from traditional face-to-face to hybrid courses, Aycock, Garnham, and Kaleta (2002) recommend instructors start small by redesigning an activity or unit of a course, then augment the process in subsequent semesters. When multiple hybrid courses are fully implemented, the hybrid degree program will accommodate the needs of today's students by offering a program that is accessible and flexible (Bonk, Olson, Wisher, & Orvis, 2002; Graham, Allen, & Ure, 2003; Sikora, 2002). This is particularly relevant when students taking multiple courses in a given semester attempt to schedule classes and internships in ways that support demands on their time.

Over the last several decades, most research on courses that blend face-to-face and technology-mediated instruction has focused on the way technologies such as audio recordings (LaRose, Gregg, & Eastin, 1998), television (Machtmes & Asher, 2000), computer conferencing (Cheng, Lehman, & Armstrong, 1991), or course management systems (Summers, Waigandt, & Whittaker,

332 Summer 2007: Volume 39 Number 4

2005) can be used to provide instruction as effective as that of a traditional face-to-face classrooms. Literature specific to hybrid courses has followed this trend and also reveals an emphasis on student achievement (Boyle, Bradley, Chalk, Jones, & Pickard, 2003; McCray, 2000; Olapiriyakul & Scher, 2006; O'Toole & Absalom, 2003) or the affective factors most valued by students or instructors in hybrid courses (Ausburn, 2004; Bailey & Morais, 2004; Parkinson, Greene, Kim & Marioni, 2003; Woods, Baker, & Hopper, 2004). More recently, attention has shifted from the technology itself to an emphasis on the pedagogical approaches that should lead the way (Bennett & Green, 2001; Buckley, 2002; Reeves, Herrington, & Oliver, 2004; Twigg, 2001).

Adding online technologies complicates instruction. Quality online instruction must incorporate learning theory and practices from traditional face-to-face courses as well as effective pedagogical use of technology (Yang & Cornelious, 2004). Since instructors rely on a number of factors to accomplish their programmatic goals, those that contribute to successful instructional design and delivery are difficult to pinpoint in degree programs, whether online, hybrid, or face-to-face (Moore, 1993).

Yet, if institutions interested in exploring hybrid delivery focus only on the design and delivery of individual course offerings, problems such as disjointedness, a lack of "program" focus, and overall poor quality can arise from neglecting to examine the program as a whole (Husmann & Miller, 2001). Limited knowledge is available regarding the programmatic implications of hybrid design (Phipps & Merisotis, 1999), the focus of this study.

As allies in the learning process, faculty and administrators must take time to identify the factors influencing student satisfaction, adapt coarse design and structure to meet diverse student needs, and actively engage in the learning process with students (Young, 2006). The present study seeks to fill this gap in the literature by understanding administrative directives and gathering input from student and instructor communities to identify the larger and more significant policy and programmatic implications related to designing and developing hybrid degree programs.

THEORETICAL ERAMEWORK Participation in Communities of Practice

Within any organization, groups of people associated with a common practice naturally come together to share success and failures and brainstorm new ideas. This is a naturally occurring phenomenon of a healthy system (Wenger, 1998). Rogers (2002) observed that although opportunities for individualized learning are increasing, there are significant advantages to group learning. Although struggles are more likely to arise within groups and group work requires certain levels of maturity among participants (Goleman, 1995; Mezirow, 2000), there are definite advantages for groups in the learning process, including (a) groups can provide a supportive environment, (b) groups create challenges unavailable in isolated learning situations, (c) groups build more complex cognitive structures due to the representation of a variety of experiences, and (d) groups are dynamic and can become a community of practice as they draw in members (Rogers, 2002).

Journal of Research on Technology in Education 333

The Communities of Practice learning theory (CoP) encompasses these elements of collaboration within groups and organizational systems. In a single CoP, members represent unique experiences and knowledge, but unite for the purpose of improving their common practice. These collaborative experiences form naturally based on the needs of the participants (Sumsion &C Patterson, 2004). Once formed, the participants develop ways of maintaining connections within and beyond their community boundaries (Sherer, Shea, & Kristensen, 2003). Constituencies outside the CoP might include those at various levels within the organization, some outside of the organization, and newcomers attempting to enter the CoP. When individuals are involved in multiple CoPs, transfer of knowledge from one CoP to the other can occur. It is difficult, however, for newcomers in unfamiliar communities to understand the community workings as fully as long-standing members (Brown & Duguid, 2000; Lave & Wenger, 1991; Wenger, 1998).

Boundary Brokers and Trajectories In some cases, CoP members can take on the role of boundary brokers to

expedite organizational change (Sherer, Shea & Kristensen, 2003). When members of a community exist on the periphery and broker information with another CoP, a boundary trajectory occurs (Wenger, McDermott, & Snyder, 2002). In such cases, the links between the CoPs cause boundaries to expand and create a practical mechanism for greater understanding between communities (Iverson & McPhee, 2002). In this way, boundary brokers seamlessly expand access to resources within relevant communities (Sherer, Shea, & PCristensen, 2003), especially in organizations that nurture membership in multiple communities (Kuhn, 2002). However, it is a very delicate challenge to sustain an identity in this type of social setting, as those who translate, coordinate, and align perspectives through ties to multiple communities must be able to legitimately influence the "development of a practice, mobilize attention, and address conflicting interests" (Kuhn, 2002, p. 109).

Although organizations can support infrastructural investment for CoPs, CoPs fijnction best when members engage in authentic interactions and negotiations based on the needs of the members. These needs bring them together in a meaningful way surrounding their individual identities, roles, intentions, realities, and agendas (Thompson, 2005). This balance between administrative or professional development forces and the organic needs of members that choose to engage in the inquiry process reaffirms the need for a professional development environment that embraces CoP functions and empowers CoP members (Cousin &: Deepwell, 2005; Foulger, 2005; Thompson, 2005).

Situating This Study As part of a college initiative to explore new modes of delivering degree

programs, the college dean approached the Elementary Education department chair (the largest department in the college) and one technology instructor with the charge of creating capacity" to offer online courses. To develop and evaluate the courses, the technology instructor solicited guidance from

334 Summer 2007: Volume 39 Number 4

/ Student CVwiifniifiity

V "-< Instructor

CoP

--- --'

dmintfiRtTiition

CoP

j BOUNDARY BROB^ERING

Figure I. Findings from this study were drawn from the convergence of student, instructor, and administrator perspectives.

information technology administrators, instructional design support personnel, college administrators, department chairs, instructors, and students. After consulting with these stakeholders, the college offered a two-day intensive seminar on designing and developing hybrid courses.

Sixteen instructors, including the Elementary Education department chair, volunteered to participate in the hands-on seminar and redesign a two-week component of one of their face-to-face courses as a hybrid unit offered half online and half face-to-face. All of the instructors were proficient with online technology tools and received additional training in hybrid course design and instruction, but they had never taught online before. I h e instructors collaborated to redesign their units using asynchronous technologies that employed Blackboard tools and methods (Blackboard, version 6.2, the university-sponsored course management system).

Because communities of practice are not necessarily fixed systems, and because each interaction among members has a multitude of influences (Wenger, 1998), a prescriptive vision for the hybrid program could not be determined at the conception of this hybrid investigation. This lack of rigidity was embraced by instructors participating in the study.

From the CoP perspective, the hybrid instructors in this study negotiated a balance between the identities associated with three specific social forces (see Figure I). The following issues were expressed prior to the beginning of this study and were used to inform the development of the hybrid design:

• Administration Community of Practice: Administrators were most concerned with decreasing use of classroom space, providing training and

Journal of Research on Technology in Education 335

support to hybrid instructors, and creating incentives for participation. Instructors served as peripheral participants and advisors to the Administrative CoP at the onset of the study by communicating the need to develop policies and procedures supportive to the transformation of a face-to-face to hybrid degree program.

• Hybrid Instructor Community of Practice: Teacher education instructors who elected to redesign a previously-taught course into a hybrid course were initially concerned with maintaining high standards and student accountability, assuring that technology would be used to enhance instruction, and understanding which activities were best suited for face- to-face or online environments.

• Hybrid Student Community: Instructors initially knew very little about the student perspective. However, they realized the importance of brokering knowledge from the student community as a way to understand their perspective and use the information to influence instructor and administrative decisions.

As the college devised initial plans for the development of the hybrid program and began implementation, purposefully exchanging information between these three critical stakeholder groups led to a greater understanding of the realities of each group. These initial conversations brought about a broader understanding of the contributing practices of administrators and instructors believed to be critical for student success in the hybrid degree program. Through the methods employed in this study, the researchers probed the instructor and student CoPs more deeply to determine the most effective practices and how this knowledge could inform the administrative CoP to advance the hybrid program.

METHODS Data reported in this study were collected from instructors and students as

they experienced the college's first attempt at transforming traditionally face-to- face instruction to a hybrid format.

Instructor Sample After completing the seminar on hybrid course design and instruction, nine

of the 16 instructor participants (56%) committed to teaching their hybrid unit the following semester. At the conclusion of their units, all nine instructor participants completed the online Instructor Hybrid Evaluation Questionnaire (see Appendix), designed to capture instructors' perceptions of their students' and their own experiences with the hybrid unit. One instructor completed the questionnaire twice for two different courses (response rate = 100%).

Student Sample Following the directions of the primary researchers in this study, instructor

participants distributed the online Student Hybrid Evaluation Questionnaire (see Appendix) to their students who participated in their hybrid unit of instruction. To assure a high response rate, each instructor solicited participation directly from their students by explaining to students that their

336 Summer 2007: Volume 39 Number 4

feedback would help improve the overall program, particularly for fliture students. Each of the nine instructors distributed the questionnaire directly to their students. Some students participated in more than one course where hybrid units were offered; these students were encouraged to take the questionnaire multiple times based on their unique experiences in each course. In cases where the relative response rate was of concern, students were sent one reminder to participate.

A total of 413 out of approximately 450 students completed the online questionnaire (response rate = 92%). The high response rate is probably due to the fact that students completed the anonymous online questionnaire during normal class time or were held accountable for their participation, predominantly through class credit.

Instrument Rather than examining success factors for students in these courses, two

complimentary Web questionnaires were designed to gather information regarding student and instructor perspectives of the hybrid instruction and activities, the hybrid degree program, and course planning and design (Benson, 2002). Similar questionnaire forms allowed for comparative analyses between instructor and student participants and more holistic analyses across groups.

Part I of both the instructor and student questionnaires collected general demographic, technology access, and course and programmatic information. Part II presented instructors and students with a list of technology tools provided within Blackboard. If tools were used, instructors and students were asked to respond to Likert-type items indicating the extent to which the tools enhanced a) the instructor participants' perceived abilities to provide quality instruction and b) the student participants' perceived abilities to learn.

Part III, Section 1 asked instructors and students to indicate their levels of agreement with statements about affective factors of hybrid instruction. This section was adapted from materials provided online as part of the Hybrid Course Project at the University of Wisconsin-Milwaukee (Learning Technology Center, 2002). To encourage students and instructors to read and reflect on each statement and decrease the likelihood that they would select the same value for continuous items, positive and negative statements were placed in a randomized sequence. Part III, Section 2 asked instructors and students to indicate their overall levels of agreement regarding face-to-face and online environments.

Part IV asked students and instructors to provide insights they thought would be useful to instructors and the college regarding online activities, hybrid course development, and hybrid degree program development.

Instrument Internal-Consistency Reliability Estimates of reliability were calculated for each section of the student and

instructor Web questionnaires. Coefficient-alpha estimates of internal- consistency reliability were computed for Parts II and III (Cronbach, 1951). Coefficient-alpha estimates for the positive and negative statements built into

Journal of Research on Technology in Education

Table 1: Coefficient Alpha Estimates of Reliability

Part II:

Part III Factors

Part III Factors

Blackboard Tools

, Section 1: Affective and Personal

, Section 2: Overall Agreeability

Student Web Questionnaire

0.724

0.718

0.853

Instructor Web Questionnaire

0.791

0.828

0.744

Part III, Section 1 were adjusted so that responses could be interpreted on the same scale, and inversely related estimates would not cancel each other out. All sections of the Web questionnaires yielded acceptable alpha levels (see Table 1 for coefficient-alpha levels of both instruments) and warranted their use for the purposes of this research study. Values below .70 are oft:en considered unacceptable (Nunnally, 1978).

Methods of Data Analysis Frequency statistics were used to analyze each demographic, course, and

programmatic question in Part I of both Web questionnaires. For Parts II and III, descriptive statistics were calculated using participant responses to the Likert items, and means were rank ordered to illustrate levels of participant agreement per item. T-tests using independent samples were also used to test for significant differences between the opinions of instructor- and student- participant groups.

Participant responses to the open-ended, free-response items in Part IV were read, coded, and reread, and emergent themes were categorized into bins (Miles & Huberman, 1994). Once bins became focused and mutually exclusive in nature, the items included within each bin were collapsed into categories, quantified, and labeled. Overall themes were validated by instructor participants during a focus group conducted by the researcher participants, and the themes were left: intact, without any additions or deletions. These themes will be discussed further in the Implications section of this study.

RESULTS Part I: Demographic Information and Technology Access

In Part I of the Web questionnaire investigators gathered demographic, technology access, and course and programmatic information from student and instructor participants. More than 60% of student participants primarily used a personal desktop computer to complete coursework. About 20% of student participants used portable laptops, and 10% completed online lessons and assignments on campus at the student computer center or the library. Approximately 90% of student participants accessed the Internet through a high-speed connection, while about 10% relied on dial-up networks.

Students reported that an average of 3.7 of their courses (out of a maximum of five courses students may take each semester) involved some hybrid

338 Summer 2007: Volume 39 Number 4

Online gradebook

Course document downloads

Internet sites/links

E-maii between instructor and

student

E-mail between student and student

SmatI gnsup discussion board

Full class discussion board

Online assignment submission

Online Qutzzes/Tests

Digital drop box

-

•̂1

.0

^ ^

122

•̂1

12,/

4

I!?

35 4

136

13.3

3 0

4

|;!8

0.0 0 5 1.0 1.5 2.0 2 5 3 0 3 5 4 0 4 5 5 0

]

Figure 2. Blackboard tools ranked by students and instructors from, most to least useful.

component during the semester of study. Instructor participants indicated that they replaced an average of six face-to-face classes (out of approximately thirty total instructional days) with online instruction. The total number of face-to- face days replaced with online instruction ranged from a low of two to a high of 10 days.

Part II: Student and Instructor Perceptions of Blackboard Learning Tools In Part II of the Web questionnaire, student and instructor participants

identified the Blackboard tools they found most and least useful in terms of enhancing student learning in the hybrid format. The closer each item mean is to 5, the more the student or instructor participants agreed with each statement. For the purposes of this study, the results from this section are used to provide larger programmatic considerations and recommendations (see Figure 2).

Of the Blackboard tools identified in the Web questionnaire, students found the online grade book and announcements most useful. Students appreciated instructors who graded assignments and posted them in the grade book in a timely and efficient manner and criticized instructors who did not use the grade book effectively or did not post grades soon after reviewing student work. Students appreciated that they could monitor their progress in courses using the grade book and thought that more college instructors should use the tool.

Journal of Research on Technology in Education 339

Although students appreciated the use of announcements, almost 50% of student participants expressed a need for instructors to be consistent with announcement frequency and to provide clear and simple written information. Students also requested that instructors e-mail students after posting an announcement, particularly if announcements are not used as part of the normal class routine.

Students found the course document downloads, Internet sites and links, and e-mails sent to them from the instructor equally useful in terms of technology tools that enhanced their learning. Some students expressed concern regarding their ability to find or download course documents and others had difficulty visiting and spending time on Internet sites if they had only dial-up access.

Students appreciated when instructors e-mailed them to clarify components of the coursework and most appreciated instructors who responded to student e-mails in a friendly, "timely" manner. Students were very critical of instructors who did not respond to student e-mails in a "timely" manner, responded in an unfriendly manner, or did not respond at all. Students questioned whether instructors who do not respond to e-mails in such a manner should be implementing online activities in their courses. Because students do not meet as often in a hybrid setting, the primary communication method between students and instructors is e-mail. When instructors did not respond in a timely manner, students expressed high levels of frustration and outright anger.

In general, students felt that discussion boards were more useful than in- class discussions because students could take their time to compose a response, students were required to participate online while they were not required to participate in face-to-face discussions, and students who normally do not participate in class were not as reluctant to express an opinion online. Students also found small-group discussion boards to be particularly useful when quizzes and tests required them to use the knowledge gained from such discussions. Despite these benefits, students felt that discussion board assignments sometimes became redundant, were not always useful, and sometimes detracted from more important course activities or assignments.

Instructors disagreed with their students in two ways. First, instructors found the Internet sites and links and the full class discussion board to be significantly more useful (p < .05) than their students found these technology features. Second, instructors found student-to-student e-mail, online assignment submissions, course document downloads, small group discussion boards, and online quizzes and tests as significandy less useful (p < .05) than their students found these technology tools.

Part III: Student and Instructor Responses to Affective Items In Part III of the Web questionnaire, student and instructor participants

indicated their level of agreement with thirteen affective statements about hybrid instruction. The closer each item mean is to 5, the more the student or instructor participants agreed with each statement (see Figures 3, 4 and 5).

Of the first 10 statements (Section 1), five were written in a favorable vernacular and five were written in an unfavorable vernacular. For this reason,

340 Summer 2007: Volume 39 Number 4

Because of Ihe online oonrponents in tiis course. I was (rry students vwre) betsf able k> balance nv

(their) couneMKk Mth olher home

s in tijs course helped me (them) leam more about t i e

I h e tochmlogy U5K) enhanced iTV (nv students) understanding of the

ccurseworti

ltbundthatlv«Bs(nv students M r e ) able to conW tie paoe of nv

(tieir) teaming rrore efle(*/Bly because of the May Ihis course used

online t o l s

I (My student5) Ibund that I ves (they were) beilBf able to dewtcp

m/ (tierr) conmjnicatKxi slqlls because of Ihe technology tools

used.

Figure 3. Instructor and student responses to favorable, affective questions.

results have been split into two sections and ranked from high to low levels of agreement.

Students agreed that the online components of their classes helped them balance their coursework with other home and/or work responsibilities and learn more about subject matter. Students most disagreed that they had to spend too much time trying to get access to a computer to do the coursework effectively, and that they were at a disadvantage because they did not understand how to use the technology tools as well as the other students. If the response rate had been lower, use of a Web questionnaire might suggest that students with technology issues were underrepresented in the sample of students who participated; however, this was not the case. Students were most ambivalent (mean = 2.5) towards whether online learning was better than learning in a face-to-face environment.

Instructors viewed the impact of online instruction on their students' learning significantly more favorably than did their students. Instructors were significantly more concerned than students with whether some students were disadvantaged by a lack of technology skills. Instructors were significantly less concerned than students with whether the time spent online would have been better spent in the classroom and whether online experiences made students feel less connected with their instructors (p < .05).

Part III, Section 2 included three overarching, open-ended questions designed to capture student and instructor participants' overall opinions and suggestions

Journal of Research on Technology in Education 341

The tme i (my students) spent oniine MHjkl have been beOer spent in ihe

dassroom

•Rie technology tools made me (my stu<Jent5) feel less connecW wth

the instructor (me as their instructor)

The technology tools made me (my students) *Bei less osnnected with Ihe oiher students in this course

I felt tiat I was (some students were) at a disadvantage in Ihis course

because I (Ihey) didni understand how to use the technology tools as

well as other students

I (My students) had to spend too mudi time trying to get acoess to a

a)mputer to do the coursevwrk etfectiwiy.

Figure 4. Instructor and student responses to unfavorable, affective questions.

regarding hybrid instruction. Each item mean is illustrated. The closer each mean is to 5, the more the student or instructor participants agreed with each statement.

Overall, students and instructors agreed that it would be a good idea if the entire teacher education program involved face-to-face and online activities and if other courses incorporated more online activities. They also believed that the content of the courses was well suited for a combination of face-to-face and online activities. Instructors agreed at higher levels, but students and instructors ranked the three statements in the same order by similar levels of agreement.

Part IV: Student Responses to Open-Ended Questions In Part IV of the Web questionnaire, student and instructor participants

were asked to provide information or insights they thought would be useful to instructors and the college regarding online activities and hybrid course development.

In response to the request for information or insights they thought would be useful to their instructors regarding hybrid activities, student participants responded with enthusiasm for increasing hybrid courses across the college, with the stipulation that the hybrid components be beneficial to students and that assignments be of reasonable length and pertinent to the students' professional development. Students requested that instructors plan online/in- class schedules in collaboration with other instructors to maximize fiexibility

342 Summer 2007: Volume 39 Number 4

O«rall.llhink the content of this course is well-suited for a

a)mbinabon of boe-to-face and oriiineactivttei

Overall. I timk it would be a good idea if olher courses would

incorporate more online advities

Overall.lMnkitwwIdbeagood idea if Ihe entire program i n w l ^ face-to-lace and online activites

45

0.0 0.5 10 1.5 2.0 25 3.0 35 4.0 45 50

oinalructors

Figure 5. Instructor and student responses to overarching, open-ended questions.

and minimize confusion. In addition, students felt that the online/in-class schedule should be organized and disclosed to students at the outset of a course so they would have the opportunity to opt out of a course with online components when scheduling their semesters. In addition, students expressed frustrations with some technologies (such as trial software) they felt compromised their opportunities to succeed in an online learning environment.

Instructor participants suggested that all instructors hold students accountable for the online work associated with any given course while maintaining a certain degree of flexibility, especially given students' busy schedules and the challenges they might face in learning new technologies. Instructors also noted that hybrid activities should not create additional work for students, but should replace less valuable work normally conducted in a face-to-face setting. Finally, instructors recommended that all instructors be clear, organized, responsive, and timely when responding to e-mail and other student communications, such as discussion boards.

The Web questionnaire also prompted students for information or insights they thought would be useful to the college regarding hybrid or online activities. A strong majority of students responded favorably towards hybrid instruction, but stated the college should proceed with caution. Approximately 10% of student participants did not encourage the college to offer more hybrid

Journal of Research on Technology in Education 343

courses or activities. This group of students felt that face-to-face interaction, rather than some online and some face-to-face interaction, was more conducive to their learning. These students also expressed frustrations that they were not made aware of the online components before opting in to the course(s). In general, student respondents thought that college instructors should not implement online activities without first obtaining the skills to teach in an online environment, committing to respond to students in a timely manner, and organizing their materials in a way that is conducive to online instruction.

All instructor participants commended the college on its exploration of a hybrid degree program and recommended that as the college progresses, evaluative efforts continue in order to ensure that hybrid instruction is implemented in a way that best benefits student learning. Instructors also requested that more training opportunities be made available to help them use existing tools, integrate online activities, and effectively collaborate with each other.

IMPLICATIONS During the process of reading, coding, and identifying emergent themes

representing the three community perspectives, several categories of programmatic issues were noted as factors contributing to the success of the hybrid program. When these issues and implications were reviewed with instructor participants during a focus group, the instructor participants validated the implications and the identified themes were left intact. These implications are programmatic in nature and mostly address the administration, yet they impact the different identities within the hybrid degree program community. Addressing these recommendations will affect the success of instructor course design and student learning.

Develop Program Policy Supportive to Teaching and Learning in Hybrid Courses

When registering for courses, students were not informed that some course materials, activities, and assignments would be delivered online. Some students adjusted well to the hybrid delivery method, but others expressed frustration with the unexpected technology requirements and non-traditional instructional methods. With the help of administrators, the researchers made use of a course catalog footnote and existing Web site that alerts students that they are signing up for a hybrid course and explains how these courses differ from more traditional face-to-face classes.

It is our recommendation that when developing and promoting a hybrid degree program, expectations, instructional and communication methods, technical requirements, and benefits of combining the face-to-face and online learning environments be fully communicated to students prior to registration. Students can then make an informed decision as to whether the hybrid format meets their particular learning styles and preferences, schedule, and other needs. This communication could take place by providing information about the hybrid degree program in college marketing material, during advising and

344 Summer 2007: Volume 39 Number 4

registration sessions, and in program or course orientations. In such a manner, instructors and students will have common understandings regarding course design and expectations, and students not wanting to participate may opt out of such courses.

Support the Creation of Common Procedures and Expectations across Courses

When the hybrid units were developed for this study, instructors for each of the courses did not collaborate to develop common class or instructional procedures. In some cases inconsistencies from course to course caused student confusion and frustration.

It is important to remember the student perspective when developing a hybrid program. Some common elements across courses could positively impact student understanding and feasibility. Instructor CoPs should be encouraged to discuss their class procedures and expectations in order to develop common procedures. This is not to say that all instructors should have identical procedures, but that collaboration for the purpose of creating some level of consistency will benefit students. Common procedures and expectations could be developed related to e-mail/discussion board use, netiquette, use of course announcements, how to handle a technology snow day (Hitch, 2002), technology assistance, method for instructor contact, frequency and deadlines for discussion board posts, mechanisms for work submission, etc.

Allocate Face-to-Face and Online Time across Courses Most of the students participating in this study enrolled in more than one

course that used a hybrid format. Because the hybrid units did not fall in the same time period during the semester, student schedules were not consistent from week to week, causing frequent confusion and aggravation. Using student feedback, instructors worked with administrators to standardize Wednesday and Thursday as face-to-face days, leaving Monday, Tuesday and Friday free for student teaching, internships, and other student activities. This simple solution provided more structure for students and less confusion across courses within the same semester.

Although face-to-face and online activities should best fit the needs of a particular subject area and course (Veronikas & Shaughnessy, 2004), this study suggests that faculty and administrative CoPs work together to coordinate a schedule that outlines specific face-to-face and online days that will accommodate students taking multiple hybrid classes in the program. Maximum flexibility for students will occur when all courses in a given semester follow a similar or complimentary pattern of online and face-to-face days.

Support Instructor CoPs as they Refine and Adopt Technology Tools All instructor participants in this study received a basic overview of online

technologies during a summer workshop on designing and developing hybrid courses. Still, instructors found it difficult to gain an in-depth working knowledge of the online tools and features commonly associated with online

Journal of Research on Technology in Education 345

instruction. The design of activities was inhibited by their limited knowledge and familiarity with the available tools. Collaborative conversations within instructor CoPs about the functions and features of online tools appeared to increase the sophistication of technology use and instructional design.

Students participating in the study clearly articulated their preferences toward certain instructional practices and activities. It was evident that students preferred more simplistic methods of delivery (instructor presentations available for effortless download), online interactions (straightforward discussion boards), and ease in work submission. Instructor CoPs should discuss the use of technology tools to support specific learning needs, but technology that does not enhance instruction should be reduced or eliminated.

As instructors within a CoP learn about technology tools and their instructional uses, they will develop activities that incorporate the best of both face-to-face and online delivery methods. A supportive environment conducive to exploration, collaboration and cooperation will result in instructionally- sound activities and shared practices which will contribute to the overall quality of the program. To support this professional development and growth among hybrid instructors, administrators should provide mechanisms for faculty to collaborate within their CoP and interact with others outside their CoP, including instructional designers and technology support staff.

Provide Instructional Design Training and Support for Instructors The online questionnaire used in this study prompted instructors to refiect

on their hybrid units and identify successes as well as areas for improvement. The resulting data prompted the need for further professional development opportunities related to technology tools and delivery options.

Becoming a good hybrid instructor is a developmental process and requires continual nurturing and support in terms of the additional time it takes to develop and teach a hybrid course, as well as the adjustment to delivering materials, interacting with students, and designing activities for a Web-based environment (Kincannon, 2000). When asking instructors to redesign a course as a hybrid, administrators should recognize that this design and development process is akin to developing a new course, and instructors will likely need technology training.

As such, administrators need to support the professional development of instructors. This can take place in many ways, including providing adequate time over the course of several semesters to collaborate with other hybrid instructors, instructional designers, experienced colleagues, technology trainers and other personnel; soliciting help from other instructors or institutions who have more experience; providing hands-on training opportunities or one-on- one tutoring; and providing opportunities for instructors to share their successes with each other.

Provide Support for Students to Gain New Skills Anecdotal evidence gathered during this study indicated that many students

sought help from one another, upgraded from dial-up to faster Internet

346 Summer 2007: Volume 39 Number 4

connections at home, accessed the wireless networks on campus via laptops, purchased home computers or laptops, and improved their general technology skills. It is likely that the need for efficiency in completing online activities and assignments drove these changes.

Although it is possible that hybrid degree programs will attract more technologically savvy and independent students, it should not be assumed that students who enroll in hybrid courses have critical technology skills (Kvavik, 2005). Those who do not will be disadvantaged by the program delivery method. In order for students to focus on course content, it is critical that technology not be an obstacle to student access to course materials and support resources. As such, hybrid degree programs should identify and require base- level technology skills or offer training opportunities that prepare students with technology skills before classes begin (Gastfriend, Gowen, & Layne, 2001). These minimum technology skills should be communicated in college materials, advising sessions, and program or course orientations.

In addition, instructors should not assume that students have experience with the technologies used or that they have the ability to adopt new skills quickly. Even if students enter the program with a minimum set of technology skills, additional training or modeling during face-to-face classes, and written procedures and tutorials made available to all students will decrease concerns with technology and increase student ability to focus on content.

Continually Evaluate the Program Instructors in this study noted that as knowledge was created and brokered

during seminars and brown bag discussions, through formative feedback from students, and via the summative online questionnaire, evaluation practices helped them better understand and assess the implications of hybrid course and program design. In addition to traditional course evaluations, ongoing program evaluation must be implemented to continually improve instruction and student learning in any hybrid degree program (Levin, Levin, Buell, & Waddoups, 2002). Also, program evaluation and assessment must be based on multiple methods and must meet specific standards to ensure representation of the program's impact on administrators, faculty and students (Quality on the line, 2000).

Normally a new program would undergo rigorous scrutiny, with intense ongoing evaluation procedures that lessen over time as issues are worked out and satisfaction levels stabilize. However, with technology playing an integral role in hybrid courses, as new tools are made available or new uses for tools become established, ongoing innovation and refinement of courses, program delivery, and program structure becomes more necessary than in traditional face-to-face design. If this is the case, then the call for ongoing program evaluation policy would be meaningful to administrators, instructors, and to students.

Granted, systematically embedding data-driven decision making within a hybrid program would require more resources of time and money than one might normally commit. Not planning at the onset for continual innovation

Journal of Research on Technology in Education 347

and evaluation would be a mistake for a hybrid program not wishing to compromise quality.

CONCLUSIONS Although the scope of this study was limited to nine instructors and their

respective students, the results provide interesting and relevant findings for those interested in hybrid program design. The data collected indicate areas of success as well as areas for improvement, but overall the hybrid design was well received. The implications drawn represent a comprehensive dataset and demonstrate practices that must be thoughtfully considered by program developers before offering a hybrid degree program. While the primary factor in any instructional initiative remains the quality of the instructional design (Johnson & Aragon, 2002), the implications identified in this article intend to affect the success of students enrolled in a hybrid degree program directly.

It is hoped that this study will spur further research in this area, as over time student profiles will include more technology-sawy populations needing to balance education with personal and professional obligations. For institutions of higher education wanting to offer innovate programs that accommodate student needs, hybrid degree programs may provide the answer. Any such program should be strategically designed, coUaboratively developed, and implemented within a community vested in offering a successful program.

Contributors Audrey Amrein-Beardsley is an assistant professor in the College of Teacher

Education and Leadership at Arizona State University. Dr. Amrein-Beardsley holds a PhD in Educational Policy and Research Methods and specializes in tests, assessment, and survey research. (Address: Audrey Amrein-Beardsley, PO Box 37100, MC 3151, Phoenix, AZ, 85069-7100, Phone: 602.543.6374; Fax : 602.543.7052 ; E-mail: [email protected].)

Teresa S. Foulger is an assistant professor in the College of Teacher Education and Leadership at Arizona State University. Dr. Foulger holds an EdD in Educational Technology and specializes in technology-rich environments where collaboration, communities of practice, and innovative professional development models spur organizational change. (Address: Teresa S. Foulger, PO Box 37100, MC 3151, Phoenix, AZ, 85069-7100; Phone: 602.543.6420; Fax : 602.543.7052; E-mail: [email protected].)

Meredith Toth is an instructional designer with the Applied Learning Technologies Institute at Arizona State University. She holds a M.A. in Learning, Design, and Technology from Stanford University and specializes in technology integration in higher education. (Address: Meredith Toth, PO Box 37100, MC 1051, Phoenix, AZ, 85069-7100 ; Phone: 602.543.3192 ; E-mail: [email protected].)

References Allen, I., & Seaman, J. (2005). Growing by degrees: Online education in the

United States, 2005. The Sloan Consortium. Retrieved November 25, 2006, from http://www.sloan-c.org/publications/survey/pdf/growing_by_degrees.pdf

348 Summer 2007: Volume 39 Number 4

Allen, I., & Seaman, J. (2006). Making the grade: Online education in the United States, 2006. The Sloan Consortium. Retrieved March 3, 2007, from http://www.sloan-c.org/publications/survey/pdf/Making_the_Grade.pdf

Anastasiades, P. S., & Retalis, S. (2001, June). The educational process in the emerging information society: Conditions for the reversal of the linear model of education and the development of an open type hybrid learning environment. Paper presented at the ED-MEDIA 2001 World Conference on Educational Multimedia, Hypermedia & Telecommunications, Tampere, Finland. ERIC Document Number: ED466129. Retrieved April 28, 2006.

Ausburn, L. (2004). Course design elements most valued by adult learners in blended online education environments: An American perspective. Educational Media International, 41 {A), iTJ-iH.

Aycock, A., Garnham, C , & Kaleta, R. (2002). Lessons learned from the hybrid course project. Teaching with Technology Today, 8(6). Accessed online April 12, 2005 at http://www.uwsa.edu/ttt/articles/garnham2.htm

Bailey, K., & Morais, D. (2004). Exploring the use of blended learning in tourism eduation. Journal of Teaching in Travel & Tourism, 4{4), 23-36.

Benbunan-Fich, R., & Hiltz, S. (2003). Mediators of the effectiveness of online courses. IEEE Transactions on Professional Communication, 46{A), 2 9 8 - 312.

Bennett, G., & Green, F. P. (2001). Student learning in the online environment: No significant diflFerence? Quest, 53, 1-13.

Benson, A. (2002). Using online learning to meet workforce demand: A case study of stakeholder influence. Quarterly Review of Distance Education, 3(4), 4 4 3 ^ 5 2 .

Benson, A. (2003). Dimensions of quality in online degree programs. American Journal of Distance Education, 170), 145-159.

Blackboard (1997). Blackboard (Version 6.2) [Computer software]. Washington, DC: Blackboard Inc.

Bonk, C , Olson, T., Wisher, R., & Orvis, K. (2002). Learning from focus groups: An examination of blended learning. Journal of Distance Education, 77(3), 97-118.

Boyle, T , Bradley, C , Chalk, P, Jones, R., & Pickard, P (2003). Using blended learning to improve student success rates in learning to program. Journal of Educational Media, 28{2/3), 165-178.

Brown, J. S., &C Duguid, P. (2000). The social life of information. Boston: Harvard Business School Press.

Buckley, D. (2002). Pursuit of the learning paradigm: Coupling faculty transformation and institutional change. Educause Review, 37(1), 28-38.

Cheng, H., Lehman, J., & Armstrong, P (1991). Comparison of performance and attitude in traditional and computer conference classes. The American Journal of Distance Education, 5(3), 5 1 - 6 4 .

Cousin, G., & Deepwell, F. (2005). Designs for network learning: A communities of practice perspective. Studies in Higher Education, 30(1), 57-66.

Journal of Research on Technology in Education 349

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.

DeTure, M. (2004). Cognitive style and self-efficacy: Predicting student success in online distance education. American Journal of Distance Education, 75(1), 21-38.

Donnelly, R. (2006). Blended problem-based learning for teacher education: Lessons learnt. Learning, Media, & Technology, 37(2), 93-116.

Dziuban, C , Hartman, J., & Moskal, P (2004). Blended learning. EDUCAUSE Center for Applied Research Research Bulletin. Accessed online January 21, 2007, at http://www.educause.edu/ir/library/pdf/ERB0407.pdf

Fotilger, T (Summer 2005). Innovating professional development standards: A shift to utilize communities of practice. Essays in Education, 14, Retrieved September 20, 2006, from http://www.usca.edu/essays/voll4summer2005.html

Garrison, D. R. (1985). Three generations of technological innovation in distance education. Distance Education, 6(2), 235—241.

Gastfriend, H. H., Gowen, S. A., & Layne, B. H. (2001, November). Transforming a lecture-based course to an Internet-based course: A case study. Paper presented at the National Convention of the Association for Educational Communications and Technology, Atlanta, Georgia. ERIC Document Number: ED470085. Retrieved April 28, 2006.

Goleman, D. (1995). Emotional intelligence: Why it can matter more than IQ. New York: Bantam Books.

Graham, C. R., Allen, S., & Ure, D. (2005). Benefits and challenges of blended learning environments. In M. Khosrow-Pour (Ed.), Encyclopedia of information science and technology (pp. 253-259). Hershey, PA: Idea Group.

Hitch, L. P. (2002). Being prepared for technology snow days. ECAR Research Bulletin, 24. Retrieved March, 2003, from http://www.educause.edu/ LibraryDetailPage/666?ID=ERB0224

Husmann, D. E., & Miller, M. T. (2001). Improving distance education: Perceptions of program administrators. Online Journal of Distance Learning Administration, IV(III). Retrieved September 23, 2006, from http://vvTvw. westga.edu/-distance/ojdla/articles/fall2001/husmann43.pdf

Iverson, J. O., & McPhee, R. D. (2002). Knowledge management in communities of practice: Being true to the communicative character of knowledge. Management Communication Quarterly, 16(2), 259—266.

Johnson, S.D. & Aragon, S.R. (2002, Spring). An instructional strategy framework for online learning environments. Paper presented at the Academy of Human Resource Development (AHRD) Conference, Honolulu, Hawaii. Retrieved April 28, 2006, from ERIC database.

Kincannon, J. M. (2002, April). From the classroom to the Web: A study of faculty change. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, Louisiana. ERIC Document Number: ED467096. Retrieved April 28, 2006.

Kuhn, T. (2002). Negotiating boundaries between scholars and practitioners: Knowledge, networks, and communities of practice. Management Communication Quarterly, 16{\), 106-112.

350 Summer 2007: Volume 39 Number 4

Kvavik, R. (2005). Convenience, communications, and control: How students use technology. In D.G. Oblinger & J.L. Oblinger (Eds.), Educating the Net Generation (pp 7.1-7.20). Washington, DC: Educause. Retrieved January 8, 2005, from http://www.educause.edu/educatingthenetgen/

LaRose, R., Gregg, J., & Eastin, M. (1998). Audiographic telecourses for the Web: An ex^crmcnx.. Journal of Computer-Mediated Communication, 4(2). Retrieved March 2, 2007, from http://jcmc.indiana.edu/vol4/issue2/larose.html

Lave, J., &C Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge: Cambridge University Press.

Learning Technology Center. (2002, March 12). Learning technology center: Our project. Retrieved May 2005 from http://www.uwm.edu/Dept/LTC/our- project.html

Leh, A. (2002). Action research on hybrid courses and their online communities. Educational Media International, 39(1), 31-38.

Levin, S. R., Levin, J. A., Buell, J. G. & Waddoups, G. L. (2002). Curriculum, Technology, and Educational Reform (CETER) online: Evaluation of an online master of education program. TechTrends, 46(5), 30—38.

Machtmes, K. & Asher, J. W (2000). A meta-analysis of the effectiveness of telecourses in distance education. The American Journal of Distance Education, 7 4 ( 0 , 2 7 - 4 6 .

McCray, G. (2000). The hybrid course: Merging on-line instruction and the traditional classroom. Information Technology & Management, 1(4), p307-327.

Mezirow, J. (2000). Learning to think like an adult: Core concepts of transformation theory. In J. Mezirow & Associates (Ed.), Learning as transformation: Critical perspectives on a theory in progress (pp. 3—33). San Francisco: Jossey-Bass.

Miles, M.B. & Huberman, A.M. (1994). Qualitative data analysis (2"'' ed.). Thousand Oaks, CA: Sage Publications.

Misko, J. (1994). Flexible delivery: Will a client-focused system mean better learning^ KdA^idt: Adelaide National Centre for Vocational Education Research.

Moore, M. (1993). Is teaching like flying? A total systems view of distance cducdMon. American Journal of Distance Education, 7(1), 1-10.

Neuhauser, C. (2002). Learning style and effectiveness of online and face-to- face instruction. American Journal of Distance Education, 16(2), 99-113.

Nunnally, J. C. (1978). Psychometric Theory (2'"' ed.). New York, NY: McGraw Hill.

Olapiriyakul, K., & Scher, J. (2006). A guide to establishing hybrid learning courses: Employing information technology to create a new learning experience, and a case study. Internet & Higher Education, 9(4), 287-301.

O'Toole, J. M., & Absalom, D. (2003). The impact of blended learning on student outcomes: is there room on the horse for Vf/oii Journal of Educational Media, 28(215), 179-190.

Overbaugh, R., & Lin, S. (2006). Student characteristics, sense of community, and cognitive achievement in web-based and lab-based learning environments. Journal of Research on Technology in Education, 39(2), 205-223.

Journal of Research on Technology in Education 351

Parkinson, D., Greene, W , Kim, Y, & Marioni, J. (2003). Emerging themes of student satisfaction in a traditional course and a blended distance course. TechTrends, 47(4), 22-28.

Phipps, R., & Merisotis, J. (1999). What's the difference? A review of contemporary research on the effectiveness of distance learning in higher education. Washington, DC: The Institute for Higher Education Policy. Accessed online on February 16, 2007 at http://www.ihep.org/Pubs/PDF/Difference.pdf

Quality on the line: Benchmarks for success in internet-based distance education. (2000). Washington, DC: The Institute for Higher Education Policy. Retrieved September 16, 2006, from http://www.ihep.com/Pubs/PDF/Quality.pdf#search =%22Quality%20on%20the%20line%20benchmarks%22

Reeves, T., Herrington, J., & Oliver, R. (2004). A development research agenda for online collaborative learning. Educational Technology Research and Development, 52(4), 53-65.

RifFell, S., & Sibley, D. (2005). Using web-based instruction to improve large undergraduate biology courses: An evaluation of a hybrid course format. Computers & Education, 44(3), 217-235.

Rogers, A. (2002). Teaching adults (3"* ed.). Philadelphia: Open University Press.

Russell, T. (1999). The no significant difference phenomenon. Chapel Hill, NC: Office of Instructional Telecommunications, University of North Carolina.

Sherer, P D., Shea, T. P, & Kristensen, E. (2003). Online communities of practice: A catalyst for faculty development. Innovative Higher Education, 27(3), 183-194.

Sikora, A. (2002). A profile of participation in distance education: 1999-2000. Retrieved January 8, 2005, from National Center for Education Statistics Web site http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2003154

Skylar, A., Higgins, K., Boone, R., Jones, P, Pierce, T , & Gelfer, J. (2005). Distance education: An exploration of alternative methods and types of instructional media in teacher education. Journal of Special Education Technology, 20(3), 25-33.

Snell, C , & Penn, E. (2005). Developing an online justice studies degree program: A case study. Journal of Criminal Jtistice Education, 16(\), 18—36.

Summers, J., Waigandt, A., & Whittaker, T. (2005). A comparison of student achievement and satisfaction in an online versus a traditional face-to-face statistics class. Innovative Higher Education, 29(3), 233-250.

Sumsion, J., & Patterson, C. (2004). The emergence of community in a preservice teacher education program. Teaching and Teacher Education, 20(6), 621-635.

Swenson, P, & Evans, M. (2003). Hybrid courses as learning communities. In S. Reisman (Ed.), Electronic learning communities issues and practices (pp. 27-72). Greenwich, CT: Information Age Publishing.

Thompson, M. (2005). Structural and epistemic parameters in communities of practice. Organization Science, 16(2), 151-164.

Twigg, C. A. (1999). Improving learning & reducing costs: Redesigning large- enrollment courses. Center for Academic Transformation. Retrieved April 20, 2005, from http://www.thencat.org/Monographs/monol.pdf

352 Summer 2007: Volume 39 Number 4

Twigg, C. A. (2001). Innovations in online learning: Moving beyond no significant difference. Taylor, NY: Pew Learning and Technology.

Twigg, C. A. (2003). Improving learning and reducing costs: New models For online learning. Educause Review, 35(5), 28-38.

Veronikas, S. W , & Shaughnessy, M.F. (2004). Teaching and learning in a hybrid world: An interview with Carol Twigg. Educause Review, 39(July/ August), 51-62. Retrieved February, 2006, From http://www.educause.edu/ apps/er/ermO4/ermO44.asp

Waits, T. & Lewis, L (2003). Distance education at degree-grantingpostsecondary institutions: 2000-2001 (NCES 2003-017). U.S. Department oFEducation, National Center For Education Statistics. Retrieved January 8, 2005 From http:// nces.ed.gov/pubsearch/pubsinFo.asp?pubid=2003017

Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge: Cambridge University Press.

Wenger, E., McDermott, R., & Snyder, W. M. (2002). Cultivating communities of practice: A guide to managing knowledge. Boston: Harvard Business School Press.

Wilke, D., & Vinton, L. (2006). Evaluation of the first web-based advanced standing MSW prograsn. Journal of Social Work Education, 42(3), 607-620.

Woods, R., Baker, J., & Hopper, D. (2004). Hybrid structures: Faculty use and perception oF web-based courseware as a supplement to face-to-Face instruction. Internet & Higher Education, 7(4), 281-297.

Yang, Y, & Corneliotis, L.F. (2004, October). Ensuring Quality in Online Education Instruction: What Instructors Should Know? Paper presented at the Association For Educational Communications and Technology conFerence, Chicago, Illinois. ERIC Document Number: ED484990. Retrieved September 23, 2006.

Young, S. (2006). Student views oFeffective online teaching in higher education. The American Journal of Distance Education, 20(2), 65—77.

Journal of Research on Technology in Education 353

APPENDLX: STUDENT/INSTRUCTOR HYBRID EVALUATION QUESTIONNAIRE

PART I: DEMOGRAPHIC QUESTIONS

What is your age? (Students only)

Where do you primarily access a computer For schoolwork? (Students onlv)

How do you most ohen col Internet? (Students only)

Degree (Students only)

Current Semester

l.ourse I

('oiirse I itlc

LAST Name oi vour Instructor

POSSIBLE RESPONSES

Younger than 18 18 to 25 26 to 35 36 to 45 46 to 55 Older than 56

Home (desktop) Mobile (laptop) Student computer center Library Friend/Relative residence Other

Home (high-speed)! Home (dial-up) Away-trom-honie (high-speed) Away-from-home (dial-up)

Undergraduate Graduate Post-Baccalaureate

Elementary Education Secondary Education Special Education

Semester 1 Semester 2 Semester 3 ScnlC'̂ ter i

One

For this semester, how many of your courses ._, incorporated online days? (Students only)

Five

For this course, approximately liow many ^ J face-to-face days were replaced vviih online activities this semester? (Instructors onlv) ^

354 Summer 2007: Volume 39 Number 4

pip |ooa S!t

S3.'>U3U3dx3 AlU

|OO1 SU(̂ J

nil MM MM

HIM MM

I c o

T3

-o

^ o

E .5aQ

••̂ c c .S p

§ J! 3i =5 E S g 2 .1 -̂ ^ 4> O VI '•.':

O O

LU

"3 o u

•6 c

.2

I I 1 "̂

P u -r: -= - :

Journal of Research on Technology in Education 355

Summer 2007: Volume 39 Number 4

..2

— o

i i.

i2 ou 3 —

c o

C C o o

E

II U 3

Journal of Research on Technology in Education 357