journal entry

Journal of Interactive Online Learning www.ncolr.org/jiol

Volume 13, Number 3, Spring 2015 ISSN: 1541-4914

 

   

112  

Student Perspectives of Assessment Strategies in Online Courses

Scott Bailey, Stacy Hendricks, and Stephanie Applewhite Stephen F. Austin State University

Abstract

Engaging professional adults in an online environment is a common challenge for online instructors. Often the temptation or commonly used approach is to mirror face-to-face strategies and practices. One premise of this study is that all strategies used in an online environment are assessment strategies, and as such should be considered for their value in measuring student experiences. This research study investigated student responses within a principal preparation course to the use of twelve assessment strategies that included: work samples, “Twitter” summaries, audio recordings, traditional papers, screencast/videos using “YouTube”, group projects, open discussion, paired discussion, response to video, field experiences, quizzes, and interviews. The redesigned course used in this research allowed the researchers to experiment with both traditional and innovative strategies within an online environment to determine how students perceive the value of each assessment strategy. Student experiences were measured in terms of level of enjoyment, level of engagement, and the extent to which students believed the assessments would result in the creation of knowledge that could be transferred to future professional practice. The results indicate that students prefer assignments that are less- traditional and which fully incorporate the technological tools available.

Online teaching is here to stay. With each passing semester, more college courses—and even entire degree programs—move online. The question is no longer one of whether teaching online is effective; the question now rests on how to maximize its effectiveness. Answering that question, or maximizing the effectiveness of online teaching and learning, requires online instructors to shift their attention “from the technology tools to the pedagogical practices and use of the tools” (Redmond, 2011, p. 1058) and “to make a transformational shift in their approach to teaching from one of disseminating information to one of creating learning environments where students co-construct knowledge through interactions” (Vaughn, 2010, p. 61). Johnson and Aragon (2003) addressed the issue head on: “the challenge for instructional designers is to devise ways to incorporate the most effective and innovative instructional strategies in courses delivered over the Internet” (p. 33). This study accepted that challenge and examined the experiences of students engaging in a variety of teaching through assessment strategies embedded in the redesign of an online educational leadership course.

For years, one regional university supported face-to-face, hybrid, and online delivery of courses in an educational leadership program. Recently, driven by the realities of economics and

Journal of Interactive Online Learning Baily, Hendricks, and Applewhite  

   

113  

student demand, the entire program moved online. Despite contradictory evidence (Johnson & Aragon, 2003; Whitlock, 2001), the online course offerings have, to date, closely mirrored their face-to-face predecessors. With both the quality and quantity of online teaching tools expanding rapidly, the authors reexamined how their online courses were designed and presented to students. The review of literature revealed several relevant themes that influenced the subsequent course redesign (Johnson & Aragon, 2003; Laurillard et al., 2011; Redmond, 2011), including: (a) engagement (Kearsley & Shneiderman,1998; O’Brien & Toms, 2008; Schlechty, 2001; Steinbronn & Merideth, 2008); (b) instructor and Learner roles and responsibility (Cunningham, 2010; Moore, Dickson-Deane, & Galyen, 2011); and (c) teaching through assessment (Edwards, 2012; Gayton & McEwen, 2007). Each of these themes was significant to both course development and research design.

Review of Literature

An outside observer could ostensibly conclude that learning online is different from

learning in a conventional classroom, yet research indicates that might not be true (Johnson & Aragon, 2003; Russell, 1999; Steinbronn & Merideth 2008). In terms of learning outcomes, Steinbrom and Merideth (2008) found “no statistical difference when using online or face-to-face formats” (p. 266). However, what happens during the learning processes to achieve those outcomes does differ. The relationships between the instructor and the students and the content and the process change in important ways, namely: “learners become more active in their responsibility for learning,” and instructors serve as the “facilitator, strategist, and coordinator for the learning activities” (p. 266). These changes inform the theoretical framework for this study which is grounded on the premise that, as learners become more responsible for their learning, instructors are tasked with redesigning their courses to successfully engage the learners to achieve the course objectives. Engagement Theory

The changes in instructor/student relationships described by Steinbronn and Merideth (2008) mesh well with the “engagement theory” developed in the infancy of online learning by Kearsley and Shneiderman (1998), who presented engagement theory as

a model for learning in technology-based environments. The major premise is that students must be engaged in their coursework in order for effective learning to occur. The theory posits three primary means to accomplish engagement: (1) an emphasis on collaborative efforts, (2) project-based assignments, and (3) non-academic focus. It is suggested that these three methods result in learning that is creative, meaningful, and authentic. (p. 23)

In their view, students become engaged when they interact with others and complete meaningful work, and it is the meaningful nature of those interactions that creates the intrinsic motivation required for their success as learners. The meaning of an assignment is created when the student makes connections to their immediate context (Cunningham, 2010), which for these courses was the emerging school administrator. The technology involved with the online presentation is just a means to create and deliver these experiences and is not a focus of the course. However, taking into consideration possible differences between classroom engagement and online engagement proves useful in the re-design of an online course. O’Brien and Toms (2008) conducted a study to “critically deconstruct the term engagement as it applies to peoples’ experiences with technology” (abstract), considering

Journal of Interactive Online Learning Baily, Hendricks, and Applewhite  

   

114  

pleasure or enjoyment as a measurable factor linked to a person’s level of engagement. Other measured attributes included “feedback, intrinsic motivation, fun, user control, and interactivity” (p. 940), with these attributes contributing to the survey design of this research study and was used to justify the inclusion of “enjoyable” as a valid factor. The results of O’Brien and Toms (2008) yielded a “common trajectory for engaging experiences” that they describe as “experiential threads” which included: “the point of engagement, period of engagement, disengagement, and reengagement” (p. 943). As Schlechty (2001) noted, “to be student centered is not to cater to student whims. Rather, it is to understand the students and the things that motivate them better than the students understand their own motivations” (p. 29). Further, “students learn from what they do” (p. 63), so the first thing instructional designers must focus upon is developing assignments that students will do, as no learning will occur until students actually do something. To create those assignments, instructional designers must tap into their students’ motives. O’Brien and Toms (2008) affirmed Schechty’s ideals in their study, identifying the period of engagement of the study participants as “marked by the attention [they] were able to focus on their task and the application, the novelty of the experience, their level of interest, and their perceptions of challenge, feedback, and user control inherent in the interaction” (p. 943). The types of assignments that Schlechty (2001) discussed are precisely the ones needed to engage online learners. These assignments cause students to “experience a sense of satisfaction, accomplishment, pride, and sometimes delight;” they “result in students’ persisting with the tasks assigned until their work meets the desired standard;” and, they result in “students learning those things they need to know to do well” (p. 71). Re-Designed Online Instructional Framework While modern educators recognize the importance of student engagement and the pathways to achieve it in traditional classrooms, including project-based, situational, and collaborative learning, Johnson and Aragon (2003) noted that those innovations often do not transfer to the online courses instructors create; instead, those courses reflect more “traditional” forms of instruction focused on the transmission/recitation of information between the instructor and the students. That more traditional framework will not stimulate the engagement Kearsley and Shneiderman (1998) deemed necessary for student success online. In short, “the challenge for instructional designers is to devise ways to incorporate the most effective and innovative instructional strategies in [online] courses” (p. 33), and they are not meeting this challenge (Johnson & Aragon, 2003). In essence, these online courses become correspondence courses, centered around readings or recorded lectures followed by quizzes or tests over the content. In response, Johnson and Aragon (2003) synthesized elements of behavioral, cognitive, and social learning theories to develop a “synergistic” (p. 34) framework that instructional designers can utilize to guide the development of more engaging and student-centered online courses. Accordingly, the framework consists of elements that “(1) address individual differences, (2) motivate the student, (3) avoid information overload, (4) create a real-life context, (5) encourage social interaction, (6) provide hands-on activities, and (7) encourage student reflection” (p. 34). This framework serves as an extension of the engagement theory proposed earlier (Kearsley & Shneiderman, 1998) and a practical way to visualize a course construct similar to Gayton and McEwen (2007) in which students affirmed high levels of online engagement through a variety of assignments that served as assessments.

Journal of Interactive Online Learning Baily, Hendricks, and Applewhite  

   

115  

Re-Designed Instructor and Student Roles and Responsibilities The instructor’s responsibility is now to re-design activities or assessments that follow the guidelines of the framework and successfully engage and motivate students to accomplish the desired learning outcomes. The redesign process involves a re-discovery of roles and responsibilities because the teacher is no longer the lecturer, but instead transforms out of the traditional identity into a facilitator or designer (Cunningham, 2010; Moore et al., 2011; Redmond, 2011). For some, this process is terribly challenging, and one solution to overcoming that challenge is to reach out to others who are also re-designing their courses. Laurillard et al. (2011) suggested that collaborative design strategies were found to be useful for academics to create meaningfully designed online courses. Their study was designed to investigate the value of an interactive design tool that allowed the sharing of strategies, experiences, testing of ideas, and sharing of results. The idea of a “Community of Inquiry” (CoI) (Anderson, Rourke, Garrison & Archer, 2001; Akyol & Garrison, 2008; Cunningham, 2010; Redmond, 2011) was researched for its value to the online learning environment. CoI framework involves the intersection of three components, all of which are framed as a type of “presence” that included social, cognitive, and teaching presence (Akyol & Garrison, 2008). As Cunningham (2010) discussed, the idea of presence in an online environment causes both instructors and learners to assess their traditional assumptions about their roles and responsibilities in the learning process. The assumptions that Cunningham (2010) challenged as needing to change included: “students do not learn without a teacher present” (p. 92); “my students cannot possibly understand this textual content without my lectures;” and finally the assumptions that both teachers and students hold regarding their views of what learning looks like and how it occurs (p. 93). Cunningham (2010) concluded that “understanding the needs of a self-directed student is paramount for a successful course design” (p. 99). Moore et al. (2011) conducted research on the roles in the online learning environment and found that even when given survey answer choices to identify themselves as “instructor/facilitator, designer, and evaluator” (p. 132) the majority of the teacher-participants selected the answer choice of “instructor” which validated the assumptions Cunningham (2010) discussed. In an effort to further the understandings of how students perceive their experiences in the online environment, the researchers of this study, using the literature as a guide, determined it was necessary to re-design their online courses using a collaborative planning approach, and developed a tool to measure the students’ level of engagement with the newly implemented strategies.

Research Questions The purpose of this study was to investigate students’ perceptions of various assessment strategies, with the intent of determining which strategies students find most engaging and meaningful. These three questions guided the study:

1. Which assessment strategies do students find enjoyable? 2. Which assessment strategies do students believe create meaningful engagement? 3. Which assessment strategies do students perceive as likely to create knowledge that will

transfer to practice?

Journal of Interactive Online Learning Baily, Hendricks, and Applewhite  

   

116  

Method

This study followed a descriptive format, relying on student-perception data collected from an online survey. An email with the link to the online survey was sent to the course email addresses of all the students concurrently enrolled in both courses under study (N = 51) and was followed by a reminder email three days later. The survey window was open for five days, or the final week of the course (so that the students had time to complete all of the assessments). Thirty-five students responded, yielding a response rate of 69%. The idea of learning design through peer collaboration (Laurillard et al., 2011) inspired the partnership of researchers in this study. Focusing on two introductory courses to the principal preparation program, the authors redesigned the courses to make them more conducive to online delivery and structured them so students would work through the courses in parallel, utilizing the same texts and resources.

For the purposes of this study, all learning activities are referred to as assessments, because all activities formatively assess student progress (Edwards, 2012; Gayton & McEwen, 2007).

In accordance with engagement theory and the instructional framework discussed earlier, a key component of this process was driven by the researchers’ desire to increase student interactions with the professor, other students, and most importantly, the content. To that end, the redesign acknowledged that, in an online setting, learning is highly self-directed (Cunningham, 2010; Johnson & Aragon, 2003); and, it is precisely that dependence on self-direction that begs for the creation of engaging assessments. Recognizing that interaction with course content results in knowledge creation and that learning occurs most intensely during the assessment process (Edwards, 2012), the authors imbedded multiple assessment methods in the courses to engage students with the content in varied ways. A total of 12 assessment strategies were included in the course and subjected to analysis, and together, these strategies addressed all seven of the elements outlined in the online instructional framework developed by Johnson and Aragon (2003).

The assessments included: 1. Work Samples: Supplied with either a data set or a scenario, students produced

documents replicating ones they would actually create as practicing administrators, such as a campus plan, a professional development plan, a campus needs assessment, and a professional growth plan for a teacher in need of assistance.

2. “Twitter” Summaries: Students followed Twitter guidelines to summarize each chapter of a course textbook in 140 characters or less.

3. Audio Recording: In lieu of written artifacts, students submitted audio recordings of themselves explaining important concepts as they might explain them to parents, faculty, or other groups.

4. Traditional Paper: Students wrote and submitted traditional papers of various lengths (2, 5, and 8 pages) or used the textbook to respond in writing to a series of questions.

5. Screencast/Video: Students were required to locate and download screencasting software which was utilized to create a screencast with embedded video of themselves. Then they uploaded the screencast to a private YouTube channel for instructor and class viewing. This assignment simulated the students’ introducing themselves and their professional priorities as an administrator to the faculty at a new school.

Journal of Interactive Online Learning Baily, Hendricks, and Applewhite  

   

117  

6. Group Project: After being randomly assigned to groups, students worked together to complete tasks and submit one finished written product representative of the group. Most students utilized the communication tools embedded in the course (email, private discussion boards, and chat tools), yet some relied on other publicly available tools from outside the course.

7. Open Discussion: Topics were posted to the course discussion board, and students created original posts to add to the board and reply to other students’ posts. Students interacted with the entire class (30 students).

8. Paired Discussion: Students were grouped (2-5 students) and discussed the assignment topics only within that group. Students could only view responses within their group.

9. Respond to Video: Rather than respond in writing to a text, students responded in writing to a video. They accessed and watched a one-hour documentary, streamed through the course management system, and produced a written analysis.

10. Field Experiences: Students must complete field experience hours (20 hours/course). Students choose what activities to engage in to complete these hours as best suits their needs and interests, but the activities must be outside the scope of their regular jobs. At the end of the course, students submit a log of their hours and a written reflection.

11. Quizzes: These quizzes were short, multiple-choice quizzes designed to test students’ factual knowledge of the content immediately after it was presented. Quizzes were auto-graded, with the grade immediately released to the student.

12. Interviews: Students interviewed practicing school administrators. For each assigned topic, students interviewed two administrators and synthesized their work into a summative reflection.

Participants The participants were graduate students in the educational leadership program concurrently enrolled in each of two sections (four sections total) of the authors’ two courses under study. Of the 35 respondents, 18 (51%) were male and 17 (49%) were female. They were relatively equally split among age groups, with 14 (40%) occupying the 25-35 years span, 10 (29%) in the 36-45 years span, and 11 (31%) in the 46+ years span. The respondents self- identified as technologically proficient. Thirty-one (89%) have owned a computer for more than eight years, and 26 (74%) had taken online classes before. Instrument The online survey used to collect student perception data was developed and designed by the authors specifically to address the research questions. The authors checked the survey for face and content validity with colleagues in the department and graduate students in one of the author’s research classes.

The survey consisted of twelve sections, with each section representing a different assessment strategy. Each section included a brief description of the assessment strategy and a reminder of the assignments which represented that strategy. Respondents answered three questions about each assessment strategy according to three criteria.

First, students rated each strategy in terms of enjoyableness, guided by the following choices: A = very enjoyable; B = somewhat enjoyable; C = just another assignment; and, D = not enjoyable at all. The survey defined enjoyable as “resulting in pleasure or satisfaction both while

Journal of Interactive Online Learning Baily, Hendricks, and Applewhite  

   

118  

working on and after completing the assignment,” and asked students to “think about how you felt in regards to the process of completing the assignment.” Then, students evaluated the level of engagement each strategy engendered, with the following choices: A = highly engaging; B = moderately engaging; C = just another assignment, and; D = not engaging at all. The survey defined engaging as “heightening interest, motivation, and attention brought to task” and asked students to “think about how the assignment caused you to interact with the course content.”

Next, students estimated the degree to which each strategy helped generate knowledge applicable to their future practice using the following options: A = always apply directly to professional practice as an administrator; B = sometimes apply to professional practice; C = rarely apply to professional practice; and, D = never apply to professional practice. The instructions directed students to “think about whether you believe the knowledge gained from this assignment resulted in something you could utilize as a practicing administrator.”

Finally, students were presented with an open-ended response box asking for “additional comments about or suggestions for these types of assessments.” Once the survey window closed, the authors utilized the tools built into the survey collection program to disaggregate the responses. Limitations

This study was limited in that the researchers relied on a non-standard data collection instrument and utilized a small sample of convenience. These factors prevent the results of the study from being generalized to a different population, but the results remain relevant as points of discussion to work toward program improvement.

Results and Discussion

This study asked students in an educational leadership program to rate 12 assessment strategies utilized in two of their online courses on the criteria of enjoyableness, engagement, and the degree to which they believed the knowledge gained from completing the assessments could be transferred to future practice as an administrator. The final percentage ratings are summarized in Table 1. These ratings, along with anecdotal evidence taken from the students’ open-ended responses, were used to provide a descriptive view of students’ feelings toward their experience with online assessments. Overview

Overall, students indicated an overwhelmingly positive experience. Thirty-five students rated twelve assessments on three criteria, constituting a total of 1,260 responses. Of that total, only 19 responses (1.5%) were negative and 145 (11.5%) were neutral, meaning that 1,096 responses (87.0%) were moderately to highly positive.

Journal of Interactive Online Learning Baily, Hendricks, and Applewhite  

   

119  

Table 1 Student Ratings of Assessment Methods by Percentage

Level of Agreement Assessment Methods High Moderate Neutral Not at All Work Samples

Enjoyableness 29 63 3 6 Engagement 40 57 3 0 Transferability 74 26 0 0 Twitter Summary Enjoyableness 71 20 6 3 Engagement 71 23 6 0 Transferability 71 23 6 0 Audio Recording Enjoyableness 54 26 11 9 Engagement 60 23 17 0 Transferability 54 34 11 0 Traditional Paper Enjoyableness 23 46 23 9 Engagement 31 40 26 3 Transferability 51 40 9 0 Screencast/Video Enjoyableness 49 40 6 6 Engagement 71 20 9 0 Transferability 71 23 6 0 Group Projects Enjoyableness 46 34 20 0 Engagement 54 23 23 0 Transferability 49 46 6 0 Open Discussion Enjoyableness 31 52 17 0 Engagement 37 51 11 0 Transferability 43 51 6 0 Paired Discussion Enjoyableness 34 49 14 3 Engagement 40 43 17 0 Transferability 43 40 17 0 Respond to Video

Enjoyableness 91 9 0 0 Engagement 91 9 0 0 Transferability 91 9 0 0 Field Experiences

Enjoyableness 57 31 9 3 Engagement 74 17 6 3 Transferability 83 14 3 0

Journal of Interactive Online Learning Baily, Hendricks, and Applewhite  

   

120  

Quizzes Enjoyableness 9 40 46 6

Engagement 17 31 49 3 Transferability 23 54 20 3 Interviews

Enjoyableness 66 26 9 0 Engagement 83 14 3 0 Transferability 83 17 0 0

For the individual assessments, students rated the response to video the highest in terms

of enjoyableness (91%), followed by the twitter summaries (71%) and the interviews (66%). In terms of engagement, the response to video was again rated highest (91%) and was followed by the interviews (83%) and field experiences (74%). For the final criterion of transferability of knowledge, the response to video was rated highest for the third time (91%), trailed by the field experiences and interviews (83%) each. Using the “neutral” response as a guide, students judged the quizzes (46%), the traditional paper (23%), and the group projects (17%) least enjoyable. The same three assessments were ranked in the same order in terms of being the least engaging (quizzes, 49%; traditional paper, 26%; and group projects 23%, respectively). For the assessments that produced the least transferable knowledge, the quizzes again received the top neutral ranking (20%), followed by the paired discussion (17%) and the audio recording (11%). Due to their small number, the negative, or “not at all” responses were not included and the “neutral” responses were used as a proxy for the negative response. Individual Assessments Response to video. The highest rated assessment across all categories was the response to video. There was only one assessment that fit this category, and the video to which the students responded was a 50 minute documentary that tracked the struggles of two principals. From prior experience in other classes, students almost always unanimously agree that the video is inspiring, moving, and very enjoyable; many students have reported watching it several times. Initially, the researchers attributed the high ratings for this assignment to the failure of the students to separate the quality of the video from the quality of the assignment; in other words, did students respond positively simply because they liked the video or was it because they actually liked the assignment? The answer to that question was found in their open-ended responses. Comments included,

“Enjoyed something visual for once.” “It is REAL LIFE! You can’t beat that.” “Able to learn from someone else’s experience.” “ROLE MODELING. Now I’m better able to synthesize the readings. Very

effective.” “Loved this assignment and the movie. It really helped give me a vision of the

kind of principal I would like to become.” These comments indicate that the students found value in the assignment beyond watching an enjoyable video. The video provided a real-world connection and a visual hook for the other work completed in the course. The video assignment addressed three criteria of the online

Journal of Interactive Online Learning Baily, Hendricks, and Applewhite  

   

121  

instructional strategy framework (Johnson & Aragon, 2003): it was motivational, created a real- life context, and encouraged reflection.

Interactive assignments. Another important criterion established in the online instructional strategy framework (Johnson & Aragon, 2003) was that assignments should encourage social interaction. A number of assessments met that criterion, including the group projects, both the open and paired discussions, the field experiences, and the interviews. Interestingly, and contrary to the expected benefit of student-to-student collaboration, students gave relatively low rankings in terms of enjoyment, engagement, and transferability to the assessments requiring interaction with other students (the discussions and group projects). The comments left by students were split evenly between the view that these interactive assessments were positive because they did foster interactions with others and the more negative view which frowned on the fact that these assessments bound students to one another in terms of time and content. Responses ranged from “loved the collaboration” and “enjoyed working as a team” to “poor group experience” and “I did not find this particularly productive.” Most students took a neutral position, best captured by the individual who stated, “I felt like it was the typical classroom group project: even if each person is involved throughout the entire process, one person is still stuck compiling it all. It was, though, an excellent way for the three of us to get to know each other a bit more even if we’ve never seen one another.” Or as another student succinctly stated, these are “a necessary evil.” However, assessments such as the field experiences and interviews which required interaction with practicing professionals were rated much higher than activities promoting interaction with other students. Students rated these activities particularly high in terms of transferability, reflecting those ratings in their comments, such as:

“I got loads of real perspective that is hard to get from a book or lecture.” “I learned more from talking to a professional in the field than I would have from

reading about it.” “I learned more from the field experiences than anything.”

The only negative comments related to these assignments were that success depended on finding cooperative administrators who would facilitate the process and the concern that not all students have “stellar examples” to work with. Yet again, one student best captured the general feeling of the group, noting that “when doing these types of assignments, you are actually getting information that you can use because the people giving the information are in the position that we are trying to get to.” Work Samples. As a group, the work samples forced students to produce the types of documents they would produce as administrators, including campus plans, professional development plans, and teacher growth plans. These assessments meet the criteria (Johnson & Aragon, 2003) of addressing individual differences, creating real-life contexts, providing hands- on activities, and encouraging reflection. Students recognized the importance of these activities to their futures as administrators by the high transferability rating (74%) awarded; however, they did not rate them as particularly enjoyable or engaging. The student comments reflected these ratings:

“Some of these assignments were time consuming, but at the end I think they will give us a good perception of real world scenarios, which is something positive.”

“These assignments provide insight into real job scenarios that I can utilize and develop my skills as a professional.”

“They are very complex . . . [but] were like training for a real job.”

Journal of Interactive Online Learning Baily, Hendricks, and Applewhite  

   

122  

In fact, the complex nature of these assignments led seven students to mention the need for clearer instructions, with four students noting that a video-based explanation would complement the written instructions well.

Twitter Summary. Based on the combined ratings and the number of positive comments (14) this was the most “popular” assessment. For this assessment, students summarized book chapters as “tweets” (140 character or less statements). It fit the framework (Johnson & Aragon, 2003) by avoiding information overload and encouraging reflection. Students remarked:

“This assignment helps us keep up with the times that we live in and allows us to think outside the box and move away from comfort zones.”

“This assignment was challenging, but fun when I put my mind to it.” “Summarizing like this really helped me think about how I could apply the information.”

Additionally, seven students said they had “stolen” this assignment to use in their own classrooms, with one noting, “I totally stole this and have begun using it with my students! They love it, it is effective, and grading is actually interesting now!” Audio/Video production. Two assessments comprise this category. For one, students had to role-play a conversation explaining the information gleaned from a required reading, and for the other, students had to create a screencast with embedded video of themselves conducting a mock presentation. These assessments fit the framework (Johnson & Aragon, 2003) by addressing individual differences, motivating the students, creating real-life contexts, providing hands-on activities, and encouraging reflection. For both assignments, some students expressed concern about technological difficulties. All of the software for the audio was embedded in the course delivery system, while the students did have to locate and download free screencasting software to complete the video assignment. Most of the problems students encountered were software-related and were resolved when students updated their browser software, switched browsers, updated Flash player, or adjusted their JAVA settings. The success of the audio submission was tempered by the fact that many of the students were uncomfortable hearing a recording of themselves (6 comments), with one admitting it “was kind of weird.” Still, students recognized the value of converting the text to the spoken word, noting:

“I still remember this assignment and the material it covered. It has lasted.” “Not the ‘norm,’ but you truly had to know the information to have the pretend

conversation.” “It was an out of the box assignment. I liked that it was different. Plus, I am a talker. It

was nice to be able to do something orally and not just written.” Interestingly, this was the only assignment for which a student wrote that he or she “personally did not enjoy this assignment.” Unfortunately, there was no elaboration as to why. As was the case for the audio submission, many students (8) commented on the technological challenges they encountered while creating the screencast and posting to YouTube. However, most (6) followed up with a statement indicating how valuable the experience was because it forced them to use technology they should already know how to use, or as one comically stated, “to be like the average 6th grader.” Only one student reported having used these tools before. Overall, comments were positive, including:

“When I read the assignment I dreaded having to do it so I procrastinated, but I actually enjoyed it once I got started.”

“That was a new experience for me and took a TON of time, but I learned so MUCH.”

Journal of Interactive Online Learning Baily, Hendricks, and Applewhite  

   

123  

“Although the content covered and the synthesis involved was deep and thorough, and greatly needed, the technological aspect of it was incredibly frustrating. However, I’m chalking that up to a ridiculous machine on my end.”

“Learning something new is always great for me. If I can use it past the current class, that’s a bonus!”

Connections to Review of Literature

This study indicated students find assignments they view as innovative more enjoyable,

more engaging—and especially important to the business of preparing future educational leaders—more likely to create knowledge that is transferable to future practice. Schlechty (2001) maintained that students learn more when they work on assignments that mattered to them. Based on the student comments regarding the assignments evaluated in this study, the authors agree with that assertion. To wit, one student wrote in an unsolicited email to one of the researchers:

I shared almost everything I learned with my co-workers. I found these assignments to be thought provoking and reflective. Prompted me to look at things in a different way. Shifted my thinking and kept me engaged! I really enjoyed being evaluated in ways other than boring papers!

That sentiment is a salient reminder to continue the push to find more creative, unique, and innovative ways to engage students and tap into the intrinsic motivation arising from the meaningful character of the assessments (Vaughn, 2010).

Implications for Practice

As noted previously, the students responded in an overwhelmingly positive fashion to the

course as a whole. Even the lowest-rated assessment, the quizzes, received a 49% positive rating. When looking at the results, however, it is clear that students preferred assessments with some sort of innovative feature over the more traditional assessments (quizzes and papers), or as one student referred to them, the “throwback assignments.” When writing about the traditional paper, one student captured the spirit of the group by stating, “I understand there has to be a ‘variety’ of assignments. I was just spoiled with the ‘fun’ assignments so it wasn’t that this was bad, it just didn’t meet the same standard as the other assignments.” Considering the positive student responses to the assessment strategies under review, the online instructional strategy framework developed by Johnson and Aragon (2003) proved to be useful and effective tool for guiding assessment design.

Steinbronn and Merideth (2008) asserted that many professors are more comfortable using traditional methods found in face-to-face teaching environments. But professors are in the business of promoting student learning, or more specifically, in the business of creating work that will engage students and result in learning (Schlechty, 2001); they are not in the business of promoting professor comfort. As Redmond (2011) noted, professors need to redefine themselves and their practices in order to create highly engaging online learning environments. The overarching implication of this study is that online professors and online instructional designers must move away from their comfort zones in pursuit of more innovative instructional strategies and assessments that will engage students, because, through their own voices, this is what

Journal of Interactive Online Learning Baily, Hendricks, and Applewhite  

   

124  

students say they want. After all, a more engaged student is a student who will, ostensibly, learn more (Kearsley & Shneiderman, 1998).

Further, though students report some difficulties with the technologies involved, this study indicates that students are willing to “push [their] own envelopes,” as one commented, to complete assessments that engage them in novel ways and produce learning experiences extending beyond the actual course content. Online instructional designers should reciprocate by pushing their own envelopes to produce the types of assessments that students find most beneficial and most engaging.

Finally, students self-report the value of interacting with professionals in the field and participating in experiences that approximate, as closely as possible, authentic work-related tasks. Increasing the frequency, accuracy, and depth of these types of assessments will be important moving forward. All of the students in the program are working professionals with a specific end-goal in sight: certification. As such, they recognize the inherent value in tasks that equip them with the skills and knowledge required for future success, or as one student aptly noted, “I think these assignments are what you put into [them]. I gave 100% to these assignments so I could learn as much as possible and not look like a fool once I do get a job.”

Recommendations for Future Research

The researchers were satisfied with the initial findings of this project, but feel that an

extension of the study would allow for a greater understanding of how the students viewed the transferability of the methods into their own practice. Gender and age connections may also prove useful for strategically designing online courses. The need for pedagogical research to inform online instructors of how to better infuse their courses with highly engaging and meaningful teaching as assessment strategies is needed, as much of the current research simply measures how traditional practices transfer into the online learning environment. This study indicated that students positively engaged in activities that were uniquely situated within an online space; therefore, additional research into the use of creative assessment approaches would be valuable.

Journal of Interactive Online Learning Baily, Hendricks, and Applewhite  

   

125  

References

Anderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing teacher presence in a computer conferencing context. Journal of Asynchronous Learning Networks, 5(2), 1-17.

Akyol, Z., & Garrison, D. R. (2008). The development of a community of inquiry over time in an online course: Understanding the progression and integration of social, cognitive and teaching presence. Journal of Asynchronous Learning Networks, 12(3-4), 3-22.

Cunningham, J. (2010). Self-direction: A critical tool in distance learning. Common Ground Journal, 7(2), 89-100. Retrieved from www.commongroundjournal.org

Edwards, S. (2012). Teaching through assessment: Reconsidering the transfer problem through a convergence of technology and assessment in the early childhood teacher education. Teachers and Teaching: Theory and Practice, 18(5), 585-599. doi: http://dx.doi.org/10.1080/13540602.2012.709733

Gayton, J. & McEwen, B. C. (2007). Effective online instructional and assessment strategies. American Journal of Distance Education, 21(3), 117-132. doi:http://dx.doi.org/10.1080/08923640701341653

Johnson, S. & Aragon, S. (2003). An instructional strategy framework for online learning environments. New Directions for Adult and Continuing Education, 2003 (100), 31-43.

Kearsley, G. & Shneiderman, B. (1998). Engagement theory: A framework for technology-based teaching and learning. Educational Technology, 38(5), 20-23.

Laurillard, D., Charlton, P., Craft, B., Dunajioiykism D., Ljubojevic, D., Magoulas, G., . . . Whittlestone, K. (2011). A constructionist learning environment for teachers to model learning designs. Journal of Computer Assisted Learning, 29(1). doi:10.111/j.1365-2729.2011.00458.x

Moore, J. L., Dickson-Deane, C., & Galyen, K. (2011). E-Learning, online learning, and distance learning environment: Are they the same? Internet and Higher Education, 14, 129-135. doi:10.1016/j.iheduc.2010.10.001

O’Brien, H. L. & Toms, E. G. (2008). What is user engagement? A conceptual framework for defining user engagement with technology. Journal of the American Society for Information Science and Technology 59(6), 938-955. doi:10.1002/asi.20801

Redmond, P. (2011). From face-to-face to online teaching: Pedagogical transitions. In G. Williams, P. Slatham, N. Brown & B. Cleland (Eds.), Changing Demands, Changing Directions. Proceedings ascilite Hobart 2011. (pp. 1050-1060). Retrieved from http://www.ascilite.org.au/conferences/hobart11/downloads/papers/Redmond-full.pdf

Russell, T. L. (1999). The No Significant Difference Phenomenon. Raleigh, NC: North Carolina State University Press.

Schelchty, P. (2001). Shaking up the schoolhouse: How to support and sustain educational innovation. San Francisco, CA: Jossey-Bass.

Steinbronn, P. & Meredith, E. (2008). Perceived utility of methods and instructional strategies used in online and face-to-face teaching environments. Innovative Higher Education, 32(5), 265-278.

Vaughn, N. D. (2010). A blended community of inquiry approach: Linking student engagement and course redesign. The Internet and Higher Education, 13(1-2), 60-65.

Whitlock, Q. (2001). Course design for online learning: What’s gone wrong. In J. Stephenson (Ed.), Teaching and learning online: Pedagogies for new technologies (pp. 182-191). London, England: Kogan Page.

Copyright of Journal of Interactive Online Learning is the property of Board of Trustees of the University of Alabama and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.