READ 3 articles

VOLUME 5 NO. 3 (2015) Journal of APPLIED LEARNING TECHNOLOGY 6

Applying A Collaborative Support Model to Assess E-Learning Technologies

Kari Burch, Jason Crustals, and Ping Lieser

ABSTRACT - With the Bring Your Own Devices evolving pace, the key question is how we can support students and meet their expectations. In particular, competency-based education is now an increasingly important trend that gives students control over their competency devel- opment (Mazoue, 2014). Our goal was to develop a systematic approach to e-learning technology integration, to solve problems for learning with technologies, and to derive instructional design principles that can inform future development and implementation decisions in blended learning. We high- lighted the development of a 1-2-3-4 guiding rubric for e-learning technology integration. A pilot was conducted to examine the theoretical ideal and the practical affordances. This study emphasized team collaboration to optimize use of campus technology services in the learner-centered BYOD environment.

Keywords: Blended learning, web-based learning, e-learning, connected learning, mobile learning (m-learning), technology adoption, Electronic Voting Systems (EVS), BYOD (Bring Your Own Device).

Introduction E-learning technologies have evolved over time,

becoming easier to use and more popular in higher education. In 2005, the Higher Education Fund- ing Council for England had a vision that “e-learn- ing has been criticized for being technology led, with a focus on providing materials, but has rela- tively recently focused more on the learner and enabling students and other users to develop more independence in learning and to share re- sources. This change matches the developments in pedagogy and the increasing need to support diversity and flexibility in higher education.” (High- er Education Fund Council for England, 2005: p. 4)

With the rapid development of Internet-capa- ble and mobile devices, e-learning technologies have been growing in popularity among students. Many students are frequently using the technolo- gies they encounter in their daily lives. In the 2013 report of the ECAR Study of Undergraduate Stu- dents and Infor mation Technology, “despite smartphones’ ability to distract, about half of re- spondents (49%) said they want their instructors to integrate the use of smartphones into class more. Even more students wanted tablets (51%) and laptops (61%) integrated into class.” (Dahl- strom et al. 2014), A recent report of the Study of Faculty and Information Technology, published by ECAR, indicated that “Technology is omnipresent in the lives of students—92% of students who own Internet-capable devices have at least two devic- es and 59% own three or more.” As we can see, the adoption of BYOD (Bring Your Own Device) is

in an almost ubiquitous state. At the same time, the rise of the BYOD movement creates challeng- es for planning e-learning integration and sup- port requirements.

As the focus of technology in education has shifted to learner-centered in the BYOD context, the goal for our project is about transforming the implementation support model that centers around the collaborative team-based community to help one another adjust current practices. Therefore the framework for this project is based on two basic desires: the need for promoting learn- er-centered pedagogies for e-learning and the need for collaboration among faculty, students, and IT staff. And it is in our best interest to improve stu- dents’ overall learning experiences and to devel- op a systematic approach to supporting e-learn- ing within this framework.

In addition to learning management systems on our campus, the need for the use of additional e- learning technologies has increased among facul- ty and students over the previous two years. The use of these technologies has implications for ped- agogy and support concerns. To examine the ef- fectiveness of these technologies, we wanted to quantify users’ feedback based on two research questions: (1) How do we identify the teaching and learning methods that work best with certain technologies? (2) How do we identify the best prac- tices to support e-learning technology implemen- tation? In this paper we described the develop- ment of a 1-2-3-4 guiding rubric for e-learning technology integration. We began this study with

VOLUME 5 NO. 3 (2015) Journal of APPLIED LEARNING TECHNOLOGY 7

a preliminary evaluation of many popular e-learn- ing technologies that are utilized among faculty and students. We selected Piazza for this pilot study because it provided features that could meet relevant pedagogical needs. For example, Piazza served as an online 24/7 hub for students and faculty to post questions and answers, and it also provided a user-friendly polling feature. Utilizing the Piazza technology in different teaching prac- tices, we conducted a pilot based on two learning interactions: (1) synchronous interactive commu- nication (using the polling feature); and (2) asyn- chronous online communication (using the on- line discussion forums). We analyzed the survey results regarding the attitudes toward such tech- nology in learning. We followed up by outlining comments and feedback.

A 1-2-3-4 Guiding Rubric for e- Learning Technologies Integration

Our team wanted to design a systematic ap- proach of assessing instructional technologies and present a review of best practices in which the se- lected technology can mediate active learning. The 1-2-3-4 concept has evolved as our guiding mod- el (as shown below in Figure 1). First and fore- most, we looked at current teaching technology needs and expectations associated with our fac- ulty and the campus IT professionals. We identi- fied the elements for selecting a tool and assess- ing it.

Second, we looked at the nature of the avail- able web-based technologies. Top on our list, we selected instructional technologies that provided both synchronous and asynchronous pathways to promote communication and active learning. We looked at both pathways to pinpoint pros and cons for both synchronous interactivity and asyn- chronous engagement.

Third was an invaluable partnership and sup- port in the BYOD environment, including the ap- plication support, teaching support, and house- keeping support from the campus IT staff.

Finally, we looked at the design principles for learning environment design. In particular, we con- sidered Mizuko Ito and her coauthors’ design prin- ciples for connected learning environments (Ito et al., 2013).

Taken together, all the above elements created a focus in which we build our guiding model.

I. Choosing a Tool Considering our 1-2-3-4 experiment, there are

obstacles with which we need to contend: • Decide which device or app is the best to

use. • Keep in compatibility with the institution’s

enterprise network.

• Keep in sync with the adoption of emerging technologies among faculty and students in the classroom.

• Keep the implementation at a realistic level. Some stakeholders may not agree and be happy with new changes. Some skeptics may not be interested in the top new tools for learning and teaching. While assessing a tool, we must ask: Can it

enhance learning and teaching? Can it promote critical thinking rather than just content delivery? We want to know how it can be integrated into students’ learning or group activities. We want to know if students can apply what they have learned. We want to evaluate students’ contextual activi- ties and see whether students use this tool as a stepping-stone to demonstrate their understand- ing of what they know.

In addition, there are four factors we need to consider. (1) We think about why is the educa- tional tool significant? We need to focus on the student-centered approach, which is the key ele- ment in the BYOD environment. In other words, the implementation of any instructional technolo- gy should allow instructors to organize technolo- gy use around students’ learning activities rather than just to use it as a repository for their cours- es. (2) We think about the risks associated with implementing a new tool. As we mentioned earli- er, stakeholders may not be happy with new chang- es. (3) We think about its growing capability and scalability. Can we utilize the tool with a broaden- ing pool of mobile devices? We could never em- phasize enough the importance of using open stan- dards to enhance the interoperability among a wide range of technologies. We also need to think about its future implementation 5 or 10 years down the road. Finally, (4) we think about the implications for teaching and learning. The is- sues surrounding evidence-based practices and learning effectiveness have been of particular in- terest to medical education over the past few years. We need to look at existing success models and best practices.

II. Providing Two Types of Interaction

Emerging technologies provide greater commu- nication and interaction in teaching and learning. Not only do we want to know if students use tech- nologies to access information in a productive way, but at the same time we should encourage stu- dents to use technologies as an effective way to increase participation and interaction in learning. When we integrate the use of web-based technol- ogies and apps into face-to-face synchronous learning and online asynchronous communica- tion, it is our goal to understand if students trans- form their learning into higher level cognitive pro-

VOLUME 5 NO. 3 (2015) Journal of APPLIED LEARNING TECHNOLOGY 8

cesses, such as questioning and reasoning. We also want to know whether synchronous interac- tivity will make the learning process an enjoyable experience for students (on a superficial level); and whether the asynchronous engagement will have a positive effect on student learning (on a more substantive level).

III. Providing a Collaborative Partnership in the Context of BYOD

As the use of e-learning technology in schools continues to grow exponentially, so do the de- mands for a team-based support partnership. Ideally, in the beginning of the adoption process, we put technical support in control of the learn- ers and allow for trial and error. From applica- tion support, to individual students and the in- structors, as well as to the instructional technolo- gy staff, we should work collaboratively with each other to provide a partnership-based IT service on campus. With this partnership, we can over- come learning curves and challenges in a more cost effective way.

IV. Applying Four Design Principles

With emerging technologies, learners are con- stantly challenged to adopt a new set of strategies to learn and to discover new ideas and resources. As educators, we need to identify and develop a more effective learning environment that can pro- vide multiple interfaces for the learners to meet their needs. To start with, we look at Mizuko Ito and her coauthors’ design principles for using new media to expand learning opportunities (Ito et al. 2013). These include: Fostering engagement and participation, learning happens by doing, expand- ing sharing and peer supports for interests, and expanding diversity. Importantly, from the sup- port points of views, we also look into criteria that Hodges and Clark have suggested, such as: ease of use, accessibility, security, and nature of sup- port resources available for the tool (Hodges & Clark, 2011).

Given these design principles, here are some of the questions to guide our project design process:

Principle 1. Fostering engagement and participation. Provide different ways for individuals and groups to contribute.

Questions for reflection: • How do we support personalized technolo-

gies to meet individual needs? • What tools can support self-paced learning? • How can we make communication inviting

and engaging?

Principle 2. Learning happens by doing. Learning is experiential and part of the pursuit of meaningful activities and projects.

Questions for reflection: • What tools can support self-directed learn-

ing? • What tools can provide customizable online

resources and personalized search results to meet individual needs?

• What tools can support secured access con- trol? A tool’s ease of use; technical and legal requirements, such as accessibility; and se- curity (Hodges and Clark, 2011).

Principle 3. Expanding sharing and peer supports for interests. Interest or cultivation of an interest creates both a “need to know” and a “need to share.”

Questions for reflection: • What tools can support group interaction? • What tools can allow for mentoring? • What tools can provide functions for peer

review?

Principle 4. Expanding diversity - everything is interconnected. Learning environments should provide individuals with multiple learning contexts, opportunities, and tools for planning and reflecting.

Questions for reflection: • What tools can support a network for diverse

learners and experts? • Tools’ interoperability within an enterprise-

wide network.

Figure 1. 1-2-3-4 Approach to instructional technology integration support model

Research Questions and Pilot Design

Research Questions We wanted to see whether students had recom-

mendations on how we could use Piazza effective-

VOLUME 5 NO. 3 (2015) Journal of APPLIED LEARNING TECHNOLOGY 9

ly. Ultimately, our goal was to understand the user relationship and attitude toward the utiliza- tion of e-learning technologies and support con- cerns. Our research questions were: 1. How do we identify the teaching and learning

methods that work best with certain technol- ogies?

2. How do we identify best practices to support e-learning technology implementations?

Pilot Design The research teams recognized that the faculty

in the Medical school have been utilizing Piazza primarily as an asynchronous communication tool, whereas the faculty in the Occupational Therapy prefer using Piazza’s polling feature as an electronic voting system (EVS) to engage stu- dents. Therefore, the pilot was divided into two parts based on these two different interactions. Part A was designed to test the polling feature for synchronous learning activities, and Part B was intended to understand both faculty and students experiences of their online communication. Our best interest is to identify the e-learning effective- ness on either a superficial level of enjoyable learn- ing experience or a substantive level of positive learning interaction.

Part A: Synchronous interactive communication (using Piazza’s polling feature)

Objective. The idea behind this pilot is that students use their Wi-Fi connected personal de- vices to answer polling questions, allowing the in- structor to collect and display active feedback within minutes. The use of such polling technol- ogy has implications for preliminary checking of existing understanding or preparation (Herr 1994; Burnstein & Lederman, 2001; Simpson & Oliver, 2006), and verifying comprehension (Elliott, 2002).

Methods. During the past year, the IT depart- ment upgraded the network infrastructure and made significant enhancements to the depart- ment’s Wi-Fi access points and bandwidth re- quired for more than 300 simultaneous users. With this pilot, we were able to explore the ap- proaches and boundaries in terms of what we should anticipate when using various Wi-Fi en- abled devices at the same time.

This pilot has taught us how to prepare a cohe- sive workflow so that the instructor can toggle eas- ily between presentations, post polls, and reveal polling responses. A practice run is important and necessary in training instructors and students to adopt new technologies. In the first step for our first polling session, we posted a practice ques- tion: “your preferred devices for polling” in order to understand what types of computers and mo- bile devices our second year students are using.

The practice question allowed students to set up their Piazza logins, prepare their computes and/ or mobile devices, and get familiar with the Piazza polling interface.

Student and instructor experiences. We relayed feedback from students and teachers who participated the pilot:

“Use of the Piazza polling system was practical and beneficial to both students and instructors”.

“For the instructor, entering questions into the poll system was straightforward. The polling sys- tem requires the students to select an answer out of provided options, and is conducive to both true/ false or multiple choice question types. The open ended formatting of questions allows for any amount of creativity on the part of the instructor. Advanced planning was required on the part of the instructor, but proved useful during implemen- tation in class. The system allows for flexibility for timing of posting as well as result viewing for stu- dents. Using the Piazza polling system was use- ful for obtaining immediate information about stu- dent understanding of course material in real time. In addition, using the polling system was benefi- cial for increasing student participation during a review session. Students were required to partici- pate and motivated to do so because participation was monitored by the system. In addition, stu- dents benefitted through receiving immediate feed- back and were also provided an objective mea- sure of performance; this was helpful for each stu- dent to know whether she or he was on track or needed to review course material to a greater ex- tent.”

“The Piazza polling system was very useful as a review session tool. Piazza would likely also be a beneficial tool to use at the end of lectures or before lectures to gauge student learning and to promote greater participation and retention of in- formation throughout the learning process.”

Outcomes. Our goal of using polling was to enrich learning interactions and help the instruc- tor to respond to student needs for clarification and practice (Duncan, et.al.). After two polling sessions completed, we identified best practices and the pedagogical implications. This experi- ence not only allowed us to understand faculty’s and students’ attitudes toward Piazza polling in the BYOD, but also enriched our emphasis of a valuable team effort among IT staff, students, and the instructors. The comments and feedback demonstrated that the Piazza polling feature is beneficial in assisting faculty in facilitating learn- ing and class discussion. For example, the poll- ing results indicate that 43% of students voted “helpful” when asked their perception of this poll- ing technology in reviewing content for their mid- term exam. (See figure 2)

VOLUME 5 NO. 3 (2015) Journal of APPLIED LEARNING TECHNOLOGY 10

Figure 2. The polling results for the question #10: “How helpful did you find this technology polling system, in which you were able to see responses of the class, to review for the midterm exam? Please rate on a scale from 1 (not helpful at all) to 5 (extremely helpful)“

“Ability to learn from other students’ questions/ notes, increased instructor engagement.” (Student empowerment)

“Easy to see if someone else has the same ques- tion as you or to see questions and answers that other students have.” (Cognitive and exploratory learning)

“Good for Pediatrics circle, square, triangle to help me review my notes and keep up with the material.” (Cognitive and exploratory learning)

“It’s nice to see questions and answers that oth- er students have.” (Critical thinking skills)

Outcomes. The outcomes of the part B pilot were summarized based on the feedback from the 20 participating students. The majority of the stu- dents identified that Piazza is easy to navigate and find answers to both their own questions and ques- tions posted by their instructor, TAs, and peers. Students liked the simple interface to interact with one another 24/7 and discuss medical cases. Students suggested that instructors should have additional training opportunities in order to use Piazza more effectively. The overall feedback in- dicated that both instructors and students bene- fit from asynchronous discussions. The effective use of such asynchronous discussion forums did have substantial implications for promoting learn- er-center learning.

Survey Results and Summary The results of both part A and part B showed

that the use of Piazza as an asynchronous and synchronous educational tool had a positive im- pact on the adoption of blended learning among faculty and students. Although we did not deploy Piazza as a full enterprise system, this web-based tool addressed the needs related to teaching and learning during the previous year by providing our faculty and students with an asynchronous dis- cussion feature and a synchronous polling fea- ture. Overall, both 24/7 discussion forums and in-class polling inquiries were considered as help- ful assignments and engaging learning activities for our faculty and students. Our team agreed that the selection process was fairly flexible in choosing an e-learning tool with our proposed 1-

Part B: Asynchronous Engagement (using Piazza’s Q&A forum)

Objective. The idea behind this part of the pi- lot is to show how to manage a web-based tool (Piazza) in the class by looking at the online inter- actions, including students’ interactions with the instructor, with content, and with peers. The ef- fective use of such asynchronous discussion tech- nology has implications for promoting learner-cen- ter learning, such as student empowerment (Kas- sop, 2003; Kubala, 1998), cognitive and explor- atory learning (Haggerty et. at., 2001), and critical thinking skills (Collison et al., 2000; Eklund & Eklund, 1996; Newman et al., 1999; Shapley, 2000).

Methods. Two surveys were used to assess instructors’ and students’ perceptions of using Piazza and their learning interactions. The instruc- tor group consisted of those faculty members and course masters who implemented Piazza in addi- tion to other campus-wide teaching resources. There were roughly 200 students who had enrolled in Piazza courses. At the end of the fall semester, students and faculty received an e-mail invitation to the online surveys with questionnaires regard- ing the effectiveness of using Piazza for teaching and learning.

Student and instructor experiences. “That it is a free tool; that it is relatively easy to

set up and use; that in it’s typicaly format, it is a “student-centered” tool where students can con- trol what questions are asked along with the abil- ity to ask them anonymously. And of course the collaborative nature of building answers, both from the student and faculty side of things.” (Student empowerment)

“Peer to peer responses are the “best”.” (Student empowerment)

“Right now, all of our courses use it but with varying effectiveness. Generally, we like it because it allows students to immediately and anonymous- ly post questions and then share those answers with the rest of the class. It’s also useful for pro- fessors as a way to send out documents and ques- tions to the class.” (Student empowerment)

VOLUME 5 NO. 3 (2015) Journal of APPLIED LEARNING TECHNOLOGY 11

2-3-4 guiding method, highlighting Ito’s four de- sign principles with emerging learning technolo- gies.

This pilot study also reaffirmed that the best practices for e-learning technology integration emphasize a great deal on working toward a part- nership among people in technology and in the learning community. Whether it is on a small scale of flipping the classroom or on a larger scale of building the learning community, using a team- based support model as the framework is the key to developing an ideal hub for instructors and stu- dents to discuss and share their learning experi- ences.

The challenge for our project team is adjusting the faculty’s mindset related to incorporating tech- nology into their current teaching practices. Rather than just helping faculty and students use educa- tional technology effectively, we should strength- en our approach to promoting learner-centered experiences in order to have a positive influence on students’ learning and involvement. That said, for future e-learning technology integration, the main focus should be to continuously evaluate how students benefit from e-learning technology with respect to the types of instructional methods used, as well as to identify the best way to increase the quality of their active learning engagement.

References Burnstein, R. A., & Lederman, L. M. (2001). Using

wireless keypads in lecture classes. The Phys- ics Teacher, 39(1), 8-11.

Collison, G., Elbaum, B., Haavind, S., & Tinker, R. (2000). Facilitating online learning: Effec- tive strategies for moderators. Atwood Publish- ing: Madison, WI.

Dahlstrom, Eden, J.D. Walker, Dziuban, Charles. (2013). ECAR Study of Undergraduate Students and Information Technology, 2013, EDUCAUSE research report (Louisville, Colo.: EDUCAUSE Center for Analysis and Research, September 2013). Retrieved March 20. 2015, from https:/ /net.educause.edu/ir/library/pdf/ERS1302/ ERS1302.pdf

Dahlstrom, Eden, Bichsel, Jacqueline. (2014), ECAR Study of Undergraduate Students and Information Technology, 2014, EDUCAUSE re- search report (Louisville, Colo.: EDUCAUSE Center for Analysis and Research, October 2014). Retrieved March 20. 2015, from http:// n e t . e d u c a u s e . e d u / i r / l i b r a r y / p d f / s s 1 4 / ERS1406.pdf

Duncan, D. (2005). Clickers in the classroom how to enhance science teaching using classroom response systems. (Vol. 1).

Elliott, C. (2001). Case study: Economics lectures using a personal response system. Economics Learning, Support and Teaching Network. Re-

trieved March 12, 2008, from http:// www.economicsnetwork.ac.uk/showcase/ elliott_prs.htm

Eklund, J., & Eklund, P. Integrating the web and the teaching of technology: Cases across two universities. (1996). In M. Nott (Ed.), Proceed- ings, Australian World Wide Web Conference. Gold Coast, AUS: Southern Cross University.

Haggerty, N., Schneberger, S., & Carr, P. (2001). Exploring media influences on individual learning: Implications for organizational learn- ing. In J. DeGross, S. Sarkar, & V. Storey (Eds.), Proceedings, International Conference on Infor- mation Systems (pp. 13-22). New Orleans, LA.

Herr, R. B. (1994). Computer assisted communi- cation within the classroom: Interactive l e c - turing.

Higher Education Fund Council for England (2005). Review of the 2005 HEFCE strategy for e-learning. Retrieved March 20. 2015, from http://www.hefce.ac.uk/data/Year/2008/ Review,of,the,2005,HEFCE,Strategy,for,e- Learning/Title,93074,en.html

Hodges, C. B., and Clark, K. F. (2011), “Selecting a Web 2.0 Presentation Tool,” LMC: Library Media Connection. Retrieved March 20. 2015, from http://www.librarymediaconnection.com/ p d f / l m c / r e v i e w s _ a n d _ a r t i c l e s / f e a t u r e d _ a r t i c l e s / Hodges_November_December2011.pdf

Ito, Mizuko; Kris Gutiérrez, Sonia Livingstone, Bill Penuel, Jean Rhodes, Katie Salen, Juliet Schor, Julian Sefton-Green, and S. Craig Wat- kins. Connected Learning: An Agenda for Research and Design. Digital Media and Learning Research Hub. Irvine, CA. January 2013. Retrieved March 20. 2015, from http://dmlhub.net/wp-content/uploads/files/ Connected_Learning_report.pdf

Kassop, M. (2003, May/June). Ten ways online education matches, or surpasses, face-to-face learning. The Technology Source.

Kubala, T. (1998). Addressing student needs: Teaching on the internet. THE Journal, 25, (8), 71- 75.

Mazoue, James G. (2014). “Beyond the MOOC Model: Changing Educational Paradigms.” EDUCAUSE Review.

Newman, D.R., Webb, B., & Cochrane, C. (1999). A content analysis method to measure critical thinking in face-to-face and computer support group learning. Retrieved March 20. 2015, from http://www.umsl.edu/~wilmarthp/mrpc- web-resources/CA-analysis-method-to- measure-thinking-in-f2f.pdf

Shapley, P. (2000). Online education to develop complex reasoning skills in organic chemistry. Journal of Asynchronous Learning Networks, 4(2), 55-65.

VOLUME 5 NO. 3 (2015) Journal of APPLIED LEARNING TECHNOLOGY 12

Simpson, V. & Oliver, M (2002). Using electronic voting systems in lectures. UCL Inter nal report. Retrieved March 20. 2015, from h t t p : / / t l c . z m m l . u n i b r e m e n . d e / r e s o u r c e _ f i l e s / r e s o u r c e s / 3 8 6 / ElectronicVotingSystemsin_lectures.pdf

About the Authors

Kari Burch is an occupational therapist at BJC HealthCare. Her research interests include cog- nition and its impact on daily performance in per- sons with chronic conditions. She received her doctorate degree in occupational therapy from Washington University School of Medicine. [email protected]

Jason Crustals is a Manager of Educational Technology in the Office of Education at Washington University School of Medicine. [email protected]

Ping Lieser is an Educational Technology Co- ordinator in the Occupational Therapy Program at Washington University School of Medicine. Her research interests include blended learning, e- learning, interactive course design, and online collaboration. She received her Ph.D. degree with a concentration in Educational Computing, De- sign, and Online Learning from Kansas State Uni- versity. [email protected]

Copyright of Journal of Applied Learning Technology is the property of Society for Applied Learning Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.