paper

THEUSABILITYANALYSISOFANE-LEARNINGENVIRONMENT.pdf

Turkish Online Journal of Distance Education-TOJDE October 2015 ISSN 1302-6488 Volume: 16 Number: 4 Article 5

THE USABILITY ANALYSIS OF AN E-LEARNING ENVIRONMENT

Fulya TORUN Faculty of Education

Adnan Menderes University, Aydın, TURKEY

Dr. Hakan TEKEDERE Health Services Vocational School

Gazi University, Ankara, TURKEY ABSTRACT In this research, an E-learning environment is developed for the teacher candidates taking the course on Scientific Research Methods. The course contents were adapted to one of the constructivist approach models referred to as 5E, and an expert opinion was received for the compliance of this model. An usability analysis was also performed to determine the usability of the e-learning environment. The participants of the research comprised 42 teacher candidates. The mixed method was used in the research. 3 different data collection tools were used in order to measure the three basic concepts of usability analyses, which are the dimensions of effectiveness, efficiency and satisfaction. Two of the data collection tools were the scales developed by different researchers and were applied with the approval received from the researchers involved. On the other hand, the usability test as another data tool was prepared by the researchers who conducted this study for the purpose of determining the participants’ success in handling the twelve tasks assigned to them with respect to the use of e- learning environment, the seconds they spent on that environment and the number of clicks they performed. Considering the results of the analyses performed within the data obtained, the usability of the developed e-learning environment proved to be at a higher rate. Keywords: e-learning, scientific research methods, usability analysis. INTRODUCTION The proximity of the technological and educational developments to one another has enabled the e-learning innovations to come forward in higher education and allowed people to focus on e-learning in the 21st century. The contributions of the new, extensive and powerful communication technologies to education and their capacity of constituting the community of learners along with the skill to keep them online have made it possible to lay the foundations of E-learning in higher education (Garrison, 2011). E-learning , both within the structure of the Turkish Council of Higher Education (YOK) and within private institutions, has attracted much attention through its popularity in accordance with the demands in this field. There are many developed e-learning environments. The design process is of great importance in order to utilize these environments in an efficient way.

The objective with regard to this subject will be achieved by being in the know of the current impacts of the environments developed within the frame of information systems and by putting this power into service (Stair and Reynolds, 2012). When an educational environment is desired to be developed, the first goal would be to explore what is learnt and the nature of the knowledge acquired as much as possible and, then, to devise the environments in the way that this acquired knowledge could be applied (Dillon and Zhu,1997). Achieving the objectives of the environments created for education depends on the capability of these environments to provide a good interaction with the users who are also the learners of this process. For this reason, it is necessary that usability analyses be performed in order to be able to measure the extent to which the e-learning environments can be applicable. In this research, it was targeted that the compulsory course on Scientific Research Methods (SRM) assigned to the learners through every curriculum in the faculties of educational sciences be devised as a learning-centered course in the e-learning environment, and it was also aimed to determine at what level this developed environment is applicable on the basis of the user. The course contents were adapted to one of the constructivist approach models, 5E, in the e-learning environment, and an expert opinion was received to confirm its compliance. Thus, it was aimed that the learners become more active in their course processes. In order to determine usability, an usability scale by means of which an interface usability could be measured was utilized, while an usability test through which the use of the educational environment could be measured and an environmental attitude scale to determine the attitudes towards the environment were used, after which the results were interpreted together, and to what extent they could be applied was explained. THEORETICAL EXPLANATIONS E-learning and Constructivism Sometimes gaining an experience in some cases may take a much longer time due to the circumstances one is in. However, e-learning provides the learners with the solution of the problems rarely encountered in real life or with the possibility to complete a long- term task within hours (Clark and Mayer, 2011). In this way, the learners can also get access to a given course any time anywhere as well as communicating with the others working or studying in the field of education. Web environment, apart from economizing on the expenses of the printed media, makes it possible to organize information in a short time, update them and allow all the students to reach these adjustments instantly (Usun, 2006). The objective of e-learning is to create a community utilizing and questioning information and communication technologies, independently of time and space. A community with a questioning mind is the one that can collaboratively make sensible criticisms, and construct them by reflecting them on personal opinions as well as showing a common understanding for the matter involved (Garrison, 2011). The advantage of e-learning, as in any other learning program, is to construct courses that are in compliance with the learning processes of an individual. In order for this to be effective, the learning strategies should support these processes (Clark and Mayer, 2011). Technology enables learners to establish a bond with the learning process and become encouraged in order to think during this process (Jonassen et al, 2003). Those who learn something in the e-learning environment acquire knowledge not from the observations or interpretations of an instructor but from the first-hand sources by experiencing it. In formal education, the instructor may personalize the information, whereas in the e-

learning environment, the learners are given the opportunity to construct the acquired knowledge by themselves. Throughout the educational program, the control of the process in the hands of the learner while the instructor is only responsible for acting as a guide throughout the process. In the education process where the constructive approach is taken as the basis, the learning process should be made reasonable for the student (Cakir et al, 2008). The fact that the learner is in interaction with the content and materials arranged in an e-learning environment which is prepared on the basis of constructivist approach is one of the important factors to be taken into consideration for the sake of the effectiveness in the learning process. Bonk (2002) developed a model consisting of 9 stages to generate motivation, which were as such: preparing the environment, encouragement, curiosity, diversity and novelty, autonomy, care, learning through interaction, challenge, and product (Cited by Cakir et al, 2008). By the time these 9 stages of the model are put into practise at the design phase of the learning process, an effective learning environment will have been formed for the learners. 5E - Educational Model Although the convictions on educational models are quite old-fashioned, their use and administration have recently been on the increase (Bybee et al, 2006). With the learner- centered educational system coming into prominence in particular, it is aimed through a number of studies performed on these models that the learners gain acquisitions more effectively. Of these models, one of the most commonly used one is 5E. Being a constructivist model of approach, 5E was created by Roger BYBEE, who was the project coordinator of The Biological Science Curriculum Study (BSCS). The 5E model has been used in the design of BSCS educational curriculum materials since late 1980s. This model has a range of educational sequences that can be used in all the curriculums, special topics and individual courses. The 5E model plays a major role in the process of developing an educational program (Bybee et al, 2006). With the adoption of 5E model in educational programs, the students are expected to be more active during the lesson. In fact, it is aimed with this model that the students’ curiosity and passion for research be enhanced and that they be allowed to use the knowledge and skills they have learnt and to create new concepts or products in their minds. With respect to this, Senturk (2010) stated that the 5E model, through the researches performed, was seen to have increased the learners’ success and enabled them to improve themselves in a conceptual sense and changed their attitudes towards the courses in a positive way, as well. The 5E model has derived its name from the English initial letters of the concepts which symbolize the stages of this model (Engage, Explore, Explain, Elaborate, Evaluate). These steps mentioned are in the form of a sequence through which they follow one another consecutively. The stages of the 5E model are as follows:

Ø Engage: The students’ attention is drawn to the subject at issue, their interest and curiosity in the subject is aroused, and they are allowed to think over the topic by posing questions in their minds (Senemoglu, 2010).

Ø Explore: The students are allowed to have the chance to make observations and predictions, to plan and arrange experiments, to create graphics and interpret the results (Senemoglu, 2010).

Ø Explain: The students are offered models, principles and concepts. The teacher guides the students in reaching consistent and valid generalizations in line with the concepts presented to them (Senemoglu, 2010).

Ø Elaborate: The students transfer the concepts they have learnt at the stage of description when they reach this stage (Senemoglu, 2010). This stage enables the students to classify and explain new information and apply it to similar situations (Kanli (2009). Cited by Tufekci-Aslim, 2011).

Ø Evaluate: The students are encouraged to evaluate their own talents and levels of comprehension. The teachers can evaluate the progress made by the students in line with the acquisitions (Pektas, (2008); Kanli, (2009). Cited by Tufekci-Aslim, 2011).

Human-Computer Interaction (HCI) and Usability The conventional design studies touches on the fact that it is sufficient to follow the rules of a good design in order to be able to acquire magnificent designs; yet, this is not sufficient. Talented designers may devise and perform better designs when compared to others, however, being experienced in a certain field does not mean that there will be no error or no wrong implications as to the expectations of the user (Brink et al, 2002). The ‘user’ factor must be taken into consideration while devising a system, because when a system is being developed, the fact that the system in question has been arranged to comply with the user is of importance in terms of the functionality of that system. For this reason, the ‘user’ factor must always be taken into account at each stage of the development process. Another important factor is use of the web environment by the user, which involves all the connections of the user’s own environment. This environment covers the physical and cognitive domains. While the physical domain represents the space in which the user exists, the cognitive domain stands for the mental processes experienced by the user. When the system is devised by taking into account the factors just mentioned, the usability of the resulting product will be at a higher rate (Shackel and Richardson, 1991; Badre, 2002). The evaluations on usability is always important for designers, since the created or existing system may apply to the new technology or there may be the need to establish a relevance between the existing system and the former technologies, and moreover, the designer may not always cope with an adaptation. When, on the other hand, all the conditions have been provided and the development stage of the system is complete, it is important to learn how efficiently the system is functioning (Shackel and Richardson, 1991). Hence, an usability analysis must be performed to understand the functionality of the whole system. Usability is not merely a case related to the user interface profile. It is concerned with the way the system interacts with the user along with five basic elements: learnability, effectiveness, memorability, error and satisfaction (Ferre et al, 2001). The explanation of each is as follows (Nielsen, 1993):

Ø Learnability: The system must be easy to learn. In this way, the user will be able to complete a task over the system rapidly.

Ø Effectiveness: The system must be applicable in an effective way. In this way, it is possible that the user learning the system be highly productive.

Ø Memorability: The system must be easy to remember. In this way, an average user will not have to learn everything over again once s/he returns to the system after a while.

Ø Errors: The system must make a low rate of errors. In this way, the users will make fewer mistakes while using the system and will be able to eliminate such errors more easily. However, the errors that may cause serious problems should not exist within the system.

Ø Satisfaction: Using the system should be enjoyable. In this way, the users will be satisfied on a subjective dimension and will appreciate the system.

The evaluation of usability is an activity which exists at the center of the usability process. It determines the usability level of the current system as well as showing how the design functions. (Ferre et al, 2001).The Methods Applied in the Process of Evaluation (Ferre et al, 2001):

Ø Usability Test: The activities of the users are recorded for analyses. It is hard

to understand whether or not the system is applicable without working with real users.

Ø Thinking Aloud: The user is requested to think aloud while performing activities, and in this way the data is collected. This method is quite important in devising the system interaction.

Ø Intuitional Evaluation: Recommendations on the usability of the system are received from the experts involved. With this method, a great deal of sound identifications in determining the problems can be performed. Yet, in order to be able to determine some of the problems, real users must take part in the usability tests.

Ø Collaborative Usability Control: An evaluation is performed with a group consisting of developers, users or field experts over a thoroughly arranged system. The advantage of this method is that there will be individuals to analyze the system from a wider perspective. The other advantage, on the other hand, is that the system can be made more applicable thanks to the extensive knowledge of the developers participating in the evaluation.

Nielsen and Loranger (2006) make these suggestions on the errors found in the wake of the usability evaluation:

Ø All the errors proved to be at high levels must be corrected. Ø Much effort must be exerted in order to be able to correct the errors at

average level. Ø The errors at low levels must be corrected at a proper time if they are not

insignificant to put an effort into.

There are three factors determining the difficulty of the problem (Nielsen and Loranger, 2006):

Ø Frequency: The minority of users encountering the problem in question

minimize the difficulty rate of the problem. Ø Impact: The number of problems experienced within the system by the user is

also one of the factors affecting usability in a negative way. Ø Continuity: The recurrence rate of the problem is also important. The users do

not confront many of the problems over and over, because they have already learnt how to cope with problems once they have solved them. However, the difficulty rate of the problem will increase under frequently experienced circumstances.

Usability, as the target of quality, has also been defined with ISO 9241-11 standard. Accordingly; usability is ‘’ the level of implementing the objectives determined by the use of a product by the selected users in an efficient, effective and satisfactory way within the criteria determined’’ (Bevan, 1995). The experts create the usability level by evaluating together the concepts of effectiveness, efficiency and satisfaction found within the definition of usability. This relationship is one of the important factors that affect the design process (Cagiltay, 2011). While preparing an educational environment, the focus should primarily be on the learners and their tasks. The experts focus on the interaction between the users and

technology by, in fact, pursuing an effective, efficient and satisfactory route (Dillon and Zhu,1997). RESEARCH OBJECTIVES The main objective of this research is to analyze the usability of the E-learning environment prepared in accordance with the 5E-model for the course on Scientific Research Methods. Within this context, answers were sought to the questions presented below:

1. How effective is the E-learning environment for the users? 2. How efficient is the E-learning environment for the users? 3. What is the degree of satisfaction the users get from the e-learning

environment? IMPORTANCE OF THE RESEARCH Nielsen (1993) points out that there must be at least 5 participants for the researches in which usability analyses will be performed. From his point of view, 5 participants are sufficient to reveal 75% of the errors to occur. In the studies conducted with more than 5 participants, the number of determining the errors does not increase at higher levels. Tullis and Albert (2008), however, contrary to Nielsen, stated that 5 participants could reveal only 30-35% of the errors, and that the study must be conducted with at least 12 participants in order to obtain sound and accurate results (Cited by Cagiltay, 2011). The research was carried out along with 42 users in order for the results of the usability analysis to prove to be more reliable. One of the most significant characteristics that distinguishes this research from others is that it is adapted to the 5E model in the web environment and is supported by multimedia elements; the second important point is that the usability analysis is performed in detail with a great number of users. METHOD The mixed method was used for the purpose of measuring the usability of e-learning environment developed for the course referred to as SRM. In the mixed method, the qualitative and quantitative data collection tools are used simultaneously and consecutively (Balci, 2013). Participants In this study, 42 undergraduate students who had taken this course were selected as participants in order to analyze the usability of the e-learning environment of the SRM course. When the involved researches are examined, the number of participants in this research is more than the general one. However, it was considered that the great number of participants would increase the reliability of the results of usability analysis. Instruments In order to measure the usability of the website, the Web Site Usability Scale developed by Kilic-Cakmak et al (2011) was used, whereas a Web-based Educational Environmental Attitude Scale developed by Erdogan et al (2007) was used for measuring the attitudes of the users towards the environment. Additionally, the usability test developed by the researchers of this study was also performed. Data Collecting Process In the data collection process that lasted for three weeks in total, the users gathered in the school laboratory upon arranging an appointment with the experts who were supposed to collect the data. The data were collected by giving the priority to the

usability test. The preparation and implementation period of the usability test was organized according to the elements required to be determined before the test mentioned by Cagiltay (2011). The Expert Training and Implementation Process of the Usability Test 5 experts were educated and trained for the research in which 42 participants were involved. Based on the criteria of the specified success evaluation of the usability test, the training was provided in accordance with how the applications would be performed. Along with 5 experts trained by the researcher who prepared the study, the groups consisting of 2 individuals each were formed, and the users were subjected to the tests. While one of the experts within the groups read out the tasks included in the test and managed the process, another expert took notes as to the time spent during the implementation of the tasks and the number of clicks performed. In this study conducted in the city of Aydın, a computer laboratory existing within the structure of the university was used as the place of research. Such a path had to be chosen due to the fact that there were no full-equipped laboratories in the city where usability studies could be conducted. The users completed the process by conferring to one of the three expert groups determined through the appointment procedure. The Success Evaluation Criteria of the Usability Test The data obtained in the process of the usability test were analyzed in the light of the criteria presented below. These criteria were determined by the researchers by taking into consideration the characteristics and the activities of the learning environment.

Ø Task 1: Log into the website involved. Ø Task 2: Sign in to the course called Scientific Research Methods. Ø Task 3: Proceed until the end page of the course content of the 1st week. Ø Task 4: Click on the 2nd Week Forum link via the site map. Ø Task 5: Share a message on the 2nd week forum. Ø Task 6: Return to the course called Scientific Research Methods. Ø Task 7: Download the Sample Article PDF file contained in the 3rd week. Ø Task 8: Download a file (Office,PDF,etc.) into the system by logging into the

assignment page contained in the 4th week. Ø Task 9: Complete the Matching Activity of the First Animation contained in the

5th week Course Content. Ø Task 10: Complete the Test by randomly answering the evaluation questions

contained in the 5th week. Ø Task 11: Return to the Main Page. Ø Task 12: Exit website.

FINDINGS AND DISCUSSION Web Site Usability Scale The scores given for each item of the Website Usability Scale with the 5 point likert type were evaluated over 5 points. The average scores were evaluated as: 1-1,79 very weak; 1,80-2,59 weak; 2,60-3,39 medium; 3,40-4,19 good; and 4,20-5 very good. In this context, the perceptions of 42 students on the usability of web-based learning (WBL) environment were analyzed.

Table: 1 Website Usability Scale Analyses

N X SS

Surfing Ease/Convenience 42 4,28 ,42 Design 42 4,34 ,50 Accessibility 42 3,98 ,59 Ease of Use 42 4,16 ,72 Applicability 42 4,23 ,41

Considering the average scores of the data collected from the participants, the scale with 4 different dimensions as the surfing ease, design, accessibility, and ease of use has put forward that the e-learning environment is applicable at a pretty good level. When the dimensions were examined separately, it was seen that the surfing convenience, design and ease of use were at quite good levels along with the pretty good level of the dimension of accessibility. Usability Test After the usability test was performed on 42 participants, the durations (seconds) of performance and the rate of clicks for each task according to the success status were given in the following table. The status of performing a task is divided into 3 dimensions as successful, partially successful and unsuccessful. How many individuals performed each task under what circumstances is shown. The success rates of 42 users which are performed for 12 tasks are given in figure 1 as the number of users and as percentage.

Figure: 1

Success Status in the Usability test Considering figure 1, the rate of usability test to be fulfilled with success is 93%. As is seen from the graphic above, the number of individuals successful in all the tasks proved to be high. The 2nd, 3rd and 11th tasks were successfully completed by all the users. On the other hand, the task in which success was at the very least level proved to be the 8th task in which documents were requested to be downloaded to the page. The rate of users partially succeeding in 12 tasks contained in the user’s test was 4%. Since the 2nd, 3rd and 11th tasks were successfully completed by all the users, there was no user who was found to be partially successful in these tasks.

Besides, there is no user who achieved a partial success in the 5th task, either; however, as will be seen in the following graphic, 2 of the participants failed in this task. The 8th task was the one in which success was at the very least level. As is seen in this graphic, it is the task with the highest rate of partial success of the participants. 9 participants in total were partially successful in the 8th task. The rate of users who failed in the user’s test was 3%. There was no user unsuccessful in the 1st, 2nd, 3rd, 10th and 11th tasks among those 12 tasks. The average number of users unsuccessful per task was 2 . The task in which there was much failure was the 8th task with 4 participants. In the 1st task, the users were requested to log into the website. While 40 people succeeded, 2 of them (K19 and K28) partially made it. The problem experienced by these users is related to the user password. They logged into the account by being challenged by the special character (star) contained within the password assigned to them by the system administrator. Yet, since they were able to get Access into the webpage and came to the final stage of the process of logging into the account, they were considered as partially successful. In the 2nd task, they were requested to log into the course called Scientific Research Methods (SRM). All the users completed this task successfully. In the 3rd task, they were requested to proceed until the end page of the course content of 1st week. All the users completed this task successfully. In the 4th task, the users were requested to get access to the froum link contained within the 2nd week course activities via the site map. In this task, 2 users were partially successful(K26 and K42), whereas 2 users (K9 and K10) failed. Those who were partially successful clicked on the forum links on the site map which were different from what was requested. They were regarded as partially successful since they were able to discover the site map. Those who were unsuccessful failed to discover the site map. In the 5th task, the users were requested to share a message on the 5th week forum. However, 2 users (K15 and K28) were unsuccessful as they failed to find out the involved link through which they would share their message on the forum. In the 6th task, they were requested to return to the SRM course page. In this task, 2 users (K19 and K31) were partially successful, while 2 users (K24 and K28) were unsuccessful. Those considered to be partially successful reached the previous pages by returning over the browser. They were considered as partially successful since they did not click on the page links but were able to access into the course. The unsuccessful ones failed to return to the course page. In the 7th task, the users were requested to download a sample article pdf file contained within the 3rd week course activities. In this task, 3 users (K18, K24 and K31) were partially successful, whereas 1 user was (K28) unsuccessful. The partially successful ones were able to reach the page of the article but failed to find out how to download the file. The user who failed could not get access to the page where the sample article was presented. In the 8th task, the users were requested to download any document on the assignment link contained in the 4th week course activities. In this task, 7 of the users (K7, K12, K14, K19, K20, K41 and K42) were partially successful, whereas 4 of them (K2, K6, K15 and K23) were regarded as unsuccessful.

Those partially successful were able to reach the file download screen via the assignment link but could not find out what link to click on in order to download a file. Since they were able to reach the file download screen, they were considered partially successful. On the other hand, those who were unsuccessful failed to reach the file download/upload screen. In the 9th task, the users were requested to complete the matching activity of the first animation contained in the 5th week course content. In this task, 2 users (K14 and K37) were partially successful, while 2 users (K1 and K28) were unsuccessful. The partially successful ones got access to the animation, performed the matchings but were not considered to be successful as the matchings were incorrect and the animation was left incomplete. The unsuccessful ones, however, were unable to reach the animation. In the 10th task, the users were requested to randomly answer the evaluation questions contained within the 5th week course activities. In this task, 3 of the users (K14, K15 and K37) became partially successful. The partially successful users, after having answered the questions, completed the test but proved to be partially successful since they failed to click on the confirm button. In the 11th task, the users were requested to return to the main page. All the users completed this task successfully. In the 12th task, the users were requested to exit the website. 1 user (K25) was regarded as unsuccessful since s/he could not exit this website properly. Determining Criteria The way of determining criteria was preferred in order to be able to evaluate, within the scope of the efficiency of the webpage, the time spent and the number of clicks performed by the users involved in the study while performing their tasks. For this purpose, 5 different users with pretty good technological proficiency were selected and were asked to perform the same tasks. In this way, the lower thresholds in performing the tasks were specified. The average period of time spent and the number of clicks performed by these 5 users for each task is as follows:

Table: 2 The Usability Test Analyses of the Criteria Group

Considering the subject in terms of the evaluation of general efficiency of the website, the average periods of time spent for each task by the group of 42 users reveal themselves in the graphic shown in Figure: 2.

G01

G02

G03

G04

G05

G06

G07

G08

G09

G10

Duration (Sec)

17 1 17 5 12 5 12 24 29 18

Clicks 5 1 6 2 4 2 2 11 11 15

Figure: 2

The evaluation of efficiency on the basis of duration. Considering the duration spent by the group determined as the efficiency criterion and by the users in the study, the users were generally seen to have spent much more time while performing their tasks. Considering the Criteria group and the proximate duration values, the 2nd, 3rd, 4th, 6th, 11th and 12th tasks indicated a closer percentage in terms of duration. Considering the subject in terms of the evaluation of general efficiency of the website, the average periods of time spent for each task by the group of 42 users reveal themselves in the graphic shown in Figure: 3.

Figure: 3

The evaluation of efficiency on the basis of clicks. Considering the rates of the number of clicks as the efficiency criterion, no critical difference was seen between the criterion group and the real user group on a digital basis. The tasks in which a lower rate of difference can be seen are the 3rd, 4th, 6th, 7th, 8th, 9th and 11th tasks.

Web-based Educational Environmental Attitude Scale (WBE-AS) The scores given for each item of WBE-AS with a 5 likert scale-type were evaluated over 5 points. The average scores were evaluated as : 1-1,79 very weak; 1,80-2,59 weak; 2,60- 3,39 medium; 3,40-4,19 good and 4,20-5 very good, respectively. In this context, the attitudes of 42 students towards e-learning were examined.

Table: 3 Attitude Scale Analyses

WBE-AS Efficacy 42 3,62 ,64 WBE-AS Resistance 42 2,77 ,66 Total 42 3,50 ,61

CONCLUSIONS While the usability of a website is being evaluated, there are 3 dimensions to take into consideration: effectivenes, efficiency and satisfaction (Cagiltay, 2011). In this research, the usability scale and usability test were utilized for the effectiveness dimension, whereas for efficiency, the time spent and the number of clicks performed were taken into account in the usability test; on the other hand, the attitude scale was used for the dimension of satisfaction. The usability test performed on the users comprise 12 tasks. While the findings were being evaluated the success rates of all the users in each of the tasks were taken into consideration. The success rates in these tasks vary between 74% and 100%. The task with the lowest success rate of 74% is the process of downloading/uploading a document into the system via the assignment page. In this task, 7 out of 42 students were able to reach the document download/upload page but could not perform the process, whereas 4 of the users could not even reach the assignment download/upload page. Following this task comes the 4 tasks with the success rates of 90%. The tasks were to get access to the involved forum page via the site map, return to the main page, download the document contained within the course content and to perform the animation activity contained in the course involved. However, 4 users each failed to achieve full success for each of these tasks. The succeeding success rate with 93% is the task of answering the evaluation questions randomly. 3 users, after having answered the questions, skipped clicking on the submit button, therefore, they failed to complete this task. Then comes the task of logging into account on the website and sharing messages on the forum with the success rate of 95%. In these tasks, 2 persons each were unable to achieve full success. The task of exiting the website, with the success rate of 98%, could not be performed accurately by 1 of the users. There are 3 tasks with a 100% success rate, which are as follows: signing in to the course involved, proceeding until the end page of the involved course content and returning to the main page. Considering the status of being successful in all the tasks in the usability test, this rate is 93%, which is the significant indicator that the site is quite applicable. Considering the findings of website usability scale, it was found that the users considered the site to be applicable at a pretty good level. In this scale with 4 dimensions, such as surfing ease, design, accessibility and ease of use, it was seen that accessibility was at a good dimension, while the other 3 dimensions were applicable at a pretty good level. In the light of the three basic concepts significant in evaluating usability, the results of the research were interpreted as follows:

Ø Effectiveness: The success rate of 93% obtained from the usability test and

the pretty well-applicable results obtained from the Website Usability Scale seemed to support and comply with each other. These results suggested that e-learning environment was quite effective.

Ø Efficiency: When the average periods of time spent and the average number of clicks performed were compared with the specified criterion, it was seen that the period of time spent by the group of 42 users was rather long while performing their tasks, however, the average number of clicks were close to the criterion. For this reason, the e-learning environment can be said to be efficient.

Ø Satisfaction: In the wake of utilizing the e-learning environment of the SRM course, the attitudes of the users towards e-learning environment were examined, and they were observed to have showed a good level of attitude in the process, which, eventually, suggests that the users are satisfied with the use of the involved e-learning environment.

The task of uploading a document to the system via the assignment page with the success rate of 74%, which was the most challenging process for the users, and the tasks of downloading documents and signing into accounts with the success rates of 90% and 95%, respectively, are the tasks associated with the users’ proficiency in the general information technologies. The users often had difficulty in uploading and downloading files to the system, which is not the general characteristic of the learning environment; furthermore, since they were unable to find out how to enter the special character contained within their user passwords via the keyboard, they ended up failing to log in to their accounts. Getting access into the forum where the users showed 90% success, returning to the course page, answering randomly the evaluation questions in which they showed 93% success, sharing messages on the forum where they had 95% success and exiting the environment where they showed 98% success are, on the other hand, the tasks associated with the interface of Moodle. The users unable to reach the forum failed to discover the site map; the users who failed to perform the task of returning to the course page could not find out the link related to the users; those unable to complete the evaluation questions ignored the submit button that appeared after the questions had been answered; those who were unable to share messages on the forum failed to discover the reply button; and one of the users who failed to exit the environment could not find out the exit link involved. Considering the success rates of the usability test, the users who were unsuccessful generally had difficulty in exploring the elements concerned with the task in the Moodle interface. Considering the criteria of effectiveness, efficiency and satisfaction separately, the high success levels and the results of the usability scale suggest that the site was quite effective; whereas, considering the results in terms of the efficiency dimension, it is observed that there is no significant difference between the criterion group and the real group on the basis of clicking. However, the fact that there was an increase on the basis of duration suggests that this increase was efficient due to the fact that the users, while performing their tasks, used the environment by successfully exploring it or thinking over it. On the other hand, it is clear that the attitudes towards the e-learning environment, when evaluated on the basis of satisfaction, showed that the users were quite satisfied with such an environment. In the light of these results, the e-learning environment proved to be quite applicable on a considerably good level.

RECOMMENDATIONS Today, the fact that there are a great number of web users and that most of them are in control of the general features of websites minimize the rate of performing errors. The users usually make errors when they confront a new website at most (Nielsen and Loranger, 2006). For this reason, the frequency of errors made by the users using, for the first time, the e-learning environment designed on the web must be monitored and taken into consideration. The recurrent errors will be those that the real users using the environment for the first time will possibly encounter. Hence, it is required that several improvements be made to minimize such errors on websites. The fact that the users can correct the errors they confront rarely in the succeeding tasks on their own suggests that the environment has a consistent structure Nielsen (1993) states that the consistency of the environment will boost the productivity of the user since it will enable the user to predict what the system will do in a similar situation once a few errors have been encountered beforehand. As in a number of respects in usability, surfing over the website is of great importance. Yet, if a problem occurs, the user may be challenged. In such cases, the users experiencing problems regarding surfing on the web mostly give up completing the task (Spool et al, 1999). The e-learning environments devised for the Web should allow for a convenient surfing process. The system users, administrators and the experts of information systems must collaborate in order to create a sound and firm information system (Stair and Reynolds, 2012). Anybody who plays a role in any stage of the environments developed for E- learning should work in co-operation in the design process and contribute to the process in order to make the environment highly applicable. For this, however, it is necessary that the evaluation stages of usability be analyzed well and the design process be performed in a healthy way. In this study where the usability analysis of the e-learning environment of the SRM course designed according to the 5E model is performed, only a user-based usability study was conducted. In similar studies, the usability function may prove to be higher when the design is made with the criteria determined by the designer, when it is subjected to an expert test afterwards, and when, eventually, a user test is performed on the system taking its final shape. As is also mentioned in the results section, the success rates are quite high; yet, the users who proved to be unsuccessful generally had problems during their exploration in the Moodle interface. Considering the situation within the context of Moodle, the developers can make arrangements in the placement of the elements included in the interface in the way that they will enable an ease of use to the users. For this purpose, various usability analysis methods can be used; both the experts and the real users can be asked to assist in the process of evaluation. The most important aspect to be considered as the main source of failure is the users’ insufficient levels of utilizing the information technologies. Users generally have difficulty in uploading or downloading the most commonly-used file programs to the environment. Considering the conditions of the age we are living in, on the other hand, it is seen that such aspects are indeed significant problems that urgently require several improvements. The fact that the undergraduate students who took part in the research as participants failed to perform such sort of applications even though they were digital natives in that

field is the kind of situation not desired at all under today’s circumstances. Therefore, no matter how directly they may be participating in the technology, there is the need for control mechanisms to guide them in the right direction. To be widespread, this can best be achieved under the guidance of the courses on Information Technologies being provided in the schools within the structure of The Ministry of National Education. The Information technology courses in schools are of great importance in terms of informing the students of all ages about the use of such technologies and guiding them in the right direction. For this reason, it can be beneficial to provide teachers involved in Information Technologies who will be available at all times and manage the process in all the schools and to provide the involved course as a compulsory element of the curriculum in all the levels of education. In this way, individuals with good levels of proficiency in information technologies can be raised and trained. Authors’ Note: This paper was taken from the first author’s master thesis named as The Usability Of E-Learning Environment Designed with 5E Model and presented in 8th International Computer and Instructional Technologies Symposium with the name of this article. BIODATA and CONTACT ADRESS of the AUTHORS

Fulya TORUN has a M.A. degree in IT Systems, Gazi University, Turkey. And now she is a PhD Student in CEIT, Hacettepe University, Turkey, 2014. She was born in Ankara, Turkey, 1989. She copmleted her secondary education in a vocational high school (department of computer) in Ankara, 2006 and graduated from department of CEIT in Anadolu University, Eskişehir, Turkey, 2011. She has been working as a Research Assistant in Adnan Menderes University, department of CEIT since 2013.

Fulya TORUN Faculty of Education, Adnan Menderes Univesity, Aydın, Turkey. Phone: +90(256) 2142023 - 1597 Mobile: +90(505) 3506389 Email: fulya.torun@adu.edu.tr

Dr. Hakan TEKEDERE has a PhD degree in Educational Technologies, Gazi University, Turkey, 2009. He was born in Malatya, Turkey, 1975. He graduated from department of Electronical Computer Education in Fırat University, Elazığ, Turkey, 1997 and completed has a M.A. degree in Electrical and Electronics Engineering from Niğde University, Niğde, Turkey, 2001. He has been working as an Assistant Professor Doctor in Gazi University since 2009.

Dr. Hakan TEKEDERE Health Services Vocational School, Gazi University, Ankara, Turkey. Phone: +90(312) 4845635 - 125 Mobile: +90(533) 4648606 Email: tekedere@gazi.edu.tr

REFERENCES Badre, A. N. (2002). Shaping Web Usability: Interaction Design in Context. Boston: Pearson Education, 28-39. Balci, A. (2013). Sosyal Bilimlerde Araştırma Yöntem Teknik ve İlkeler. Ankara: Pegem Akademi, 42-43. Bevan, N. (1995). Human-Computer Interaction Standards. Paper presented at Sixth International Conference on Human Computer Interaction, Yokohama. Brinck, T., Gergle, D. & Wood, S. D. (2002). Usability For The Web: Designing Web Sites That Work. San Diego: Academic Press, 15. Bybee, R. W., Taylor, J. A., Gardner, A, Scotter, P. V., Powell, J. C., Westbrook, A. & Landes, N. (2006). The BSCS 5E Instructional Model: Origins and Effectiveness. Colorado: BSCS, 1-4. Clark, R. C. & Mayer, R. E. (2011). e-learning and the Science of Instruction: Proven Guidelines for Consumers and Designers of Multimedia Learning (Third Edition). San Francisco: Pfeiffer, 18-25. Cagiltay, K. (2011). İnsan Bilgisayar Etkileşimi ve Kullanılabilirlik Mühendisliği: Teoriden Pratiğe [Human Computer Interaction and Usability Engineering: from Theory to Practice]. Ankara: ODTÜ Yayıncılık, 94-115. Cakir, H., Uluyol, C. & Karadeniz, S. (2008). Öğretim Stratejileri. İnternet Temelli Eğitim [Internet Based Education]. Yalın, H. İ. (Editor). Ankara: Nobel Yayın Dağıtım, 77-78. Dillon, A. & Zhu, E. (1997). Designing Web-Based Instruction: A Human-Computer Interaction Perspective. Web Based Instruction. Kahn, B., H. (Editor). New Jersey: Educational Technology Publications, 223. Erdogan, Y. Bayram, S. & Deniz, L. (2007). Web Tabanlı Öğretim Tutum Ölçeği: Açıklayıcı ve Doğrulayıcı Faktör Analizi Çalışması [Web Based Education Attitude Scale: Study of Exploratory and Confirmatory Factor Analysis]. Uluslararası İnsan Bilimleri Dergisi, 4(2). Ferre, X., Juristo, N., Windl, H. & Constantine, L. (2001). Usability Basics for Software Developers. IEEE Software, (January, February 2001), 22-29. Garrison, D. R. (2011).e-Learning In The 21st Century: A Framework For Research and Practice (Second Edition). New York: Routledge, 1-2. Jonassen, D. H., Howland, J., Moore, J. & Marra, R. M. (2003). Learning to Solve Problems with Technology (Second edition). New Jersey: Merrill / Prentice Hall, 11. Kilic-Cakmak, E., Gunes, E., Ciftci, S. & Ustundag, M. T. (2011). Web Sitesi Kullanılabilirlik Ölçeğinin Geliştirilmesi: Geçerlik, Güvenirlik Analizi Uygulama Sonuçları [Developing Web Site Usability Scale: Validity, Reliability Analysis of Implementation Results]. Pegem Eğitim ve Öğretim Dergisi, 1(2). Nielsen, J. (1993). Usability Engineering. San Diego: Academic Press, 26-228. Nielsen, J. & Loranger, H. (2006). Prioritizing Web Usability. Berkeley: New Riders. Senemoglu, N. (2010). Gelişim, Öğrenme ve Öğretim (Onsekizinci Baskı) [Development, Learning and Teaching (18th Edition)]. Ankara: Pegem Akademi, 629-630.

Shackel, B. & Richardson, S. (1991). Human Factors for Informatics Usability. Cambridge: Cambridge University Press, 40-360. Spool J. M., Scanlom, T., Schroeder, W., Snyder, C. & D’Angelo, T. (1999). Web Site Usability: A Designer’s Guide. San Diego: Academic Press, 15. Stair, R. & Reynolds, G. (2012). Principles of Information Systems. Boston: Course Technology, 36-37. Senturk, C. (2010). Yapılandırmacı Yaklaşım ve 5E Öğrenme Döngüsü Modeli [Constructivist Approach and 5E Learning Cycle Model]. Eğitim-Bir-Sen, 6(17), 62. Tufekci-Aslim, S. (2011). Yapılandırmacı Yaklaşım. Öğrenme Öğretme Kuram ve Yaklaşımları [Theory and Approaches of Teaching and Learning]. Büyükalan-Filiz, S. (Editor). Ankara: Pegem Akademi, 346-347. Usun, S. (2006). Uzaktan Eğitim [Distance Education]. Ankara: Nobel Yayın Dağıtım, 113.

Copyright of Turkish Online Journal of Distance Education (TOJDE) is the property of Turkish Online Journal of Distance Education and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.