paper

IOP Conference Series: Materials Science and Engineering

PAPER • OPEN ACCESS

Usability Analysis of an Academic Faculty Website: A Case Study of the Selected University To cite this article: H Soewardi and M H Ramadhan 2020 IOP Conf. Ser.: Mater. Sci. Eng. 722 012020

View the article online for updates and enhancements.

You may also like Categorization of quantum mechanics problems by professors and students Shih-Yin Lin and Chandralekha Singh

-

Editorial: CAMS2016-

V International Conference of Computational Methods in Engineering Science

-

This content was downloaded from IP address 173.64.65.195 on 16/10/2021 at 13:22

Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.

Published under licence by IOP Publishing Ltd

ICET4SD

IOP Conf. Series: Materials Science and Engineering 722 (2020) 012020 IOP Publishing

doi:10.1088/1757-899X/722/1/012020

1

Usability Analysis of an Academic Faculty Website: A Case

Study of the Selected University

H Soewardi, and M H Ramadhan

Industrial Engineering Department, Faculty of Industrial Technology

Islamic University of Indonesia, Yogyakarta-Indonesia

E-mail: [email protected]; [email protected]

Abstract. Website constitutes a pivotal element in an academic institution. It is useful to provide

academic information for stakeholder requirements like final scores, academic calendar,

scholarship, student competition, etc. This paper presents an investigation about the level of

usability of the website at the selected faculty of the university in Indonesia based on 5

dimensions, namely learnability, efficiency, memorability, error, and satisfaction. It is

noteworthy that the faculty should supply the information in detail and accurately. However, as

a matter of fact, few of a faculty website in some certain universities sometimes experiencing

matters for the user. Thus, it is crucial to do the analysis of the usability of the website at the

faculty level. A qualitative study was conducted by distributing some Likert-scale questionnaire

to stakeholders that consisted of students, academic staff, student’s parents, and lecturers. The

questionnaire contains 25 statements divided into five groups of elements by referring five

dimensions above with each group having 5 questions or statements. The non-parametric statistic

was conducted to test the hypothesis. The result of this study shows that the website of the

selected faculty is classified into a moderate level with 68% of usability grade. Thus, this study

recommends that the faculty improve its website in the layout of menu, content, add some

features to ease login and increase usability level.

1. Introduction

Website constitutes a pivotal element in an academic institution. It is useful to provide academic

information for stakeholder requirement like final scores, academic calendar, scholarship, student

competition, etc. Meanwhile, website is also used as a media to promote university educational

qualification and recognition [1] [2]. A good website should meet the user needs such as easy access

and ease of getting information [3]. To find out whether a website is good or not, it is necessary to

measure its usability level.

A preliminary study has been conducted to stakeholder of the website. It reveals that intensity level

in using the selected academic website is 50% of the respondents occasionally accessing the website,

42% of respondents infrequently accessing the website and 7.5% of respondents oftentimes accessing

the website. This finding pops up a question why the stakeholder does not frequently access such

website. Thus, it is important to test the academic faculty website in the selected university about its

usability level.

The objective of this study is to analyze the usability level of the academic website of the selected

university as a case study based on its learnability, efficiency, error, memorability and satisfaction.

ICET4SD

IOP Conf. Series: Materials Science and Engineering 722 (2020) 012020 IOP Publishing

doi:10.1088/1757-899X/722/1/012020

2

2. Research Method

2.1. Usability Analysis Method

Usability is a measure that describes the level of ease of use of some product with learnability, efficiency,

error, memorability and satisfaction as the parameter [4] [5] [6]. Academic websites are said to be usable

if the users feel satisfied after using the website; find them easy to learn the process for finding certain

information in the website; users can find the information they want quickly and with a minimum error;

and the contents of the website meet user’s needs [7][8].

The testing was done by giving respondents a series of tasks that they must do on the website being

tested. This test aims to form opinions from respondents about the particular website tohelp them when

filling out the questionnaire. The test contains some tasks related to menus and features provided by the

website. Once the respondent completed the tasks, they would be given a Likert-scale questionnaire.

The data from the questionnaire would be used to measure the usability level of the website by

calculating its mode value. The mode value is analysed to find the usability level of the website. To

support the results of the analysis, interviews were conducted with respondents to find out their

difficulties when using this website.

2.2. Survey

The survey was conducted by distributing questionnaires to stakeholders consisting of students, lecturer,

student’s parents and faculty’s staff, and by interviewing them about their complaints related to the

particular website. This survey aims to identify the intensity level of access for respondents to the

faculty’s website and to test the usability level of the website. The first survey contains questions about

the intensity of accessing the website under study; what device they mostly used to access the website;

and what information that they wanted to find on the website. The second survey is to test hypothesis

by asking about the use of the website.

2.3. Apparatus

The tools used to support this experiment are:

• Likert-scale questionnaire Likert-Scale is a type of questionnaire that uses a number of statements to measure individual

behaviour by responding to 5 choice points on each item statement. The choice is a strongly

agree, an agree, a quite agree, a disagree, and a strongly disagree [9] [10]. The questionnaire

consists of 25 statements divided into five group of elements, which can be seen on TABLE I.

• Laptop with a stable internet connection was required to do the test.

• Stopwatch was used to calculate loading and downloading time

• Stationary was used to write the results of interviews.

• Microsoft Excel 2013 software was used to process the data.

Table 1. List of the questionnaire statement

Elements Statements Elements

Learnability

The texts on the website are easy

to understand.

Errors

I rarely find errors that occur on

the website.

The information that I am looking

for can be found easily based on

the menu available on the main

page of the website without using

the search feature.

I always find the information I'm

looking for on this website.

The search feature can be found

easily. The search feature works well.

ICET4SD

IOP Conf. Series: Materials Science and Engineering 722 (2020) 012020 IOP Publishing

doi:10.1088/1757-899X/722/1/012020

3

Elements Statements Elements

I don't need technical assistance

when using this website. All links on this website work

well.

The menu functions of this website

are easy to understand.

The website provides procedures

for correcting errors that I have

encountered

Efficiency

The loading time needed to access

this website is quite fast.

Errors

I rarely find errors that occur on

the website.

With the search feature, I can find

the information I want quickly.

I always find the information I'm

looking for on this website.

Without the search feature, I can

find the information I want

quickly.

The search feature works well.

The time needed to download a file

from this website is fast.

All links on this website work

well.

This website can be opened at any

time.

The website provides procedures

for correcting errors that I have

encountered

Memorability

I can memorize how to use this

website easily.

Satisfaction

The appearance of this website is

attractive.

The symbols on this website are

easy to remember I will visit this website often.

The functions of the menus of this

website are easy to remember.

I feel comfortable with the menu

layout of this website.

The address of this website is easy

to remember.

I feel comfortable with the colors

of this website.

I can remember the step that must

be taken to get the information I

want easily.

I am satisfied with the

information I got from this

website.

2.4. Design of Experiment

The usability testing was done in campus by assigning participant to complete the tasks. 30 subjects

participated in this experiment. When conducting the test, the participant was allowed to tell their

opinion of the website and the researcher recorded it. The tests were carried out using a laptop provided

and with no time limit. Layout design of the test room is presented in Fig.1, while the homepage of the

website under study can be seen in Fig. 2

ICET4SD

IOP Conf. Series: Materials Science and Engineering 722 (2020) 012020 IOP Publishing

doi:10.1088/1757-899X/722/1/012020

4

Figure 1. Layout of the usability testing room

Figure 2. Website’s Homepage

2.5. Task

The followings tasks were to complete by the respondents:

• Find the information regarding the profile of this faculty (history, vision and mission, and organization structure).

• Find the information about this faculty study program and the facility.

• Find information about what innovations the faculty is doing.

• Find the information regarding the academic services such as test scores, academic calendar, exam schedule, submission of academic leave and thesis.

• Find information about this faculty partner.

2.6. Procedure of experiment

Participants were explained about the purpose of an experiment and a task done. Subsequently,

participants were instructed to operate the computer and login to open the website and to complete the

tasks. After that, the participants were asked to fill the Likert-scale questionnaire provided via Google

Forms. Then, the participants were interviewed to find out the difficulties that they felt during the test.

2.7. Statistical Analysis

The data obtained using the Likert-scale questionnaire will be processed using Microsoft Excel software

by calculating its mode value. Mode is the number or value that appears most often in a set of data,

which can be used to define the meaning of those data [10] [11]. Thus an discriptive analysis is done.

ICET4SD

IOP Conf. Series: Materials Science and Engineering 722 (2020) 012020 IOP Publishing

doi:10.1088/1757-899X/722/1/012020

5

3. Result and Discussion

3.1. Result of The Survey

Fig.3 shows the result of the intensity level survey. It shows that 7% of respondent frequently access the

website, 50% of respondent occasionally access the website, 43% of respondent infrequently access the

website, and no respondent never access the website. It means that almost no users are attracted to open

this website gaining some information required.

Figure 3. Result of intensity level survey

Table 2 presents the summary of interview, which contains complaints about the website. The first

complaint is about the menu layout of test score portal and online mail because the menu is not on main

menu of the website. Also, it does not appear when we firstly open the website and on the search feature

which confused the respondents. The second complaint is about the fact that not all test scores are

available on the menu. Some courses test score can be clicked and seen, while some cannot. The third

complaint is related to the sixth complaint. Since the academic calendar is located on the announcement

menu, it made it difficult for the users to find it without the search feature because the announcement

menu cannot be clicked easily. For the fourth complaint, the respondent thought that the username and

password for logging in are the same as another core feature of the university. As for the fifth complaint,

there are 3 menus on the main menu that contains a blank page when opened. No information can be

found on those 3 menus.

Table 2. Summary of interview

No Complaint Percentage of

Respondent

1 Menu Layout of Test Score Portal and Online Mail 57% (17)

2 Some test score are not available. 100% (30)

3 It’s difficult to find information such as academic calendar & exam

schedule without using the Search feature. 80% (24)

4 Difficult to login into the Online Mail feature. 73% (22)

5 There are still 3 links containing empty content. 100% (30)

6 Announcement menu cannot be clicked. 100% (30)

7%

50%

43%

0%

Intensity Level

Frequent Occasional Infrequent Never

ICET4SD

IOP Conf. Series: Materials Science and Engineering 722 (2020) 012020 IOP Publishing

doi:10.1088/1757-899X/722/1/012020

6

3.2. Usability Analysis Result

Fig. 4 presents the mode value of usability attributes of this website. They are learnability, efficiency,

memorability, error, and satisfaction. Learnability attribute has mode value of 4, which means the

respondent agree with the statements related. The text and the menu of the website are easy to understand

because they use a word that represent the function of the menus. For efficiency attribute, it has mode

value of 3 or average. The loading and downloading time for the website is fast, but there is some

information that cannot be found quickly without search feature, such as academic calendar.

Figure 4. Mode value of usability attributes

Memorability attribute has mode value of 3, which means that the memorability level of this website is

on average. Some information can be easily found, while some others are not such as exam schedule.

Exam schedule information can be found on announcement menu, but the menu cannot be clicked and

the respondent must find another way to reach the menu. For error attribute, it has mode value of 3 or

average. There are still broken links or empty content on the website. Also, the search feature does not

display all information on the website such as test score. As for satisfaction attribute, it has mode value

of 3. It means that the respondent satisfaction level is still on average. Although the respondent like the

interface of the website, they do not like the menu layout of score portal feature. Also, the respondents

felt that the information on the website was incomplete.

Based on the explanation above, this website is not entirely easy to use and not all information is

easy to find. Thus, the usability of this website is still on moderate level-3.

4. Conclusion

This study concluded that the usability level of this website is still on moderate level-3. Therefore, some

features need improvements such as menu layout, empty content and incomplete information.

Acknowledgment

We wish to acknowledge Faculty of Industrial Technology Islamic University of Indonesia Yogyakarta

Indonesia for supporting this project.

References

[1] Kothainayaki, S., Sivakumaren, K.S., Gopalakrishnan, S. 2012. User Preferences on University

Websites: AStudy. Library philosophy and Practice, 788-795.

[2] Mentes S.A., Turan A.H. 2012. Assessing the usability of university websites: an empirical study

on Namik Kemal University. TOJET. 11: (3)61-69

[3] Nielsen, J. 1999. Designing Web Usability. New Riders.

0

1

2

3

4

5

Learnability Efficiency Memorability Error Satisfaction

Mode Value

ICET4SD

IOP Conf. Series: Materials Science and Engineering 722 (2020) 012020 IOP Publishing

doi:10.1088/1757-899X/722/1/012020

7

[4] Krug, S. 2006. Don't Make Me Think! A Common Sense Approach to Web Usability (Second

Edition). Berkeley: New Riders Publishing

[5] Dumas S.J, Redish. C.J.1999, A Practical guide to Usability Testing, John Wiley & Sons.

[6] Nielsen, J. 1994. Usability Engineering. Elsevier.

[7] Rubin, J. and Chisnell, D. (2008), Handbook of Usability Testing: How to Plan, Design, and

Conduct Effective Tests, 2nd ed. Wiley: Indianapolis.

[8] Pant, A. 2015. Usability evaluation of an academic library website. The Electronic Library. 33:896

- 915

[9] Likert RA. 1932. Technique for the measurement of attitudes. Archives of Psychology, 140 pp: 1-

55

[10] Boone Jr, Harry N, and Deborah A. Boone. 2012. Analyzing Likert Data. Journal of Extension 50

(2)

[11] Gravetter, F. J., & Wallnau, L. B. 2007. Statistics for the behavioral sciences. Australia:

Thomson/Wadsworth.