paper
16 J. ICT Res. Appl., Vol. 14, No. 1, 2020, 16-33
Received May 20th, 2019, Revised May 8th, 2020, Accepted for publication May 12th, 2020 Copyright © 2020 Published by IRCS-ITB, ISSN: 2337-5787, DOI: 10.5614/itbj.ict.res.appl.2020.14.1.2
User Efficiency Model in Usability Engineering for User
Interface Design Refinement of Mobile Application
Gladina Desideria1 & Yoanes Bandung2,*
Information Technology Research Group,
School of Electrical Engineering and Informatics
Institut Teknologi Bandung, Jalan Ganesha No.10, Bandung 40123, Indonesia
*E-mail: bandung@stei.itb.ac.id
Abstract. Designers are often confronted with challenges or difficulties when
designing user interfaces for mobile applications. The information must be clearly
delivered to the user and also have an aesthetic appearance and good
discoverability. One widely used method for conveying information is directing
user attention to a component that is related to his or her task. We propose a
recommender system by considering user efficiency in a user attention model. It
can give suggestions for designers to improve the appearance of the most efficient
component. This recommender system is aimed to help designers in the iteration
process of usability engineering, especially to direct user attention to the most
efficient component. This system analyzes actual user attention and then refines
the user interface based on the energy of each component compared with the
baseline energy. Our proposed model successfully increased the efficiency of a
mobile learning application from 83.65% to 85.58% and improved discoverability
of the most efficient component, thus reducing the task completion time.
Keywords: efficiency; mobile application; usability; user experience; user interface;
visual attention.
1 Introduction
The development of technology affects the learning methods used in education
systems with new concepts such as distance flexible learning (DFL), electronic
learning (eLearning) and mobile learning (mLearning) [1]. MLearning is an
evolution of ITB’s eLearning program (http://kuliah.itb.ac.id), combining the
ability of electronic learning with the benefit of flexibility [2]. The MLearning
application with video conference system was developed by Bandung, et al. [3],
motivated by the problem of eLearning not being sufficiently flexible. User
interface limitations on the mobile web makes students prefer accessing
applications via a computer desktop. Therefore, an Android-based learning
application was developed to ensure that students who are unable to attend classes
can access it anytime and anywhere [3]. However, the development of the
mLearning application was mainly focused on the implementation and
functionality of the application, without considering the user interface design.
User Efficiency Model in Usability Engineering 17
The user interface (UI) plays an important role in applications. It affects user
effectiveness and learning outcomes. It is important for a designer to understand
how to design UIs in a good way [4,5]. The purpose of the UI is to improve
usability and shape the user experience. Usability is a quality attribute that
represents the ease of use of a UI design [6] and how it can be used by a certain
group of users to achieve certain goals effectively and efficiently, and make them
satisfied within a certain context of use [7]. The user experience is the response
of the user when interacting with certain products [8].
In a computer-based learning system, information is processed by the user
through the UI as interaction intermediary [9]. The design process of an UI often
poses challenges and difficulties to designers, as it must clearly deliver the
information and deal with aesthetic goals [10]. The most common and widely
adopted strategy for designers to convey information clearly and make users
achieve their goals effectively is by visually directing the user’s attention along
certain paths through the web page design layout [11-13]. Meanwhile, to find the
final design, the designer must design the layout iteratively, exploratively and
correctively [14] to match user requirements and expectations.
Previous research has tried to solve the visual attention problem. One of them is
the research by Pang, et al. in [13]. They proposed a user attention model based
on temporal user attention behavior, applied in an automatic system. It is aimed
at helping designers to create a visual path along the components of a web page
by directing the user’s attention to achieve his or her goal effectively. This
approach improves user attention by automatically managing the scale, position,
distance, and color of the content on dynamic web pages. Thus, it effectively
directs the user’s attention to the information that is relevant for achieving his or
her goal.
Web and mobile design have similar characteristics. They both consist of a group
of different elements that are semantically related to each other (e.g. text, images,
and buttons) in 2D space [13]. This study tried to implement this model on a
different platform, i.e. a mobile application, for which there are specific
requirements. Mobile application design focuses on efficiency and
discoverability [15]. Efficiency is a decisive aspect of user performance related
to effectiveness [7]. Since there are limitations related to the screen size of mobile
devices, it is important to deliver the information precisely and maintain user
efficiency when they interact with the system [15]. Moreover, it is important to
satisfy the usability goal of discoverability by supporting the ease of use in
exploring the content, features and functions of the application [16,17].
Hence, this study proposes an attention model for mobile application design
based on the user attention model from Pang, et al. in [13], which can
18 Gladina Desideria & Yoanes Bandung
automatically analyze user attention and give recommendations for design
improvement based on the efficiency of the layout design. The proposed model
aims to help designers to direct the user’s attention so he or she can reach their
goal in the most efficient way and automatically analyze the iteration process in
usability engineering by considering effectiveness and efficiency. By directing
the user’s attention, it is hoped that the mLearning application can be improved
in such a way that it results in a better learning experience and improved student
performance (effectiveness and efficiency) toward the UI design.
2 Related Works
2.1 Design Optimization and Automation
In [18], Jamil developed the Interior Point Algorithm and improved a UI layout
adaptively by using quadratic programming for linear object functions. However,
UI design is not just a collection of linear components or content that is read in
sequence from top to bottom. UI design can use several reading patterns,
depending on how the designer wants their design to be read [10]. The research
about design automation by O’Donovan, et al. in [10] proposed an energy-based
model. The model optimizes the scale and position of each component but does
not consider other aspects, such as color, font type and text line breaks. It was
implemented on single-page graphic designs such as posters, advertisements,
leaflets, and others [10].
In contrast, web-based and mobile-based applications have a dynamic graphic
design covering multiple pages. Thus, Pang, et al. [13] built their model to enable
web designers to manage user attention through the visual flow of the interface
design. The user attention model was built by analyzing eye-gaze fixation and
transformed it into a transition and order path. The model was used to build a
system that can automatically adjust the ratio, scale, distance, position,
orientation, intensity, and color contrast of the components of web page designs
[13].
The research by Pang, et al. [13] is the most relevant model for our research, a
study on mobile applications. However, the user attention model still has some
room for improvement. Firstly, the adjustment of the design depends on what the
designer wants or needs. Secondly, the model directs the user’s attention in order
to make users accomplish their task effectively, but the efficiency of the usability
factor is not considered.
User Efficiency Model in Usability Engineering 19
2.2 User Interface Impact on Users
A user interface (UI) is the visual display or appearance of an application,
website, or other technologies [19]. A UI is a medium for users, the access point
to interact with the design. The UI design process aims to result in a good
interaction for users in order to improve the usability of an user interface [20].
Usability represents how easily users can access an interface design. Usability is
a quality feature that measures the usage convenience of a UI and refers to
methods for improving the ease of use of a system through the design process [6].
ISO 9241-11 [7] explains that usability is how a product can be used by a certain
group of users to achieve certain goals effectively, efficiently and gain
satisfaction within a certain context of use.
The usability of a technology is important because it deals with user performance,
effectiveness, and efficiency [7]. Effectiveness refers to accuracy and
completeness in achieving certain goals. Efficiency refers to resources spent by
the user to increase the accuracy and completeness in achieving their goals [7].
Moreover, users always have their own insights, feelings and responses toward
the product; this is called the user experience (UX) [8,21].
2.3 Visual Attention
Visual attention is what users perceive in a design or the specific area on which
they focus their attention. Understanding how users perceive and how users pay
attention to a visual design is an important factor that can help designers make a
design more effective [13]. As mentioned in prior studies, visual attention follows
certain reading patterns that are widely used, e.g. the Z-pattern, the F-pattern,
bottom-up and top-down [22,23]. However, the user’s visual attention is not only
determined by these patterns. Designers can manipulate the user’s attention by
arranging the interface’s visualization appropriate to the goal or task the user
wants to accomplish [13].
Creating an attention path to guide users toward the achievement of their goal is
done by many web designers, since it delivers the information efficiently and
helps users accomplish their goals more easily and quickly [11,24].
3 Methodology
Since this study concerns usability, two methodologies that have been developed
to create UIs (usability engineering by Jakob Nielsen [25] and the user attention
model by Pang, et. al [13]) were adopted in the iteration process. The user
attention model was modified to help the designer refine the UI design and help
users achieve their goals efficiently. This paper proposes a recommender system
by adapting and modifying the model of Pang, et al. as shown in Figure 1.
20 Gladina Desideria & Yoanes Bandung
Figure 1 Proposed method.
3.1 Pre-Design Stage
The first step was identifying the user requirements of the mLearning application
and transforming them into a flow diagram to know the interaction of the features
over the different pages. After that, the designers defined the usability and the
user experience goals that must be fulfilled by the design. In this study, the
usability goal was based on user performance: the effectiveness and efficiency of
the UI design and the UX. The usability goal was discoverability, since this factor
is affected by user attention.
User Efficiency Model in Usability Engineering 21
3.2 Design Stage
In this stage, the designers made a high-fidelity prototype that already had a well-
balanced layout of the user interface, called the first version of the design. Then,
user testing was performed to test the first design version among 20 participants.
During the test, their faces and screen activities were recorded.
User testing was held under task-driven condition, as shown in Table 1. Each
participant was asked to finish tasks without receiving any clues about task
completion and with no time limitation. After finishing several tasks, they were
interviewed to collect user attention data within the design to validate the record.
Table 1 User testing tasks.
Scenario Related Page
Task 1 User should enroll certain subject Home
Task 2 User should find certain material in certain subject Subject Menu
3.2.1 User Efficiency Model
This research proposes a user efficiency model based on Pang, et al. in [13] to
develop a recommender system for design optimization to direct user attention to
the most efficient component. The attention model of Pang, et al. consists of two
related constituents of attention motion among the components, i.e. transition and
order. Both have their own modeling. Our proposed model creates a transition-
and-order model based on the efficiency factor. Since efficiency is determined by
time [7], the proposed model modifies both constituents by calculating the
number of steps there are between a component and achieving the goal. The
transition model is expressed as 𝑝𝑡 (𝑐𝑖 (𝑙𝑖 ) → 𝑐𝑗 (𝑙𝑗 )), where 𝑙𝑖 − 𝑙𝑗 is the number
of users that shift from 𝑐𝑖 to 𝑐𝑗. The order model is expressed as 𝑝𝑜 (𝑐𝑖 (𝑙𝑖 ) <
𝑐𝑗 (𝑙𝑗 )), where 𝑙𝑖 < 𝑙𝑗 is the number of users that see component 𝑐𝑖 right before
𝑐𝑗.
3.2.2 Component Segmentation
The next step was segmenting the components of each one-page design. Design
elements that are classified as components are icons, buttons, text related to tasks,
and hyperlinked images. Each component of the design page was separated by
the designers into a single component by making a virtual boundary
segmentation.
3.2.3 Component Efficiency Hierarchy
Our proposed model modifies the arrangement of hierarchy in [13] by choosing
the most efficient component as the highest level of the hierarchy because this
22 Gladina Desideria & Yoanes Bandung
research aimed to make a model that can guide users to the most efficient
component. This means that the most efficient component has the least number
of steps (l) to task completion.
More steps are less efficient or less related or there is no step that cannot be used
to achieve the goal. Thus, this paper modified the ordered sequence (𝑆) in [13] to 𝑆𝐸 , where 𝑆𝐸 is the expected efficiency sequence. Each page design in this research has a specified path in an ordered sequence of hierarchical component
efficiency, expresssed as 𝑆𝐸 = {𝑐1, 𝑐2, … , 𝑐𝑘 }, where 𝑐1 is the most efficient component and 𝑐𝑘 is the least efficient component.
3.2.4 Transition Matrix
The user attention data from the first user test were used to make a transition
matrix as in [13]. This matrix is used to see the number of users who transition
from one component to another component in a page design [13]. The transition
matrix is expressed as 𝑇(𝑐𝑖 , 𝑐𝑗 ) [13], where 𝑇 is the percentage of users that shift
their attention from component 𝑐𝑖 to component 𝑐𝑗.
3.2.5 Order Matrix
An order matrix was used to study the order of the temporal sequence of user
attention toward the design during the first user test [13]. The order matrix is
expressed as 𝑂(𝑐𝑖 , 𝑐𝑗 ) [13], where 𝑂 is the percentage of users that see component
𝑐𝑖 before component 𝑐𝑗. The order matrix was normalized as Eq. (1) [13]:
Ȏ = ∑ 𝑂(𝑐𝑖,𝑐𝑗)
∑ 𝑂(𝑐𝑖,𝑐𝑗) 𝑘 𝑖
𝑘 𝑖=1 (1)
Then the normalized matrix is added up across the rows of each component as
the order probability of actual user attention, expressed as in Eq. (2) [13]:
ℎ(𝑐𝑖 ) = ∑ �̂�(𝑐𝑖 , 𝑐𝑘 ) 𝑘 𝑖=1 (2)
Getting the actual user attention 𝑆𝐴 from the calculation of ℎ(𝑐𝑖 ) is done by sorting them from the largest to the smallest.
3.2.6 Design Optimization
Design optimization was based on how much energy was needed by each
component compared with the baseline of the total page design energy. The
energy baseline was calculated from the condition where the hierarchy matches
100% with hierarchy 𝑆𝐸 . This means that the user attention of the tested users conformed to 𝑆𝐸 . Furthermore, the actual energy of the page design that we want to refine is also calculated.
User Efficiency Model in Usability Engineering 23
This energy term was developed based on Pang, et. al [13] as attention term 𝐸𝑎𝑡𝑡. 𝐸𝑎𝑡𝑡 is used to direct actual user attention to conform to the sequence hierarchy 𝑆𝐸 . The attention term is formed from the order term 𝐸𝑎𝑡𝑡
𝑂 and the transition term
𝐸𝑎𝑡𝑡 𝑇 with Eq. (3) [13]:
𝐸𝑎𝑡𝑡 = 𝐸𝑎𝑡𝑡 𝑂 + 𝐸𝑎𝑡𝑡
𝑇 (3)
The order term 𝐸𝑎𝑡𝑡 𝑂 is used to ensure that 𝑐𝑖 in 𝑆𝐸 is noticed before its successor
𝑆𝑖. This term is defined as in Eq. (4) as follows [13]:
𝐸𝑎𝑡𝑡 𝑂 = −
1
|𝑆𝐸|−1 ∑
1
|𝑆𝑖| ∑ [ 𝑝𝑜 (𝑐𝑖 < 𝑠) − 𝑝𝑜 (𝑠 < 𝑐𝑖 ) ]𝑠 ∈ 𝑆𝑖𝐶𝑖∈𝑆 (4)
where 𝑠 is a member of set 𝑆𝑖 and 𝑆𝑖 = { 𝑐𝑗 | 𝑗 > 𝑖, 𝑐𝑗 ∈ 𝑆𝐸 }. Then, the transition
term 𝐸𝑎𝑡𝑡 𝑇 is used to ensure that the way the user reads the components matches
the sequence of the components in 𝑆𝐸 . This term is determined by Eq. (5) below [13]:
𝐸𝑎𝑡𝑡 𝑇 = −
1
|𝑆𝐸|−1 ∑ 𝐶𝑖∈ 𝑆𝐸\𝑐𝑘 [𝑝𝑡 (𝑐𝑗 |𝑐𝑖 ) − max𝑒 ∈ 𝑁𝑖\𝑐𝑗
𝑝𝑡 (𝑐𝑘 |𝑐𝑖 )] (5)
where 𝑁𝑖 is the neighboring component of 𝑐𝑖 directly adjacent to it and 𝑒 is a member of set 𝑁𝑖 . Since mobile phones have a compact screen and our model includes all components on a page, we replaced variable 𝑒 from [13] with 𝑐𝑘. Thus, the transformation of variable 𝑒 in 𝑝𝑡 (𝑐𝑗 |𝑐𝑖 ) and 𝑝𝑡 (𝑐𝑘 |𝑐𝑖 ) is defined as in
Eqs. (6) and (7) as follows, respectively:
𝑝𝑡 (𝑐𝑗 |𝑐𝑖 ) = 𝑝𝑡 (𝑐𝑖→𝑐𝑗)
∑ 𝑝𝑡(𝑐𝑖→𝑐𝑘) 𝑘 𝑖=1
(6)
𝑝𝑡 (𝑐𝑘 |𝑐𝑖 ) = 𝑝𝑡 (𝑐𝑖→ 𝑐𝑘)
∑ 𝑝𝑡(𝑐𝑖→𝑐𝑘) 𝑘 𝑖=1
(7)
The attention energy of each component is compared with its baseline. The
difference shows the designer if the energy of each component should be
increased, decreased, or retained. Then, the designer iterates the design based on
the recommendation from the energy term.
3.3 Post-Design Stage
After the redesign process was finished, a second user test was performed with
the same participants. It was used to evaluate the usability and user experience of
mLearning after its UI was iterated based on the proposed model. The efficiency
was measured by the overall relative efficiency metric, the effectiveness was
measured by task completion, and the user experience was measured by the first-
click percentage.
24 Gladina Desideria & Yoanes Bandung
4 Result and Discussion
Two page designs (‘Home’ and ‘Subject Menu’) were chosen to represent
different tasks. Figures 2 and 3 show a comparison of the designer’s expected
user attention (left) and the actual user attention (right), which were totally
different.
Then, the attention energy was calculated to know each component’s attention
energy in order to make a recommendation for the design iteration process based
on Eq. (3). Based on the 𝐸𝑎𝑡𝑡 𝑇 for the page ‘Home’ (see Figure 4), the lowest
attention energy (indicated by a solid line) was shown by 𝑐1, which means that this component had the largest gap with its baseline (dash line). The users passed
over 𝑐1 and 𝑐2 because they were too small and did not look like clickable buttons.
The highest energy was owned by 𝑐5 and 𝑐6, which indicates that the users tended to focus their attention on large pictures with bright colors contrasting with the
background. As shown in Figure 2, 𝑐1 was read after 𝑐5, 𝑐3, 𝑐6. This component had low energy because it was not read according to the sequence in which the
page should be read. The actual user attention of page ‘Home’ was 𝑆𝐴 = {𝑐5, 𝑐3, 𝑐6, 𝑐1, 𝑐2, 𝑐7, 𝑐8, 𝑐4, 𝑐9}.
Figure 2 Design version 1: user attention comparison for the page ‘Home’.
User Efficiency Model in Usability Engineering 25
Figure 3 Design version 1: user attention comparison for the page ‘Subject Menu’.
Figure 4 Attention gaps for the page ‘Home’.
26 Gladina Desideria & Yoanes Bandung
The 𝐸𝑎𝑡𝑡 𝑇 for page ‘Subject Menu’ (in Figure 5) shows that 𝑐1 (solid line) had the
lowest attention energy and had the largest gap with its baseline (dash line).
Meanwhile, 𝑐2 had the highest energy and was closest to its baseline, since most users accomplished task 2 by first looking at this large and eye-catching menu.
They said that they did not even see the other components because they already
found 𝑐2. Moreover, they did not realize that the goal of the second task could be achieved with only one click on 𝑐1, because its color did not attract their attention. As shown in Figure 3, the actual user attention of the page ‘Subject Menu’ was
𝑆𝐴 = {𝑐2, 𝑐5, 𝑐3, 𝑐6, 𝑐4, 𝑐1}.
Figure 5 Attention gaps for the page ‘Subject Menu’.
After that, the designer refined both UIs referring to the energy gaps generated
by the recommender system by looking at the stripe-bar in Figure 4 to iterate the
page ‘Home’ and to Figure 5 to iterate the page ‘Subject Menu’. Making an
automatic design refinement in graphic design is very difficult [10] and
sometimes does not deliver a satisfying result [13], since it involves artistic taste,
which is impossible to compute. Hence, this study used the energy gaps as a
benchmark for the designer to iterate it manually
Figure 6 presents the result of the design iteration process by adjusting the color,
contrast, brightness, size, and font thickness in each design. For example, 𝑐1 of page ‘Home’ (left) was changed by adding a boundary that makes it look like a
clickable button and it was made yellow and the font was made larger and bold
to make it stand out more. Then, 𝑐5 and 𝑐6 were modified by changing the
User Efficiency Model in Usability Engineering 27
partition into three columns with a darker background color and making the icons
smaller. The icon’s color was deliberately made less bright than that of buttons
𝑐1 and 𝑐2. Another significant change was in component 𝑐2 of the page ‘Subject Menu’ (right). To make the users notice 𝑡ℎ𝑐1, the icon of the subject was changed by adding a correlated icon with a larger size and make it vertically arranged with
a lighter background (grey). To make the user read 𝑐1 before 𝑐2, the designer decreased the size of each icon of menu 𝑐2 and changed the background to a darker color (black).
Figure 6 Result of design refinement (design version 2).
After the design iteration was conducted, design version 2 was tested among the
20 participants. The result of the second user test for the page ‘Home’ showed
that the proposed model could increase the attention energy 𝐸𝑎𝑡𝑡 of some of the components. As presented in Figure 7, 𝑐1, 𝑐2, 𝑐3 and 𝑐6 had higher attention energy than before (see the solid line). Even though the model could not
completely raise the user attention to baseline energy (shown in Figure 4), at least
𝑐1 now had the highest attention energy. This means that the users chose 𝑐1 as the
28 Gladina Desideria & Yoanes Bandung
initial point of their order and transition matrix and saw it as the most efficient
component to complete their task. Although the energy of some of the
components was increased, the actual user attention of the page ‘Home’ was still
not sequenced as expected. The actual user attention was 𝑆𝐴 = {𝑐7, 𝑐1, 𝑐3, 𝑐5, 𝑐6, 𝑐2, 𝑐4, 𝑐8, 𝑐9}. Users tended to read the label of component (𝑐7) first and then looked for the component that only related to their task (𝑐1). Also, they preferred looking at an attractive picture (𝑐5,𝑐6) or at the bottom navigation (𝑐3).
Figure 7 Attention energy comparison for the page ‘Home’.
The result for the page ‘Subject Menu’ in Figure 8 shows that some components
(𝑐1, 𝑐3, 𝑐4) had attention energy enhancement, indicated by the solid line. Although 𝑐1 strongly increased, it still could not exceed the energy of 𝑐2. 𝑐2 constantly grabbed the user’s attention and was the component that was most
visited by the users. Nevertheless, the actual user attention of this page almost
reached the expected user attention 𝑆𝐸 with 𝑆𝐴 = {𝑐1, 𝑐4, 𝑐2, 𝑐3, 𝑐5, 𝑐6}. This proves that the design iteration could change how the users read the components.
The same as with the page ‘Home’, the reason why 𝑐4 was second in the order of 𝑆𝐴 was simply because the users tended to read the label of something first or right after they looked at something to make certain.
User Efficiency Model in Usability Engineering 29
Figure 8 Attention energy comparison for the page ‘Subject Menu’
4.1 Usability Evaluation
An evaluation related to reaching the usability goals (effectiveness and
efficiency) was performed. The effectiveness was measured by task completion
according to [26,27] and the efficiency was measured by overall relative
efficiency based on [26,27]. Table 2 presents the result of the usability evaluation.
Table 2 Usability evaluation.
Usability goals Metrics Design
version 1
Design
version 2
Effectiveness Completion rate 92.5% 90%
Efficiency Overall relative efficiency 83.65% 85.58%
After the design iteration from design version 1 to version 2, the effectiveness of
mLearning decreased from 92.5% to 90%. However, this number is still in the
safe range and above average. Sergeev in [26] states that products with an
effectiveness range of 90-100% are within the ‘good’ range and Sauro in [28]
states that the average rate of task completion is 78%. In contrast, the efficiency
increased from 83.65% to 85.58%. The design refinement helped the users to
complete their task more rapidly by directing the user attention to the most
efficient component.
30 Gladina Desideria & Yoanes Bandung
4.2 User Experience Evaluation
Figure 9 shows the discoverability of each component on the page ‘Home’. The
result shows that in the usability test on design version 1 (stripe bar), 50% of the
users first clicked on 𝑐3, 30% first clicked on 𝑐1 and 20% first clicked on 𝑐6. However, in the user test of design version 2 (solid bar), the discoverability of the
most efficient component increased: 65% of the users first clicked 𝑐1, 25% first clicked 𝑐2 and 10% first clicked 𝑐3.
Figure 9 Discoverability for the page ‘Home’.
Figure 10 shows the discoverability of the page ‘Subject Menu’. The result shows
that in the usability test of design version 1 (stripe bar), most users (80%) first
clicked 𝑐2, followed by 𝑐3 (20%). Similar to the previous page, in the user test of design version 2 (solid bar), the discoverability of the most efficient component
increased: 50% of the users first clicked 𝑐1, 15% first clicked 𝑐2 and 35% first clicked 𝑐3. The result shows better discoverability of the most efficient component within 𝑆𝐸 on both pages, which reduced the task completion time and increase efficiency.
User Efficiency Model in Usability Engineering 31
Figure 10 Discoverability for the page ‘Subject Menu’.
5 Conclusion
The proposed model for design refinement could successfully direct user
attention to the most efficient component. The components in the first user test
that failed to attract user attention were noticed by the users in the second user
test. The user experience goal, i.e. discoverability, helps users finish their task by
being able to quickly find the most efficient component related to their task. The
efficiency of the mLearning application increased from 83.65% to 85.58%
between the first and the second user test, but the effectiveness decreased slightly.
From this study, other facts about visual user attention were revealed. First, users
were interested in components with pictures and were attracted by colorful, bright
and contrasting components. Size and layout also grabbed their attention more.
The users tended to look at the textual information in a button first to make sure
they clicked on the right button. The area of a mobile application that received
most attention from the users was from the middle-top to the bottom of the screen.
Discoverability has a positive impact on the efficiency of a user interface design.
The more discoverable a component, the faster users complete their task. Thus, it
is better if the designer directs the user attention to the most efficient component.
This research had several limitations. Some things still need to be improved in a
future research. First, this recommender system is not yet able to provide definite
and detailed suggestions about which color is supposed to be changed, to what
32 Gladina Desideria & Yoanes Bandung
size a component should be changed, and other adjustments. It only shows what
components should be improved and the designers then has to refine them
intuitively. Second, in a next research it would be better to consider other usability
and user experience goals to give a broader insight into user attention in mobile
applications. Finally, since it is difficult to see at what component a user’s eye
gaze is directed at while using a compact mobile phone screen, the position of
attention a could be determined more accurately if it was analyzed by an eye
tracker.
References
[1] Singh, V., Android Based Student Learning System, 2nd Asia-Pacific World Congress on Computer Science and Engineering, 2015.
[2] Hamidi, H. & Chavoshi, A., Analysis of the Essential Factors for the Adoption of Mobile Learning in Higher Education: A Case Study of
Students of the University of Technology, Elsevier Journal Telematics and
Informatics, 35(4), pp. 1053-1070, 2018.
[3] Bandung, Y., Tanjung, D. & Subekti, L.B., Design of mLearning Application with Video Conference System for Higher Education, Int.
Conf. Electr. Eng. Informatics, 2017.
[4] Khalifa, M. & Lam, R., Web-based Learning: Effects on Learning Process and outcome, IEEE Trans. Educ., 45(4), pp. 350-356, 2002.
[5] Hwang, Y. & Kim, D.J., Understanding Affective Commitment, Collectivist Culture, and Social Influence in Relation to Knowledge
Sharing in Technology Mediated Learning, IEEE Trans. Prof. Commun.,
50(3), pp. 232-248, Sep 2007.
[6] Nielsen, J., Usability 101: Introduction to Usability, https://www.nngroup.com/articles/usability-101-introduction-to-usability/
(28 June 2019).
[7] International Organization for Standardization (ISO), International Standard: ISO 9241-11, 1998, p. 28, 1998.
[8] International Organization for Standardization (ISO), ISO 9241-210: Ergonomics of Human–system interaction – Human-centred Design for
Interactive Systems, p. 32, 2010.
[9] Kumi R., Conway, C., Limayem, M. & Goyal, S., Learning in Color: How Color and Affect Influence Learning Outcomes, IEEE Trans. Prof.
Commun., 56(1), pp. 2-15, 2013.
[10] O’Donovan, P., Agarwala, A. & Hertzmann, A., Learning Layouts for Single-page Graphic Designs, IEEE Trans. Vis. Comput. Graph., 20(8),
pp. 1200-1213, 2014.
[11] Bradley, S., Design Fundamentals: Elements, Attributes & Principles, 2013.
User Efficiency Model in Usability Engineering 33
[12] Guy, T., Visual Design and Usability Yellow Brick Road, from http://uxmag.com/articles/visual-design-and-usability-yellow-brick-road
(3 July 2019).
[13] Pang, X., Cao, Y., Lau, R.W.H. & Chan, A.B., Directing User Attention via Visual Flow on Web Designs, ACM Trans. Graph., 240, pp. 1-11, 2016.
[14] Oulasvirta, A., Feit, A., Lähteenlahti, P. & Karrenbauer, A., Computational Support for Functionality Selection in Interaction Design,
ACM Trans. Comput. Interact., 24(5), pp. 1-30, 2017.
[15] Interaction Design Foundation, Mobile User Experience (UX) Design, https://www.interaction-design.org/literature/topics/mobile-ux-design,
(17 June 2019).
[16] Cardello, J., Low Findability and Discoverability: Four Testing Methods to Identify the Causes, https://www.nngroup.com/articles/navigation-ia-
tests/ (28 June 2019).
[17] Interaction Design Foundation, Make it Easy on the User: Designing for Discoverability within Mobile Apps, https://www.interaction-
design.org/literature/article/make-it-easy-on-the-user-designing-for-
discoverability-within-mobile-apps (2 July 2019).
[18] Jamil, N., Quadratic Optimization for User Interface Layout, 8th Int. Conf. Digit. Inf. Manag. ICDIM 2013, pp. 280-285, 2013.
[19] Interaction Design Foundation, User Interface (UI) Design, from https://www.interaction-design.org/literature/topics/ui-design. (11 June
2019).
[20] K. Finstad, The Usability Metric for User Experience, Interact. Comput., 22(5), pp. 323-327, 2010.
[21] Knemeyer, D. & Svoboda, E., User Experience - UX in The Glossary of Human Computer Interaction, The Interaction Design Foundation, 2015.
[22] Nielsen, J. & Pernice, K., Eyetracking Web Usability, Berkeley, United States: New Riders, 2009.
[23] Borji, A. & Itti, L., State-of-the-Art in Visual Attention Modeling, IEEE Trans. Pattern Anal. Mach. Intell., 35(1), pp. 185-207, 2013.
[24] Cox, P., Visual Movement: Flow in Web Design, https://tympanus.net/ codrops/2011/01/14/visual-movement-flow/, (18 June 2019).
[25] Nielsen, J., Usability Engineering, California, Academic Press, 1993. [26] Sergeev, A., User Interface Design, UX Research and Usability
Evaluation, http://ui-designer.net/usability/, (last accessed: 11-June-2019).
[27] Mifsud, J., Usability Metric – A Guide To Quantify The Usability Of Any System, http://usabilitygeek.com/usability-metrics-a-guide-to-quantify-
system-usability/, (27 June 2019).
[28] Sauro, J., What is a Good Task-Completion Rate?, https://measuringu.com /task-completion/ (3 July 2019).