1
Research Paper: Modality and Redundancy Effects for all Learners
Courtney Zimmerman
School of Education, Liberty University
EDUC 734: Implementation and Assessment of Multimedia Learning
Tools Dr. David Abraham
May 2, 2021
2
Abstract
Modality and redundancy are common principles that can be associated with on screen text and
narration effects with regard to multimedia learning. Some literature suggests that the
redundancy of narration with on screen, visual text can cause extraneous cognitive load in
learners which leads to lowered knowledge acquisition and retention. There is, however,
additional literature which argues that under certain circumstances, these same factors contribute
to reverse redundancy, with the redundant narration and on screen text providing opportunity for
improved knowledge acquisition and retention. This research paper is designed to articulate the
concepts of modality and redundancy with respect to multimedia learning as well as to examine
circumstances and situations in which these principles do not hold up or a reverse redundancy
effect is reached and how each of these effects are being noted as affecting the cognitive load of
its learners.
Keywords: redundancy, modality, multimedia learning, reverse redundancy, cognitive
load
3
Introduction
Multimedia learning tools have been a part of the education spectrum for a number of
years, however, the recent flip to distance and hybrid learning due to COVID-19 lockdowns has
really highlighted the necessity and efficacy of these learning tools. Moreover, now that the
frequency and volume of use of multimedia learning tools has increased, so have the research
surrounding relative theories of the use and integration of these tools into school based
curriculum. Among these theories are the basic concept of cognitive load, and associated theory
of multimedia learning, as well as the modality and redundancy effects. While cognitive load
theory is more philosophical in its inception and development, there has been research and case
studies to support how cognitive load affects learning in the classroom. From that, many
educators and instructional designers, as well as many of the authors of the articles cited in this
paper, recognize Richard Mayer’s cognitive theory of multimedia learning which, in turn, gives
merit to the modality and redundancy effects (Clark and Mayer, 2016). Together, these
principles and theories have governed the concept of multimedia learning tool development,
with minor given exceptions. It is these exceptions, however, that have sparked numerous
additional studies into the concepts of modality and redundancy as they relate to cognitive load,
and how the noted exceptions may be more of a common occurrence than previous studies have
considered and that the theories may have the total opposite effect on some students.
Cognitive Load
The cognitive load theory identifies the ability our brain has to articulate materials
presented to us through visual and auditory stimuli and how much our brain is able to process
4
into short term memory in order to move toward knowledge acquisition and retention. This
theory, which is very much related to the theory of working memory postulates that we have two
separate storages within our working memory, one phonological and the other visuo-spacial, for
processing and retaining information, but each store has a finite amount of space (Chan et al.,
2020). Therefore, the concept of excessive cognitive load, for example, is expressed when too
much information is being presented through the visual processing of both visual images and on
screen text, therefore decreasing the ability of the learning to memorize the material (Munir et
al., 2018) This means that our brain essentially gets overloaded on processing visual information
and is not able to retain as much, if any, due to the overload. Another aligned theory is that of
dual coding which states that when information is presented both visually and auditorily, our
brain is better able to code the information into the respective working memory stores (Chan et
al., 2020).
While the concept of cognitive load and excessive cognitive load are fairly simplistic and
can make sense in certain context, the cognitive theory of multimedia learning takes the concept
with direct relation to multimedia learning tools. The cognitive theory of multimedia learning
argues that there are separate systems in our brain to process visual and auditory information and
that the most meaningful learning occurs when we are able to establish a meaningful connection
between the two (Ozdemir & Sahin-Izmirili, 2016). Therefore, utilizing multimedia tools with
effective modality and redundancy effects in consideration, should make more students capable
of relating material in both an auditory and visual way in order to achieve appropriate knowledge
acquisition. In order to dive into how this theory is most directly applied to education, the effects
of modality and redundancy must also be explained.
5
Modality and Redundancy
Modality, in the context of education, is simply the method or mode that curriculum is
taught or expressed. Multiple-modality teaching allows students to utilize multiple methods of
learning to help ensure knowledge acquisition and retention. Related to multimedia learning, the
modality principle suggests that multimedia information should be presented with visual and
auditory narration instead of full, on-screen text (Munir et al., 2018). The idea is that curricular
information will be presented in a dual modality delivery and because information is presented
visually through images/animations, and auditorily through media, students do not have to dive
their attention in order to access the knowledge (Munir et al., 2018).
Through visual processing of information (which can be visual animations and/or visual
text) we come across the concept of redundancy. Idea of the redundancy effect is that working
memory/cognitive load can become overloaded or extraneous if learners are taught using
multimedia learning tools that combine spoken and written text simultaneously (Ozdemir &
Sahin-Izmirili, 2016). This double dose of visual information may cause excessive cognitive
load and present information from appropriately being transferred into working memory. The
redundancy of the on-screen text visuals with the on-screen animation visuals can lead to what is
called split attention. split attention hypothesis assumes that the simultaneous on-screen text
coupled with visual animation in multimedia learning tools would split the learners visual
channel by attempting to maintain focus on both sources (Chan et al., 2020). Many studies,
including ones done by Mayer, provide substantial, empirical evidence that learners have more
significant knowledge retention when audio is used to describe visual graphics and animations
rather than on-screen text, and that adding the on screen text can cause redundancy and cognitive
overload which will prevent knowledge retention. Despite all the evidence to support the concept
6
of cognitive theory of multimedia learning and support of the redundancy and modality
principles, there is still concerns about the homogenous nature of these research studies and a
growing concern that the cited exceptions to the cognitive theory of multimedia learning may
have a very different effect on learning with regard to redundancy and cognitive overload and
may be more of a standard than a minimal exception.
Exceptions and Reverse Redundancy
Although the concept of redundancy is simple, its effect has shown to be profound in
regards to cognitive load and retention. Colvin and Mayer (2016) mention that learners fare
better from audio narration with multimedia graphics without the support of on-screen text.
Additionally, they mention that “When their eyes are on the printed words, learners cannot be
looking at the on-screen graphics” (Colvin and Mayer, 2016, pg 133). This qualifies the on-
screen text as redundant and causing cognitive overload via the visuo-spacial storage in learner
memory. However, Covlin and Mayer (2016) also talk about the situations where adding on
screen text may be beneficial and refer to these specific “Special Situations”:
1. When there are no graphics.
2. When the on-screen text and graphics are presented at a pace slow enough that the
learner is able to visually process everything presented.
3. The on screen text is short or summarized and not the full text.
4. The learner struggles to understand auditory information either because of a
disability or language deficit.
Combining with these special situations, which in some cases are far more common than they
seem to be given credit for, there is some concern that many of the studies and research done
7
about the redundancy principle are done with a homogenous set of neurotypical students which
does not give adequate demographic representation to the current educational classroom
structure. In addition, some believe that Mayer’s cognitive theory of multimedia learning was
developed based on studies of English speaking students, not considering the concepts for
students who are considered English Language Learners (Liu et al., 2018). This is not just a
struggle for ELL students, either. Students with special needs often expect visual teaching and
learning as well as involvement directly in the learning process (Munir et al., 2018). Therefore,
adding closed captioning, given the special factors identified by Colvin and Mayer (2016) and
more recent studies that focus on identifying the effect of redundancy principal on students with
disabilities and English Language Learners (ELL) along with neurotypical and native English
speaking students, may have more of a positive effect on a larger group of students than
previously thought.
Mitigating factors of redundancy and reverse redundancy
Redundancy effect promotes the idea that having a multimedia presentation with both
auditory and on screen text can have a negative effect on learning due to the overload of the
system trying to process the visual of the multimedia and the visual of the on screen text
simultaneously (Ozdemir & Sahin-Izmirili, 2016). There is also evidence that while multimedia
tools for learning are an effective method of promoting knowledge acquisition, time restraints
for viewing the presentation and processing of these multimedia tools can increase the negative
effect of cognitive load with or without the redundancy effect (Munir et al., 2018). There are
even studies that have shown the redundancy effect doesn’t have as significant or negative an
impact as other studies have shown. In their study, Ozdemir & Sahin-Izmirili (2016) found that
closed captioning on instructional videos either had no significant impact on achievement, or
8
seemingly improved the level of achievement for the learner. Yet another study showed that
short summary text that accompanies visual animation in multimedia learning tools can actually
promote knowledge acquisition rather than diminish it as full on screen text has been shown to
do (Chan et al., 2020).
With this in mind, and while there is a special circumstance mentioned about ELL
students, there is also data to support that closed captioning or on-screen text can be widely
beneficial to students who are not only not familiar with the English language, but also when the
material being presented is newer vocabulary or complicated information for the learners to
disseminate on their own. ELL students can actually face excessive cognitive load when
decoding auditory material so on screen text may reduce the cognitive load and support
comprehension of the material more effectively (Liu et al., 2018). Additionally, a study by John
et al. (2016) showed that students who are considered ELL had increased performance and a
relatively reduced cognitive load when subtitles accompanied multimedia learning tools.
Even more, there is a growing concern with regard to the accent of the video’s narration
and if that narration is not presented in a native accent, that can cause its own cognitive overload
and prevent appropriate access to the multimedia material. If the narration used in the
multimedia learning tool is in a foreign accent, oftentimes it requires additional effort to listen
and understand, essentially overloading the phonological working memory channel (Chan et al.,
2020). This can also be applied to concepts of native speakers learning a foreign language.
When watching an instructional video in or about a foreign language, closed captioning videos
have shown to be more effective in knowledge acquisition than non-captioned videos. (Ozdemir
& Sahin-Izmirili, 2016). Concurrently, On screen text may support learner acquisition of
knowledge by reconciling ambiguous, accented speech with on screen text (Chan et al., 2020).
9
Conclusion
While there has been data and evidence to support the concept of the modality and
redundancy effects, especially as it relates to the philosophical concept of cognitive load and its
roots in the Cognitive Theory of Multimedia Learning, there is a facet of the educational
community that is concerned more that these principles and effects are only representing a
portion of the learners that are present in traditional educational classrooms. With an increase in
ELL students in traditional classrooms, multimedia learning tools designed for native English
speakers can likely be a disadvantage to non native speakers in the classroom (Liu et al., 2018).
It can also be said that students who are native English speakers may struggle to appropriately
access multimedia learning tools without subtitles due to deficits in the area of auditory
processing, in cases where the information is new or heavy in vocabulary, or in cases where
there ae no appropriate graphics to represent the material. Adding on screen text can not only
support students to reconcile difficult vocabulary or complexe concepts, but some studies show
ELL students tend to have more comprehension with written input rather than auditory input
therefore full on-screen text may improve knowledge acquisition in ELL learners, essentially
promoting reverse redundancy (Liu et al., 2018).
Above all, educators need to ensure that their modality of delivery of curriculum to
students gives the most comprehensive opportunity to the students to access the knowledge being
presented. “The integration of online learning technologies must consider the alignment among
many pedagogical factors to optimize the effects of the employed technologies on learners”
(Dutra de Oliviera Neto et al., 2015, pg 566). This is where effects like the modality and
10
redundancy principles as well as the cognitive theory of multimedia learning have evolved from.
THe Voice effect states that students will be better able to acquire knowledge if multimedia
learning tools are narrated in a native accent rather than in a foreign one (Chan et al., 2020)
Therefore, multimedia learning tools which use dual modality presentation (auditory and visual)
expand processing capacity of the working memory by utilizing both channels of working
memory (Chan et al., 2020). However, there needs to be consideration made to the accent of the
narration so that there isn’t confusion from the learners. Study by Chan et al. (2020) found that
when accented narration was included in multimedia learning tools, retention increased when full
on screen text was presented concurrently. Adding in the studies that show students with learning
disabilities and auditory processing deficits actually struggle with excessive cognitive load due to
the lack of on-screen text and studies like that by Liu et al. (2018) which found no significant
differences between learning outcomes with on screen full text simultaneous with narrated
multimedia learning tools than with no on screen text accompanying narrated multimedia
learning tools, show that there might not be a one size fits all approach to multimedia learning.
Therefore, some compromise may be made in order to grant access to the most amount of
students. One notable compromise that can be made to allow closed captions on multimedia
learning tools and still provide adequate learning opportunities to the most students is to allow
students to self-pace the multimedia learning tool. This way students can take their time to not
only watch the visual animations and the listen to the audio narration, but they can also pause to
read the on-screen text, or rewind to review the entire material all over again. Simply utilizing
multimedia learning tools will not fully ensure knowledge acquisition by students (Dutra de
Oliviera Neto et al., 2015). As educators, consideration needs to be made about not only how to
supply the information to our students with consideration to the evidence based practices and
11
current research information available, but to look at the composition of our students and apply
our curriculum in a way that will benefit the majority of our students, and given consideration to
how we can still support those that may be left out.
12
Lesson Plan
1. Gaining Attention
a. Students are given a bell ringer which asks them to identify the order of cells
within a complex organism (cell, tissue, organ, organ system, organism).
Students write down their response to the bell ringer in their Journal.
2. Informing Learners of objective
a. How is protein involved in cells being able to communicate and integrate
within an organism? (GoogleSlide with presented question, classroom
discussion)
3. Stimulating recall of prior learning
a. Google slide about the organelles of a cell (review from last unit, class discussion)
4. Presenting the stimulus
a. Introduce the concept of active transport, passive transport, and osmosis (video
from amoeba sisters students watch video on their own devices and can chose to
turn on the closed captioning or not)
5. Providing learning guidance
a. Students will take notes on their interactive notebook worksheet, then cut out the
indicated sections to create a filp-book which will be glued into their notebook.
i.
13
6. Eliciting performance
a. After completing the INB flip book, students are grouped in A and B sections
of the classrooms and asked to identify processes as being active transport,
passive transport, or osmosis as part of a class competition game.
7. Providing feedback
a. Answers are discussed as a class, whether they are right or wrong, and
information is reviewed if necessary.
8. Assessing Performance
a. Students must individually answer two questions about the material covered in
class. These answers must be written as part of the exit ticket that is
completed before they are allowed to leave class for the day.
9. Enhancing Retention and Transfer
a. Next class, students have a bell ringer that reviews the information that was
covered in class this time. (asking to explain the difference between active
transport and passive transport)
14
References
Ari, F., Flores, R., Inan, F. A., Cheon, J., Crooks, S. M., Paniukov, D., & Kurucay, M. (2014).
The effects of verbally redundant information on student learning: An instance of reverse
redundancy. Computers and Education, 76, 199-204.
https://doi.org/10.1016/j.compedu.2014.04.002
Chan, K. Y., Lyons, C., Kon, L. L., Stine, K., Manley, M., & Crossley, A. (2020). Effect of on-
screen text on multimedia learning with native and foreign-accented narration. Learning
and Instruction, 67, 101305. https://doi.org/10.1016/j.learninstruc.2020.101305
Clark, R. C., & Mayer, R. E. (2016). E-learning and the science of instruction: Proven
guidelines for consumers and designers of multimedia learning. John Wiley & sons.
Dousay, T. A. (2016). Effects of redundancy and modality on the situational interest of
adult learners in multimedia learning. Educational Technology Research and
Development, 64(6), 1251-1271. https://doi.org/10.1007/s11423-016-9456-3
Dutra de Oliveira Neto, J., Huang, W. D., & Cristina de Azevedo Melli, N. (2015). Online
learning: Audio or text? Educational Technology Research and Development, 63(4), 555-
573. https://doi.org/10.1007/s11423-015-9392-7
John J. H. Lin, Lee, Y., Wang, D., & Sunny S. J. Lin. (2016). Reading subtitles and taking
enotes while learning scientific materials in a multimedia environment: Cognitive load
perspectives on EFL students. Educational Technology & Society, 19(4), 47-58.
15
Knoop-van Campen, C. A. N., Segers, E., & Verhoeven, L. (2019). Modality and redundancy
effects, and their relation to executive functioning in children with dyslexia. Research in
Developmental Disabilities, 90, 41-50. https://doi.org/10.1016/j.ridd.2019.04.007
Liu, Y., Jang, B. G., & Roy-Campbell, Z. (2018). Optimum input mode in the modality and
redundancy principles for university ESL students' multimedia learning. Computers and
Education, 127, 190-200. https://doi.org/10.1016/j.compedu.2018.08.025
Munir, M., Setiawan, W., Nugroho, E. P., Kusnendar, J., & Wibawa, A. P. (2018). The
effectiveness of multimedia in education for special education (MESE) to improve reading
ability and memorizing for children with intellectual disability. International Journal of
Emerging Technologies in Learning, 13(8), 254-263.
https://doi.org/10.3991/ijet.v13i08.8291
Ozdemir, M., Izmirli, S., & Sahin-Izmirli, O. (2016). The effects of captioning videos on
academic achievement and motivation: Reconsideration of redundancy principle in
instructional videos. Educational Technology & Society, 19(4), 1-10..
