According to hypothesis 4: "Subjective fatigue and multitasking behaviors will be positively

What else do college students “do” while studying? An investigation of multitasking

Charles Calderwood a,*, Phillip L. Ackerman b, Erin Marie Conklin b a Virginia Commonwealth University, Department of Psychology, 806 West Franklin Street, P.O. Box 842018, Richmond, VA, 23284-2018, USA b Georgia Institute of Technology, School of Psychology, 654 Cherry Street, Atlanta, GA, 30332-0170, USA

a r t i c l e i n f o

Article history: Received 26 September 2013 Received in revised form 30 January 2014 Accepted 1 February 2014 Available online 18 February 2014

Keywords: Media in education Post-secondary education Learning strategies

a b s t r a c t

We investigated the frequency and duration of distractions and media multitasking among college students engaged in a 3-h solitary study/homework session. Participant distractions were assessed with three different kinds of apparatus with increasing levels of potential intrusiveness: remote surveillance cameras, a head-mounted point-of-view video camera, and a mobile eyetracker. No evidence was ob- tained to indicate that method of assessment impacted multitasking behaviors. On average, students spent 73 min of the session listening to music while studying. In addition, students engaged with an average of 35 distractions of 6 s or longer over the course of 3 h, with an aggregated mean duration of 25 min. Higher homework task motivation and self-efficacy to concentrate on homework were associ- ated with less frequent and shorter duration multitasking behaviors, while greater negative affect was linked to longer duration multitasking behaviors during the session. We discuss the implications of these data for assessment and for understanding the nature of distractions and media multitasking during solitary studying.

! 2014 Elsevier Ltd. All rights reserved.

1. Introduction

For school children, the amount of schoolwork completed at home typically increases with increasing grade levels. For example, the typical 9th grade student is in the classroom for about 30 h per week, and has 7½ h per week of assigned homework. A 12th grade student typically will spend the same amount of weekly time in the classroom, but will have about 10 h per week of homework. In contrast, when students reach college, they will spend only approximately 15 h per week in the classroom, but are expected to spend 30 or more h per week engaged in studying and homework outside of the classroom. While cultural differences may exist in the amount of homework assigned in different countries (e.g., Chen & Stevenson, 1989), the trend of more homework assigned with increasing grade levels has been empirically supported (see Cooper, Robinson, & Patall, 2006; Cooper & Valentine, 2001).

The increased prevalence of cell phones, other communication technologies, and portable audio devices in contemporary college student populations (see Jacobsen & Forste, 2011) has created the potential for significant attentional conflicts when students complete schoolwork outside of the classroom. One major source of conflict stems from a desire to engage in non-schoolwork activities. A second major source of conflict results from a lack of intrinsic interest in homework activities, and a desire to do anything other than study (Leone & Richards,1989). In combination, these conflicts likely exacerbate the appeal of using technological devices in the study environment, as these sources of distraction present an easy outlet for the alleviation of boredom during homework completion. Distractions and media multitasking are important issues to study in college student populations, as these students experience little parental or instructor oversight of their study habits. These issues are also particularly salient for members of the current generation of college students, who have been dubbed the “Multitasking Generation” (Wallis, 2006) due to the ubiquity with which they incorporate technology into their daily lives.

1.1. Quantifying college student media multitasking

Despite the widespread recognition of the pervasiveness of technology in contemporary college student life, investigators have yet to objectively explore the frequency and duration with which students multitask with media in their homework environment. Instead,

* Corresponding author. Tel.: þ1 804 828 8352. E-mail addresses: [email protected], [email protected] (C. Calderwood).

Contents lists available at ScienceDirect

Computers & Education

journal homepage: www.elsevier.com/locate/compedu

http://dx.doi.org/10.1016/j.compedu.2014.02.004 0360-1315/! 2014 Elsevier Ltd. All rights reserved.

Computers & Education 75 (2014) 19–29

researchers have placed primary emphasis on observing or experimentally manipulating multitasking behaviors in classroom environments (e.g., Hembrooke & Gay, 2003; Kraushaar & Novak, 2010; Sana, Weston, & Cepeda, 2013). Those studies which have featured explorations of media multitasking outside of the classroom have relied almost exclusively on self-report data (e.g., Jacobsen & Forste, 2011); despite a lack of evidence that students can or are willing to accurately report multitasking behaviors. In a seven day experience sampling study of student internet use, Moreno et al. (2012) found that the correlation between students’ estimated hours per day using the internet and the sum- mation of several within-day concurrent internet-use reports was only r ¼ .31, suggesting that students have a limited ability or willingness to accurately estimate their media use. Due to these limitations, there is a strong need for research that objectively quantifies media multitasking in college students completing schoolwork outside of the classroom.

An overreliance on self-report measures in media multitasking studies has also created a dearth of knowledge regarding alternative methodological approaches to examine student multitasking. A variety of observational technologies, such as surveillance systems, head- mounted video cameras, and eyetracking devices, have the potential to be useful tools to explore media multitasking. However, research on these observational technologies has not yet been extended to the homework environment. While such technologies may allow for a more accurate assessment of rates of student multitasking, it is also possible that these more intrusive observational technologies alter student behavior by engendering participant reactivity effects (see Whitley, 2002). Accordingly, it is necessary to analyze both the degree to which alternative observational technologies allow for the quantification of multitasking behaviors and whether these technologies are differ- entially associated with participant reactivity effects.

1.2. Exploring why college students media multitask

In addition to placing primary emphasis on subjective reports of media multitasking, investigators have generally focused on how students multitask with media and who is likely to engage in media multitasking, rather than why students do or do not engage in these behaviors (see, for example, Foehr, 2006). As investigators have linked media multitasking to impaired academic task performance (e.g., Fox, Rosen, & Crawford, 2009), there is a paradox as to why students would choose to multitask with media during homework completion (see Wang & Tchernev, 2012). Wang and Tchernev (2012) have recently provided evidence to suggest that, although perceived cognitive needs usually drive the initiation of multitasking behaviors, multitasking with media primarily satisfies emotional needs. Despite this recognition, there have been no studies to link objective observations of multitasking behaviors during homework completion to mood or task moti- vation in college student samples.

The first step in exploring the relationship of mood and task motivation to multitasking during homework completion is to establish whether these processes change over the course of the homework period. Mental work refers to activities accomplished against resistance (Dodge, 1913). All else being equal, sustained periods of mental work are theorized to drain cognitive and attentional resources (Hockey, 1997; Kahneman, 1973). In turn, the depletion of cognitive and attentional resources has been shown to have implications for mood and motivational processes (see Hockey, 2011). Although empirical researchers have observed subjective mood decrements during periods of sustained academic task performance (e.g., Ackerman & Kanfer, 2009), these findings have not yet been generalized to the homework environment. Based on predictions derived from attentional and cognitive resource theories, we predict that decrements in mood and motivation will accompany sustained periods of homework completion.

Hypothesis 1. Mood and motivation will be impaired over sustained periods of homework completion. While resource-based perspectives imply that engagement in homework tasks will impair mood and task motivation, they do not directly

address which mood and motivational processes are related to multitasking behaviors. In the following sections, we review three sets of mood and motivational variables which are likely to be associated with multitasking behaviors during homework completion. Throughout these sections, we provide specific hypotheses regarding the relationships of these mood and motivational variables to multitasking.

1.2.1. Negative and positive affect Negative and positive affect (NA and PA) respectively refer to the experience of negative and positive mood states (Watson, Clark, &

Tellegen, 1988). Theorists have argued that affective experiences have important implications for the allocation of cognitive resources between on-task thoughts and off-task distractions (Beal, Weiss, Barros, & MacDermid, 2005). However, there is evidence to suggest that negative and positive affective experiences are differentially related to this resource allocation process. NA has been consistently linked to engagement in ruminative thought (see Thomsen, 2006 for a review), which has been proposed as a factor in the allocation of cognitive resources to off-task distractions (Beal et al., 2005). In contrast, PA has been theorized to broaden attentional and cognitive resources (Carver, 2003; Fredrickson, 2001), and positive mood states have been associated with more careful processing of goal-relevant (i.e., on- task) information (see Aspinwall, 1998). Based on theories linking affective experiences to cognitive resource allocation and past empir- ical findings regarding the effects of NA and PA on cognitive resources and attention, we anticipate NA to be linked to more frequent and longer duration multitasking behaviors, while we expect PA to be associated with less frequent and shorter duration multitasking behaviors.

Hypothesis 2. NA and multitasking behaviors will be positively correlated.

Hypothesis 3. PA and multitasking behaviors will be negatively correlated.

1.2.2. Subjective fatigue Subjective fatigue refers to feelings of tiredness or lack of energy that are not related exclusively to exertion (see Brown & Schutte, 2006).

As homework tasks are identified by several characteristics commonly associated with fatigue (see Ackerman, Calderwood, & Conklin, 2012 for a review), sustained periods of homework activity are likely to correspond to increasing levels of fatigue over time. As it relates to off-task distractions, Davis (1946) was one of the first researchers to identify that some individuals divert attention away from primary tasks under conditions of fatigue. While not studied in relation to media multitasking specifically, researchers have generally supported this observation, finding that the ability to regulate goal-directed perceptual and motor processes is compromised under fatiguing conditions (e.g., van der

C. Calderwood et al. / Computers & Education 75 (2014) 19–2920

Linden, Frese, & Meijman, 2003). Therefore, students experiencing subjective fatigue while completing homework are more likely to engage with off-task distractions in their study environment, as fatigue compromises their ability to regulate their goal-directed homework behavior. Based on these lines of empirical evidence, we predict that higher subjective fatigue will be associated with more frequent and longer duration multitasking.

Hypothesis 4. Subjective fatigue and multitasking behaviors will be positively correlated.

1.2.3. Task motivation and self-efficacy Homework task motivation refers to a drive to perform well on homework tasks, while homework self-efficacy reflects a judgment of

one’s abilities to accomplish homework tasks (for a discussion of self-efficacy and task motivation constructs, see Bandura, 1982; Deci, 1975, respectively). These motivational processes are seen as integral in the achievement of goal-directed behavior, and have important impli- cations for academic task performance and related outcomes (see Pajares, 1996 for a review). The ability to resist distraction is seen as a component of the regulation of behavior in the service of accomplishing goal-directed behavior (see Zimmerman, Bandura, & Martinez- Pons, 1992). Applied to the homework environment, individuals who exhibit a higher motivation and self-efficacy to perform well and focus on their homework tasks should be less likely to engage with off-task distractions in their homework environment. Based on this reasoning, we expect that higher homework task motivation and self-efficacy to focus on homework tasks will be associated with less frequent and shorter duration multitasking.

Hypothesis 5. Homework task motivation and multitasking behaviors will be negatively correlated.

Hypothesis 6. Homework self-efficacy and multitasking behaviors will be negatively correlated.

2. Goals of the current investigation

There were three primary goals of the current investigation. First, we used a variety of observational methods to determine how many interruptions college students engage with, the duration of these interruptions, and the proportion of time spent media multitasking during homework completion. Second, we sought to determine whether potentially intrusive objective means of assessing multitasking behaviors have reactive effects on student behaviors. To accomplish this goal, we tested for potential behavioral differences when students were monitored with: (1) A system of four surveillance cameras; (2) A system of four surveillance cameras and a head-mounted point-of-view (POV) camera; and (3) A system of four surveillance cameras and a mobile eyetracker. Third, we sought to explore the relationships of distraction frequency and duration to self-report measures of affect, fatigue, motivation, and self-efficacy during homework completion. Deriving our hypotheses from theoretical models of cognitive resources, resource allocation, and task motivation, we expected higher NA and fatigue to be associated with greater multitasking behaviors, while we anticipated higher PA, homework task motivation, and self- efficacy to be linked to fewer multitasking behaviors.

Although other studies have examined student media multitasking in a laboratory setting (e.g., Ie, Haller, Langer, & Courvoisier, 2012), they have typically done so using somewhat artificial laboratory tasks. To increase the external validity of our study results, we explored media multitasking in a less constrained environment, by having students complete their own homework during the study session. As such, we consider the current investigation to be complementary to studies utilizing more constrained tasks.

3. Method

3.1. Sample

Sixty undergraduate students at the Georgia Institute of Technology participated in the study. Participants were recruited through recruitment flyers posted on campus and in-class announcements in undergraduate psychology classes. Participants were offered course extra credit for their participation. Inclusion criteria were that students were currently enrolled in at least one course each of math, science, and one other subject. Participants were randomly assigned to one of three conditions. Because of hardware/software failures, eyetracking data were lost for two participants. Therefore, complete data were available for 58 participants (N ¼ 58).

3.2. Procedure

Participants were instructed to bring 3 h of homework comprising three different subjects to the laboratory. They were told that they could bring their laptop, mp3 player/CDs or other audio materials, and cell phone to the study, and that they would be allowed to use all of these items. We also informed participants that we would provide an Internet-connected desktop computer and a printer for them to use during the session. On arrival, participants in the POV and mobile eyetracker conditions were fitted with the respective apparatus. Par- ticipants were instructed to “do your homework as you would anywhere else,” and asked to complete a short questionnaire at the beginning of each hour of the session. They were allowed to leave the room if necessary (e.g., for a bathroom break) by first removing any apparatus, or to eat or drink in an area away from the computer. Self-report questionnaires, which took approximately 3 min to complete, were administered at 0 min, 63 min, and 126 min after the equipment calibration protocols were completed.

Participants were provided with a desk, a computer connected to the Internet, a printer, a JVC ‘boombox’ with an mp3 input cable, and a flat work surface. Partitions were setup in the work area to limit the participant’s view of other areas of the laboratory (see Fig.1). A research assistant entered the room once each hour to administer brief questionnaires or to swap out a memory card. Other than these brief in- terruptions, research assistants did not intrude during the 3 h homework session.

Four small high-definition cameras and a microphone were placed to observe the participant’s activities while studying/completing homework. One dome camera was placed overhead, one camera above and in front of the participant, and two cameras above and on either

C. Calderwood et al. / Computers & Education 75 (2014) 19–29 21

side behind the participant (a still sample from the cameras is shown in Fig. 1). Video was recorded from all four cameras simultaneously. The cameras were active in all three experimental conditions.

3.3. Experimental conditions

3.3.1. POV Camera condition In this condition, participants had a small high-definition V.I.O. POV camera attached via a headband. The physical apparatus is shown in

Fig. 2a, and a screen-shot from the POV is shown in Fig. 2b. Twenty participants were randomly assigned to the POV condition (n ¼ 20).

3.3.2. Mobile eyetracker condition In this condition, participants wore a Tobii mobile eyetracker device, which is similar to a large pair of glasses (see Fig. 3a). The mobile

eyetracker contains a small video camera that records where the participant is looking, and the playback system overlays a red dot (see the web version of this article) indicating eye fixations and gaze movements over the video stream (see Fig. 3b). Twenty participants were randomly assigned to this condition, but due to equipment failures, complete data were only available for 18 participants (n ¼ 18).

3.3.3. Surveillance-only condition In this condition, participants wore no recording devices during the session. Twenty participants were randomly assigned to this

condition (n ¼ 20).

3.4. Data coding

Recorded videos from the three sets of devices were played-back while research assistants coded the frequency and duration of any non- homework-related events drawn from ten different distraction categories (see Table 1). Distraction categories were developed from commonly reported behaviors in a 5-day pilot study in which students self-reported their multitasking behaviors while completing homework. All video footage included a running time-stamp used to calculate distraction duration. Coders were instructed to record the following pieces of information any time that a participant engaged in a behavior corresponding to one or more of the distraction categories: 1) The distraction category or categories to which the behavior(s) belonged; 2) The start time of the behavior(s) on the time-stamp; 3) The end time of the behavior(s) on the time-stamp; and 4) Any additional comments necessary to identify the behavior(s) in question. The recorded videos for a subsample of 20 participants were randomly selected to be re-coded by a second research assistant to estimate inter- rater agreement. Inter-rater agreement was quantified via computation of an intraclass correlation coefficient (see Shrout & Fleiss, 1979), with values closer to 1 representing greater agreement. The estimated inter-rater agreement for multitasking frequency was ICC ¼ .72, while the estimated inter-rater agreement for multitasking duration was ICC ¼ .90.

Fig. 1. Student workstation layout as seen from four surveillance cameras. Clockwise from upper left: Overhead dome camera display, distant camera from the right of the workstation, distant camera from the left and behind the workstation, and computer table display from behind and above the workstation. Shown in the displays are the computer, workstation chair, printer, and work surfaces. The boombox (with mp3 input access) can be seen in the upper part of the first camera display, on the floor behind the workstation.

C. Calderwood et al. / Computers & Education 75 (2014) 19–2922

3.5. Self-report measures

A brief 34-item measure of state affect, fatigue, self-efficacy, and positive motivation (comprised of items from the Positive and Negative Affect Schedule; Watson et al., 1988; Profile of Mood States, McNair, Lorr, & Droppleman, 2003; and locally developed items) was administered at the beginning of each hour of the laboratory session. This measure was drawn from a subset of items developed in a previous study of mood, fatigue, and motivation during sustained academic task performance (see Ackerman & Kanfer, 2009).

3.5.1. Negative and positive affect Twelve items referring to the experience of negative mood states (a ¼ .83–.86 across the three hourly administrations in the session) and

seven items referring to the experience of positive mood states (a ¼ .90–.91). An example negative affect item is “distressed”, while an example positive affect item is “enthusiastic.” Students were asked to indicate the degree to which each statement described how they currently felt on a 5-point Likert-type scale (1 ¼ Very slightly or not at all, 2 ¼ A little, 3 ¼ Moderately, 4 ¼ Quite a bit, 5 ¼ Extremely).

3.5.2. Subjective fatigue Twelve items referring to feelings of fatigue, sluggishness, stiffness or strain in neck or eyes (a ¼ .84–.87). An example item is “worn out.”

Students rated the degree to which each statement described how they currently felt on a 5-point Likert-type scale (1 ¼ Very slightly or not at all, 2 ¼ A little, 3 ¼ Moderately, 4 ¼ Quite a bit, 5 ¼ Extremely).

3.5.3. Homework task motivation Five items referring to motivation to perform well on homework tasks and assignments (a ¼ .77–.93). An example item is “I am pushing

myself to work hard.” Students provided a rating of the degree to which each statement described their current attitude on a 6-point Likert- type scale (1 ¼ Strongly disagree, 2 ¼ Moderately disagree, 3 ¼ Slightly disagree, 4 ¼ Slightly agree, 5 ¼ Moderately agree, 6 ¼ Strongly agree).

3.5.4. Self-efficacy Five items referring tothe confidence that the student can concentrate on his/her homework/studyactivities in the next hour (a ¼ .83–.87).

An example item is “In the next hour, how confident are you that you can concentrate on your homework/study activities. 50% of the time?” Students rated their confidence to concentrate on their homework/study activities on a 9-point Likert-type scale (0 ¼ No confidence,

Fig. 2. (a) Upper panel. V.I.O. point-of-view (POV) camera, mounted on headband; (b) Lower panel. Still screen-shot from POV camera.

C. Calderwood et al. / Computers & Education 75 (2014) 19–29 23

1 ¼ Extremely little confidence, 2 ¼ Very little confidence, 3 ¼ Somewhat confident, 4 ¼ Moderately confident, 5 ¼ Quite confident, 6 ¼ Very confident, 7 ¼ Extremely confident, 8 ¼ Certain).

4. Results

The results are presented in five sections. First, we compare codings derived from the surveillance cameras against codings derived from the POV camera and eyetracker. Second, we present a descriptive analysis of the frequency of student engagement with various sources of distraction and the duration of these distractions during the homework session. Third, we assess participant reactivity to the different forms of assessment by comparing multitasking frequency and duration across the three experimental conditions. Fourth, we present a series of repeated measures analyses targeted at testing hypothesized changes in mood and motivation across the homework period. Fifth, we provide a set of correlational analyses conducted to test hypotheses pertaining to the relationships of mood and motivational variables to multitasking frequency and duration.

4.1. Assessment of interruptions/media multitasking

Although the POV and mobile eyetracker data provided more precise information about participant gaze, separate coding of the same participant data from the surveillance cameras and the POV or eyetracker yielded results that were largely concordant, r ¼ .67, .74, .74, all p’s < .01, for the first, second, and third hour of the session, respectively. Due to this substantial overlap, subsequent reported analyses are based on information coded from the surveillance cameras1.

Fig. 3. (a) Upper panel. Tobii mobile eyetracker; (b) Lower panel. Still screen-shot from mobile eyetracker. Red circles indicate eye fixations, and lines indicate shifts in gaze. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

1 Although the decision to base the remaining analyses on the surveillance camera data is advantageous in that it provides a common method of observation when quantifying multitasking behaviors, this decision is not without limitations. Specifically, a degree of precision in estimating the duration of multitasking behaviors is lost when coding data from the surveillance cameras, in comparison to both the eyetracker and POV camera. By providing a focused indicator of participant gaze, the eyetracker allowed for the most precise estimate of the onset and cessation of individual multitasking behaviors. Because the POV camera allowed coders to detect when a participant’s general field of view shifted to and away from a source of distraction, this method also allowed for more precise estimates of multitasking duration than the surveillance cameras. However, despite the increased precision of the eyetracker and POV camera, the large correlations between data coded from the surveillance cameras and the POV or eyetracker indicate that the different observational technologies provide roughly equivalent estimates when individual multitasking behaviors are summed across an hour, consistent with the principle of aggregation (see Rushton, Brainerd, & Pressley, 1983).

C. Calderwood et al. / Computers & Education 75 (2014) 19–2924

4.2. Frequency and duration of distractions and media multitasking

Frequency and duration of distractions by category across the 3 h session are shown in Table 1. Given the large percentage of time students spent listening to music during the session, we separated out this category of distraction when analyzing the multitasking duration data. On average, the mean number of distractions students engaged with was 34.97 across the 3-h session, while the total duration spent engaged with distractions (excluding listening to music) was 25.55 min. However, as indicated by the large relative standard deviations on the frequency and duration data (24.03 and 21.58 min, respectively), these distributions are not normally distributed. Skewness and kurtosis values for distraction frequency were 1.45 and 2.13, respectively, while for total duration the skewness value was 1.54 and the kurtosis value was 2.77. Due to this evidence of non-normality, we provide values at the 25th, 50th, and 75th percentile for the frequency and duration of each category of distraction. In the aggregate, students at the 25th percentile had 42% of the total number of distractions encountered by students in the 75th percentile, and spent 26% of the total time on distractions. Distractions from cell phone use (where reading/sending text messages was the dominant activity) and computer use (non-homework Internet activities) represented the greatest frequency and duration of distractions.

For music listening duration, the mean elapsed time was 72.74 min (over 40% of the study session), with a standard deviation of 72.98 min. Fifty-nine percent (n ¼ 34) of the students had music playing in the background while studying, and 21% (n ¼ 12) of these students had music playing for over 90% of the study session. The 24 students who did not listen to music during the session had fewer overall distractions (M ¼ 25.04), compared to students who listened to music during part or all of the session (M ¼ 41.97), t (56) ¼ #2.80, p < .01, d ¼ #.77.

4.3. Method of assessment and distractions

Between-condition ANOVAs (surveillance camera only, POV, or eyetracker) were conducted for the cumulative frequency of distractions and total duration of distractions (excluding listening to music) across the 3 h of the session. No significant effects were found for method of assessment, F (2, 55) ¼ .54, n.s., and F (2, 55) ¼ .03, n.s., for frequency and duration, respectively, and no significant effects were found for the interaction between condition and hour in session. It is important to note that in the current study only the statistical power to detect a large effect (f $ .40) exceeded the conventional threshold of .80 (Cohen, 1988), 1–b ¼ .85 and 1–b ¼ .94 for frequency and duration, respectively. Therefore, while method of assessment did not have a large effect on multitasking frequency or duration, we lacked the statistical power to detect a more moderate effect of method of assessment on these behaviors. However, the data for the three conditions were combined for all remaining analyses conducted to test specific hypotheses, based on the lack of a large effect of method of assessment on multitasking behaviors and to maximize the statistical power of the tests of our hypotheses.

4.4. Mood, motivation, and distractions

4.4.1. Changes in mood and motivation across the homework session Hypothesis 1 stated that mood and task motivation would be impaired over sustained periods of homework completion. Table 2 presents

the results of repeated measures ANOVAs conducted to examine changes in mood and task motivation across the 3 h study session. An analysis of fatigue measures indicated that there was a statistically significant increase in self-reported fatigue across the 3 h of the study. In addition, both positive affect and homework task motivation for studying decreased over the session. There were no statistically significant changes in negative affect or self-efficacy for concentrating on homework across the session. This pattern of results provides partial support for Hypothesis 1, with the caveat that we did not observe statistically significant changes in negative affect or self-efficacy across the period of homework completion.

Table 1 Frequency and duration of distractions and media multitasking.

Activity Mean SD 25%ile 50%ile 75%ile

Activity frequency Cellphone (reading/sending texts, Internet) 8.53 9.66 2 5 12 Other distractions (checking backpack, snacking, etc.) 7.26 5.84 3 5 9.25 Internet (non-homework) 6.17 8.79 1 3 8 Music setup 5.26 7.45 0 1.5 10 Music Listening 4.95 15.86 0 2 3 Watching TV/Video 3.97 16.57 0 0 0 Computer e-mail 3.19 4.83 0 1 4.25 Cellphone (talking) .52 .84 0 0 1 Leaving the room (bathroom, vending) .03 .18 0 0 0

All distractions (except listening to music) 34.97 24.03 18.75 30.00 44.25

Time Spent in Activities Music Listening 72.74 72.98 .00 63.77 140.25 Internet (non-homework) 7.69 10.80 .28 2.69 9.29 Cellphone (reading/sending texts, Internet) 4.63 8.87 .36 1.78 6.32 Other distractions (checking backpack, snacking, etc.) 3.88 4.57 1.42 2.79 4.31 Computer e-mail 3.38 6.95 .00 .73 3.76 Watching TV/Video 2.97 12.55 .00 .00 .00 Music setup 1.49 2.86 .00 .12 1.85 Cellphone (talking) 1.36 2.86 .00 .00 .76 Leaving the room (bathroom, vending) .15 .85 .00 .00 .00

All Distractions (except listening to music) 25.55 21.58 9.27 19.18 35.69

Note. Distraction frequency and duration were measured across a 180 min laboratory session. %ile ¼ percentile.

C. Calderwood et al. / Computers & Education 75 (2014) 19–29 25

For exploratory purposes, we sought to analyze whether changes in mood and motivation were influenced by method of assessment. In contrast to the statistical tests of specific hypotheses, we did not combine the data from the three conditions in these exploratory analyses, in order to allow us to include method of assessment as a between-subjects factor in tests of these potential interactional relationships. These exploratory analyses were conducted via a set of repeated measures ANOVAs in which we examined the interaction between Condition and Hour in Session in predicting each of the mood and motivational variables. Only one interaction term was found to be statistically significant, with a joint effect observed for the outcome of fatigue, F (4, 110) ¼ 3.06, p < .05, f ¼ .33. The nature of the interaction was such that par- ticipants in the surveillance-only and eyetracker conditions experienced increasing fatigue across the 3-h session, while fatigue levels remained relatively constant for participants in the POV-condition. No other interaction terms were statistically significant in the prediction of mood and motivational effects.

4.4.2. Associations of mood and motivational variables with multitasking behaviors To explore the relationships of mood and motivational variables to multitasking frequency and duration, we examined the inter-

correlation among these variables when aggregated across the 3 h session. The results of these analyses are presented in Table 3. In terms of inter-relationships among indicators of affect and motivation during the study session, there was a strong positive relationship linking homework task motivation and self-efficacy, r ¼ .58, p < .01. Higher levels of positive affect were associated with lower fatigue, r ¼ #.38, p < .01, and higher homework task motivation, r ¼ .30, p < .05, while greater negative affect was linked to higher fatigue and lower homework task motivation, r ¼ .48 and r ¼ #.41, both p’s < .01, respectively.

Hypotheses 2 and 4 stated that higher NA and fatigue would be associated with greater multitasking behaviors. As can be seen in Table 3, higher levels of negative affect were associated with a longer duration of multitasking during the session, r ¼ .33, p < .05, and a longer duration of time spent listening to music during the session, r ¼ .37, p < .05. This pattern of results provides partial support for Hypothesis 2, with the caveat that negative affect was not linked to multitasking frequency. We found no evidence to link subjective fatigue to any in- dicators of multitasking behavior, providing no support for Hypothesis 4.

Hypotheses 3, 5, and 6 stated that higher PA, homework task motivation, and self-efficacy to concentrate on homework would be associated with reduced multitasking behaviors. As displayed in Table 3, higher levels of both homework task motivation and self-efficacy to concentrate on homework were associated with less frequent multitasking, r ¼ #.47 and r ¼ #.34, both p’s < .01, respectively, and shorter duration multitasking, r ¼ #.58 and r ¼ #.38, both p’s < .01, respectively. This pattern of results provides full support to Hypotheses 5 and 6. In contrast, we found no statistically significant correlations to link PA to multitasking behaviors, failing to provide support for Hypothesis 3.

5. Discussion/conclusions

In this study, we accomplished our goals of quantifying the frequency and duration with which college students engage in media multitaskingwhile completing schoolworkoutsideof the classroomand wereable to linkmultitasking behaviors to affective andmotivational

Table 3 Inter-correlations of multitasking behaviors and self-reported affective and motivational variables (aggregated across the 3 h session).

Variable M S.D. 1 2 3 4 5 6 7 8

1 Multitasking frequency 34.97 24.03 1.00

2 Multitasking duration 25.55 21.58 .74** 1.00

3 Listening to music duration 72.74 72.98 .34* .19 1.00

4 Positive affect 4.62 .64 .02 .02 .15 1.00

5 Negative affect 3.40 .39 .07 .33* .37* .08 1.00

6 Subjective fatigue 3.48 .60 .10 .21 .13 #.38** .48** 1.00

7 Homework task motivation 5.03 .73 #.47** #.58** #.22 .30* #.41** #.24 1.00

8 Self-efficacy 27.10 6.18 #.34** #.38** #.05 .17 #.18 #.19 .58** 1.00

Note. N ¼ 58. *p < .05. **p < .01.

Table 2 Changes in mood and task motivation by hour of study time.

Variable Hour 1 mean (S.D.)

Hour 2 mean (S.D.)

Hour 3 mean (S.D.)

F M.S. (error) f

Positive affect 4.81 (.65)

4.61 (.67)

4.43 (.73)

26.18** .08 .68

Negative affect 3.41 (.40)

3.37 (.40)

3.42 (.44)

1.14 .03 .14

Subjective fatigue 3.31 (.68)

3.50 (.66)

3.62 (.69)

9.04** .15 .40

Homework task motivation 5.18 (.62)

4.99 (.84)

4.92 (.87)

7.71** .14 .37

Self-efficacy 27.40 (7.42)

27.38 (6.68)

26.53 (6.74)

.92 15.37 .13

Note. N ¼ 58. d.f. ¼ 2, 114. Multitasking duration and frequency ratings refer to within-hour observer coded multitasking behaviors. Self-report ratings refer to ratings provided prior to the beginning of the indicated hour of homework completion. **p < .01.

C. Calderwood et al. / Computers & Education 75 (2014) 19–2926

processes throughout the study period. On average, students encountered 35 distractions during 3 h of independent study, and were engaged with these distractions for approximately 26 min (about 14% of the study session). A majority of students (59%) listened to music for part or all of the session, with an average amount of time engaged with this source of distraction of 73 min (over 40% of the study session). Subjective fatigue was found to increase across 3 h of independent study, while positive affect and homework task motivation decreased across this time period. Higher homework task motivation and self-efficacy for concentrating on homework were associated with less frequent and shorter duration multitasking behaviors, while higher negative affect was linked to greater multitasking duration during the study session.

The first goal of this study was to objectively quantify the frequency and duration of multitasking behaviors in college students completing schoolwork outside of the classroom. Even in a laboratory environment under observation and isolated from some potential external distractions (e.g., the physical presence of roomates), college students spent a substantial amount of time engaged in media multitasking while doing homework. This pattern of results is complementary to the findings of self-report studies of multitasking con- ducted in younger and older adolescents, in which students have reported multitasking while completing homework (Foehr, 2006) and using the Internet (Moreno et al., 2012). There also appeared to be large differences in students’ approaches to incorporating media multitasking into their study habits. For example, students at the 25th percentile of distraction duration only averaged 9 min engaged with distractions, and did not listen to music at all. In contrast, students at the 75th percentile averaged 36 min engaged with distractions, and listened to music for an average of 140 min while studying. These differences represent clear divergences in study patterns, and likely have a significant impact on homework task performance and student grades (Junco & Cotten, 2012). Accordingly, identifying those students who are likely to devote a large proportion of their independent study time to multitasking is an important topic for future researchers. In light of evidence to suggest that students may not accurately estimate the frequency with which they engage with technology (Moreno et al., 2012), the adoption of objective methods, such as those used in this study, may allow for a more accurate assessment of students at high-risk for frequent and long duration multitasking behaviors while studying.

The second goal of this investigation was to explore whether there were differential participant reactivity effects associated with the use of three increasingly intrusive methods to observe multitasking behaviors. While a number of researchers have explored the effects of multitasking on student performance (e.g., Junco & Cotten, 2012), there has been little consideration given to participant reactivity effects stemming from the self-reporting or observation of multitasking behaviors. This represents a major limitation of the multitasking literature, when considering the long-standing recognition that participant behavior may change under observation (see Adair, 1984 for a review). In the current study, we were able to conclude that there were not large differences in media multitasking as a function of the recording equipment used to monitor participant behavior. Subsequent exploratory analyses did reveal an interaction between method of assessment and hour in session when considering the outcome of fatigue, in which the trajectory of fatigue was relatively flat for participants in the POV- condition, relative to the other conditions. When considering applications for these observational technologies, the least expensive device (the POV camera) was also the most portable, which leads to the possibility that such devices could be used outside of the laboratory for future investigations in more naturalistic settings. For more nuanced investigations of attentional processes in response to and after dis- tractions, the mobile eyetracker provides a more precise depiction of the student’s eye movements during studying. Accordingly, this more expensive monitoring option may be more suitable for laboratory-based studies of micro-level processes of attentional engagement during homework completion. Finally, while it is certainly not feasible or even ethical to install sets of surveillance cameras in college students’ study environments, this methodology could usefully be implemented in university departments targeted at assisting students who are academically at-risk, as a means to gauge potential aspects of their study habits which can be improved.

The third goal of this study was to investigate whether multitasking behaviors were associated with mood and task motivational var- iables during homework completion. As a first step in this process, we explored whether sustained periods of homework were linked to changes in mood and motivation. Although it has long been recognized that time-on-task may have an impact on fatigue and task moti- vation in relation to effort expenditure (Davis,1946; Dodge,1917), changes in these processes over time had not yet been explored in relation to schoolwork outside of the classroom. In this study, we observed that subjective fatigue increased over the course of a 3 h session of homework, while positive affect and homework task motivation decreased during this time period. These results generalize past research on the impact of time-on-task in mood and well-being processes to encompass the homework environment. A greater emphasis on the alteration of subjective mood and motivation during homework completion should allow for the development of more nuanced theoretical models of changes in these processes over time.

As a second step in our effort to explore links between multitasking behaviors, mood, and task motivation, we examined the inter- correlations of these variables during homework completion. These analyses served as an initial exploration of potential reasons as to why students engage in multitasking behaviors, despite evidence that these behaviors are associated with performance decrements (see Wang & Tchernev, 2012). In the current study, more frequent and longer duration multitasking behaviors were correlated with lower homework task motivation and lower self-efficacy to concentrate on homework, while higher negative affect was linked to a greater multitasking duration across the session. The logical next step in this line of research will be to use mood or motivation induction paradigms (see Westermann, Spies, Stahl, & Hesse, 1996 for a review) to experimentally evaluate causal relationships linking mood or motivational variables to multitasking behaviors. Such studies would make a major contribution to answering the question of why students engage in multitasking behaviors while studying, and may serve as a first step towards the development of interventions targeted at reducing multitasking behaviors through mood and motivation improvement.

Although a major strength of this study was the exploration of student multitasking behaviors in a less constrained study environment than has typically been investigated in experimental research, such an approach is not without limitations. Perhaps the most significant of these limitations was an inability to assess performance on homework tasks in an objective fashion. Researchers have linked several cognitive and attentional variables to multitasking performance (e.g., König, Bühner, & Mürling, 2005). Therefore, while we did not observe statistically significant participant reactivity effects in relation to multitasking behaviors, it is possible that distractions stemming from the use of these technologies may have secondarily impacted homework performance. Future research targeted at conceptually replicating our findings using more constrained laboratory tasks would provide a useful complement to the more naturalistic experimental approach utilized in the current study.

Although allowing students to complete their own homework did serve to increase the external validity of our study design, there is still a degree of artificiality inherent to studying student behavior in a laboratory environment. While efforts were made to provide students

C. Calderwood et al. / Computers & Education 75 (2014) 19–29 27

with many of the technological devices which they typically use while studying, it was not possible to replicate certain aspects of the naturalistic study environment while maintaining the methodological controls of an experimental design. For example, several students commented that their roommate often served as a major source of distraction when they were completing homework and studying. Given that roommate-initiated distractions are more outside of a student’s control than self-initiated distractions, these external distractions may have a more substantial impact on students’ mood, motivation, and homework performance. When considering that our study design primarily focused on self-initiated distractions, it is possible that our findings under-estimate the amount of distraction students encounter when completing homework in their naturalistic study environment.

The integration of technology into student study environments has come a long way since early concerns were expressed about the completion of homework while watching television (see Maccoby, 1951). In light of the accessibility and portability of modern commu- nication and information technology devices, it is doubtful that this genie can be put back in the bottle. However, the results of this study indicate that it may be possible to use objective means to identify students who may perform poorly on their homework due to an inability or unwillingness to disconnect from interruptions when engaging with schoolwork outside of the classroom. The various methodologies tested in this study have elucidated a number of different observational tools for investigating student multitasking behaviors during homework completion. Even in a relatively constrained laboratory study environment, students were observed to frequently engage with off-task distractions during a substantial portion of their homework time. This study has provided a methodology and laid the groundwork for future investigators to use objective methods to explore processes within the homework session that influence student engagement with off-task distractions and related outcomes.

References

Ackerman, P. L., Calderwood, C., & Conklin, E. M. (2012). Task characteristics and fatigue. In G. Matthews, P. A. Desmond, C. Neubauer, & P. A. Hancock (Eds.), The handbook of operator fatigue (pp. 91–101). Farnham, Surrey, UK: Ashgate Publishing.

Ackerman, P. L., & Kanfer, R. (2009). Test length and cognitive fatigue: an empirical examination of effects on performance and test-taker reactions. Journal of Experimental Psychology: Applied, 15(2), 163–181.

Adair, J. G. (1984). The Hawthorne effect: a reconsideration of the methodological artifact. Journal of Applied Psychology, 69(2), 334–345. Aspinwall, L. G. (1998). Rethinking the role of positive affect in self-regulation. Motivation and Emotion, 22(1), 1–32. Bandura, A. (1982). Self-efficacy mechanism in human agency. American Psychologist, 37(2), 122–147. Beal, D. J., Weiss, H. M., Barros, E., & MacDermid, S. M. (2005). An episodic process model of affective influences on performance. Journal of Applied Psychology, 90(6), 1054–

1068. Brown, R., & Schutte, N. (2006). Direct and indirect relationships between emotional intelligence and subjective fatigue in university students. Journal of Psychosomatic

Research, 60, 585–593. Carver, C. (2003). Pleasure as a sign you can attend to something else: placing positive feelings within a general model of affect. Cognition and Emotion, 17(2), 241–261. Chen, C., & Stevenson, H. W. (1989). Homework: a cross-cultural examination. Child Development, 60(3), 551–561. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, New Jersey: Lawrence Erlbaum Associates. Cooper, H., Robinson, J. C., & Patall, E. A. (2006). Does homework improve academic achievement? A synthesis of research, 1987–2003. Review of Educational Research, 76(1),

1–62. Cooper, H., & Valentine, J. C. (2001). Using research to answer practical questions about homework. Educational Psychologist, 36(3), 143–153. Davis, D. R. (1946). The disorganization of behavior in fatigue. Journal of Neurology, Neurosurgery, and Psychiatry, 9(1), 23–29. Deci, E. L. (1975). Intrinsic motivation. New York: Plenum. Dodge, R. (1913). Mental work: a study in psychodynamics. Psychological Review, 20, 1–42. Dodge, R. (1917). The laws of relative fatigue. Psychological Review, 24(2), 89–113. Foehr, U. G. (2006). Media multitasking among American youth: Prevalence, predictors, and pairings. Retrieved May 18, 2009, from http://www.kff.org/entmedia/upload/7592.

pdf. Fox, A. B., Rosen, J., & Crawford, M. (2009). Distractions, distractions: does instant messaging affect college students’ performance on a concurrent reading comprehension

task? Cyberpsychology & Behavior, 12(1), 51–53. Fredrickson, B. L. (2001). The role of positive emotions in positive psychology: the broaden-and-build theory of positive emotions. American Psychologist, 56(3), 218–226. Hembrooke, H., & Gay, G. (2003). The laptop and the lecture: the effects of multitasking in learning environments. Journal of Computing in Higher Education, 15(1), 46–64. Hockey, G. R. J. (1997). Compensatory control in the regulation of human performance under stress and high workload: a cognitive – energetical framework. Biological

Psychology, 45, 73–93. Hockey, G. R. J. (2011). A motivational control theory of cognitive fatigue. In P. L. Ackerman (Ed.), Cognitive fatigue: Multidisciplinary perspectives on current research and future

applications (pp. 167–187). Washington, DC: American Psychological Association. Ie, A., Haller, C. S., Langer, E. J., & Courvoisier, D. S. (2012). Mindful multitasking: the relationship between mindful flexibility and media multitasking. Computers in Human

Behavior, 28, 1526–1532. Jacobsen, W. C., & Forste, R. (2011). The wired generation: academic and social outcomes of electronic media use among university students. Cyberpsychology, Behavior, and

Social Networking, 14(5), 275–280. Junco, R., & Cotten, S. R. (2012). No A 4 U: the relationship between multitasking and academic performance. Computers & Education, 59, 505–514. Kahneman, D. (1973). Attention and effort. Englewood Cliffs, NJ: Prentice Hall. König, C. J., Bühner, M., & Mürling, G. (2005). Working memory, fluid intelligence, and attention are predictors of multitasking performance, but polychronicity and ex-

traversion are not. Human Performance, 18(3), 243–266. Kraushaar, J. M., & Novak, D. C. (2010). Examining the effects of student multitasking with laptops during the lecture. Journal of Information Systems Education, 21(2), 241–251. Leone, C. M., & Richards, M. H. (1989). Classwork and homework in early adolescence: the ecology of achievement. Journal of Youth and Adolescence, 18(6), 531–548. Maccoby, E. E. (1951). Television: its impact on school children. Public Opinion Quarterly, 15, 421–444. McNair, D. M., Lorr, J., & Droppleman, L. F. (2003). Profile of mood states standard form. North Tonawanda, NY: Multi-Health Systems. Moreno, M. A., Jelenchick, L., Koff, R., Eikoff, J., Diermyer, C., & Christakis, D. A. (2012). Internet use and multitasking among older adolescents: an experience sampling

approach. Computers in Human Behavior, 28(4), 1097–1102. Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of Educational Research, 66(4), 543–578. Rushton, J. P., Brainerd, C. J., & Pressley, M. (1983). Behavioral development and construct validity: the principle of aggregation. Psychological Bulletin, 94(1), 18–38. Sana, F., Weston, T., & Cepeda, N. J. (2013). Laptop multitasking hinders classroom learning for both users and nearby peers. Computers & Education, 62, 24–31. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428. Thomsen, D. K. (2006). The association between rumination and negative affect: a review. Cognition and Emotion, 20(8), 1216–1235. van der Linden, D., Frese, M., & Meijman, T. F. (2003). Mental fatigue and the control of cognitive processes: effects on perseveration and planning. Acta Psychologica, 113,

45–65. Wallis, C. (2006, March 27). genM: the multitasking generation. Time, 167(13). Retrieved September 6th, 2013 from http://content.time.com/time/magazine/article/

0,9171,1174696,00.html. Wang, Z., & Tchernev, J. M. (2012). The “myth” of media multitasking: reciprocal dynamics of media multitasking, personal needs, and gratifications. Journal of Communication,

62(3), 493–513. Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social

Psychology, 54(6), 1063–1070.

C. Calderwood et al. / Computers & Education 75 (2014) 19–2928

Westermann, R., Spies, K., Stahl, G., & Hesse, F. W. (1996). Relative effectiveness and validity of mood induction procedures: a meta-analysis. European Journal of Social Psychology, 26(4), 557–580.

Whitley, B. E. (2002). Principles of research in behavioral science (2nd ed.). New York: Mcgraw-Hill. Zimmerman, B. J., Bandura, A., & Martinez-Pons, M. (1992). Self-motivation for academic attainment: the role of self-efficacy beliefs and personal goal setting. American

Educational Research Journal, 29(3), 663–676.

C. Calderwood et al. / Computers & Education 75 (2014) 19–29 29