I have 4 assignments! Due different dates.

The Role of Situational Interest in Personalized Learning Matthew L. Bernacki

University of Nevada, Las Vegas Candace Walkington

Southern Methodist University

Context personalization—the incorporation of students’ out-of-school interests into learning tasks— has recently been shown to positively affect students’ situational interest and their performance and learning in mathematics. However, few studies have shown effects on both interest and achievement, drawing into question whether context personalization interventions can achieve both ends. The effects of personalization are theorized to result from activation of students’ prior knowledge of personal interests and generation of situational interest in math tasks, though theorists have begun to question whether situational interest serves as a mechanism by which learning outcomes are achieved. This experimental study examines whether personalizing 4 units of algebra problems that high school students (N � 150) solve in an intelligent tutoring system could improve their performance in units (i.e., accuracy and learning efficiency) and on classroom exams, whether adolescents who solved personalized problems would report greater situational interest in units (and later, individual interest in math) than peers who solved standard problems, and whether paths through situational interest would contribute to effects of personalization on outcomes. High school students in the personalization condition reported greater triggered situational interest in experimental units, and triggered interest predicted in-tutor outcomes (accuracy, learning efficiency). A total effect of personalization was also observed on classroom exam performance and individual interest in mathematics. Implications for theories of interest and context personalization are discussed, as are implications for math instruction and design of personalized learning environments.

Educational Impact and Implications Statement Context personalization refers to an instructional design strategy that incorporates students’ out-of-school interests into learning tasks like math problems. Recent research has shown that personalization positively affects students’ situational interest and their performance and learning in math, but students seldom obtain both outcomes. This study confirmed that personalizing 4 units of algebra story problems to students’ out-of-school interests was sufficient to increase their situational interest in the task and to improve the efficiency with which they solved problems within the intelligent tutoring system. Months later, those who solved personalized problems also reported greater interest in mathematics and scored higher on a classroom math test than a control group. These results extend evidence for the benefits of personalization and confirm that personalizing problems to incorporate student interests at an appropriate depth and specificity can simultaneously produce effects on math interest and learning.

Keywords: personalization, interest, algebra, mathematics, intelligent tutors

The learning experience is a personal one, particularly when learn- ers use adaptive technologies. Individuals’ prior knowledge, beliefs, interests, and motivations are known to influence the way they engage

in computer-based learning tasks and the outcomes they obtain (e.g., Bernacki, Byrnes, & Cromley, 2012; Mitchell, Chen, & Macredie, 2005; Moos & Azevedo, 2009; Renninger, Hidi, & Krapp, 2014). By assessing and accommodating these individual characteristics, adap- tive technologies help individuals learn more and do so more effi- ciently (Koedinger, Corbett, & Perfetti, 2012). One example is the way that cognitive tutors account for an individual’s prior knowledge and adaptively provide problem-solving practice until a student dem- onstrates mastery (Koedinger & Corbett, 2006).

As learning tasks are adapted in light of learners’ individual dif- ferences, learning becomes individualized, and when the experience is differentiated for learners based on their interests, preferences, and experiences, it becomes personalized (U.S. Department of Education, 2010). Consumer demand for individualized learning experiences for mathematics is growing and has led to a robust market for software like cognitive tutors (Carnegie Learning, 2016) and Khan Academy (2016), which adapt instruction to students’ prior knowledge and skills. National technology plans now make specific reference to the need for adaptive software to optimize learning (U.S. Department of

This article was published Online First March 15, 2018. Matthew L. Bernacki, Department of Educational Psychology & Higher

Education, University of Nevada, Las Vegas; Candace Walkington, De- partment of Teaching and Learning, Southern Methodist University.

This project was sponsored by the Metacognition and Motivation Thrust of LearnLab, the Pittsburgh Science of Learning Center. Original support for LearnLab was provided by the National Science Foundation (SBE-0836012). We also wish to thank collaborators at Carnegie Learning (including Steve Ritter, Tristan Nixon, Steven Fancsali, and Susan Berman) for their assistance with personalizing Cognitive Tutor software and the teachers and the admin- istrators of the school district where the study was conducted.

Correspondence concerning this article should be addressed to Matthew L. Bernacki, Department of Educational Psychology & Higher Education, University of Nevada, Las Vegas, 4505 S. Maryland Parkway #3003, Las Vegas, NV 89154. E-mail: [email protected]

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

Journal of Educational Psychology © 2018 American Psychological Association 2018, Vol. 110, No. 6, 864 – 881 0022-0663/18/$12.00 http://dx.doi.org/10.1037/edu0000250

864

Education, 2010, 2016), and professional societies are recognizing the “grand challenge” that personalization poses to engineers and devel- opers (Ellis, 2008). A grand challenge facing the educational psychol- ogy community is to better understand the mechanisms by which personalization can improve learning.

Personalized learning most often adapts instruction based on learners’ prior knowledge and performance, but research continues to emerge that indicates learning improves further when tasks are personalized to both cognitive and noncognitive factors (Walking- ton & Bernacki, 2014). This approach— called context personal- ization—adapts the content of learning materials using adaptive technologies to also reflect students’ out-of-school interests.

Prior research indicates that context personalization has a positive effect on learning (Cordova & Lepper, 1996; Walkington, 2013). For example, Walkington (2013) found that students who solved prob- lems personalized to reflect their interest (e.g., in sports or food) performed better than students who solved matched standard prob- lems. In addition to these immediate effects on performance, the personalized group mastered skills more quickly than the control group, and continued to perform better in a future math unit that was not personalized to student interests. However, a key weakness of this study was that no measures were collected to examine why this effect might have occurred. Theoretical work suggests two factors might be important in explaining this effect—the potential of personalization to provide contextual grounding by drawing upon students’ prior knowl- edge of their interest areas (Goldstone & Son, 2005), and the potential of personalization to trigger students’ interest (Hidi & Renninger, 2006).

In this article, we describe a study that explores the second assumption of context personalization theory: that personalization promotes interest in the learning task and that this greater interest contributes to effects on learning. Although prior research has established effects of personalization on interest or learning, no studies to date have examined how motivational variables like developing interest contribute to performance and learning effects over time as students continuously interact with a system for personalization. We ground this line of inquiry in interest theory, summarize research on the way interest affects learning, and provide an overview of context personalization theory and re- search.

Theories of Interest and the Effects of Interest on Learning

Interest has been defined as being both the state of engaging and the predisposition to reengage with particular activities, events, and ideas over time (Hidi & Renninger, 2006). Theorists have defined two types of interest. Situational interest is a state of heightened attention and increased engagement elicited by elements of an environment that are surprising, salient, evocative, or personally relevant. Situational inter- est can be triggered in response to stimuli, and becomes maintained over time as a learner (re)engages further with the stimuli. Situational interest can also be subdivided into interest based on enjoyment of the activity and interest based on valuing of the activity with respect to other things the learner values. Value-based situational interest has also been referred to as utility value—a learner’s awareness of the usefulness of a topic to their life and goals (Eccles, 1983). Individual interest is an enduring preference for certain objects or activities that persists over time and involves knowledge, value, and enjoyment;

individual interest can be emerging or well-developed (Hidi & Ren- ninger, 2006).

Hidi and Renninger (2006, p. 112) asserted that “[e]ach phase of interest is characterized by varying amounts of affect, knowledge, and value.” Further, the maintenance of situational interest is influenced by the “meaningfulness of tasks specifically” (p. 114), and as situa- tional interest in a task is sustained during this second maintenance phase; the individual “develop[s] value for content” within the task (p. 119). This development of value leads to the reengagement charac- teristic of an emerging individual interest. Based on this conceptual- ization, Linnenbrink-Garcia and colleagues (2010) developed a mea- sure to capture triggered situational interest as well as separate measures of maintained situational interest due to feelings of enjoyment and perceptions of value. Their measurement ap- proach did not differentiate between emerging and well- developed individual interest.

Research has shown that when students are interested in a task or activities, they engage longer and with more effort and produc- tive learning behaviors including generative questioning, self- regulation, deep strategy use, and problem-solving (Lipstein & Renninger, 2006; Mitchell, 1993; Renninger & Hidi, 2002; Ren- ninger & Shumar, 2002). Students who express greater interest also achieve improved learning outcomes (e.g., Harackiewicz, Durik, Barron, Linnenbrink-Garcia, & Tauer, 2008). An important question, then, is how to elicit and develop learners’ interests for academic content areas. One potential method is context person- alization.

Context Personalization

Context personalization (Cordova & Lepper, 1996; Walkington & Bernacki, 2014; “personalization” hereafter) is an intervention ap- proach that can be used to integrate students’ out-of-school interests into learning. Context personalization is thought to achieve its effects through cognitive and motivational mechanisms (Walkington & Ber- nacki, 2014, in press). Students’ out-of-school interests in topics like sports or music are considered individual interests, and students tend to have significant prior knowledge of, and feelings of value and enjoyment for, their individual interests (Renninger & Su, 2012). Personalization is the practice of using students’ individual interests in topic areas outside of academic tasks to facilitate improved outcomes within these tasks. Personalization may facilitate learning by ground- ing tasks in familiar contexts, and may increase students’ interest in tasks (see Walkington & Bernacki, 2014).

Incorporating a student’s out-of-school interests into a mathematics problem is thought to activate the student’s prior knowledge by grounding the problem in a context that is familiar (Goldstone & Son, 2005). Contextual grounding has a variety of purported cognitive benefits for learners. Grounded representations can be easier to access in long-term memory due to familiarity; they can be less prone to conceptual error as real-world knowledge can provide insight into reasonableness of results; and including concrete details can illumi- nate domain principles and make them more understandable (Gold- stone & Son, 2005; Koedinger, Alibali, & Nathan, 2008).

In mathematics, one important way to provide contextual ground- ing is to connect learning materials to students’ “funds of knowledge” relating to their quantitative experiences outside of school (Civil, 2007). That is, students come to school with contextual knowledge from their homes and communities that have been historically accu-

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

865SITUATIONAL INTEREST AND PERSONALIZED LEARNING

mulated and culturally developed (Moll, Amanti, Neff, & Gonzalez, 1992; Civil, 2007). They experience numbers, quantities, and change as they engage in home- and community-based activities like playing video games, cooking, and shopping (see Walkington, Sherman, & Howell, 2014). These quantitative resources can be brought into the classroom through thoughtfully designed learning experiences in- tended to make connections between home or community knowledge bases and academic mathematics. Although making such connections between these disparate contexts can be fraught with difficulty (e.g., Nunes, Schliemann, & Carraher, 1993), there are examples of suc- cessful interventions for integrating funds of knowledge into the classroom (e.g., Barton & Tan, 2009; Walkington, in press).

In addition to providing grounding, personalization is thought to increase students’ interest in the learning task. Personalized contexts may trigger students’ situational interest (Hidi & Renninger, 2006), as students notice their out-of-school interests being incorporated into mathematics problems. This triggering of situational interest may occur through the novelty, surprise, incongruity, evocativeness, sa- lience, or personal relevance that seeing personalized connections may elicit (Hidi & Anderson, 1992; Hidi & Renninger, 2006; Schraw, Flowerday, & Lehman, 2001). This situational interest may become maintained as students enjoy seeing the personalized connections, or because the math problems connect to out-of-school topics they value—two dimensions of maintained situational interest (Hidi & Renninger, 2006; Linnenbrink-Garcia et al., 2010). These feelings of enjoyment and value may proceed to transfer over to the math tasks themselves—students may begin to enjoy solving the math tasks and find value for the math tasks as they see how mathematics relates to their lives and has utility (see Walkington & Bernacki, 2014). Over time, situational interest in the personalized math tasks may transform into individual interest in the mathematical domain itself (Hidi & Renninger, 2006). Thus, the act of personalizing a math problem can be seen as a triggering event for students’ interest, that has the potential to—if repeated and maintained— have significant down- stream consequences for students’ interest and learning. These theo- retical assumptions possess ample face validity, but evidence to val- idate such assumptions is not yet available.

Prior research on context personalization has been mixed and has not focused on the motivational mechanisms by which personalization can achieve effects on learning. In addition, most prior work has examined relatively short, usually “one- shot,” interventions conducted in laboratories. This is problem- atic given that research on interest suggests it develops over time. There are a number of recent studies that have found no effect for personalization on performance or learning (Bates & Wiest, 2004; Cakir & Simsek, 2010; Høgheim & Reber, 2015; Ku & Sullivan, 2000; Simsek & Cakir, 2009), and one recent study that actually suggested a slight negative effect (Fancsali & Ritter, 2014). The studies that have found gains for learning have not examined motivational mechanisms as potential con- tributors, either omitting these measures entirely (Walkington, 2013), or simply showing pre-/postdifferences (Cordova & Lep- per, 1996; Ku & Sullivan, 2000; López & Sullivan, 1992). The present investigation is the first to offer a glimpse of how motivational variables develop over time in real classrooms during an extended personalization intervention, and how these mechanisms contribute to immediate performance and long- term learning effects.

Utility Value Interventions

One area of theoretically grounded research on improving learn- ing through motivational mechanisms is the enhancement of utility value by directly communicating to students that content they are learning is relevant to their lives, or asking them to self-generate the reasons for relevance. Directly communicating utility value to learners (e.g., “This strategy is useful because . . .”) has shown mixed results. This information seems beneficial for students with high individual interest in math (Durik & Harackiewicz, 2007; Durik, Shechter, Noh, Rozek, & Harackiewicz, 2015) or high perceived competence in math (Durik et al., 2015; Canning & Harackiewicz, 2015), but it is potentially detrimental for students with low individual interest in math (Durik & Harackiewicz, 2007) or low perceived competence in math (Durik et al., 2015). On the other hand, having learners themselves generate how course con- tent is related to their lives (by, for instance, writing an essay about this relationship), has been found to be beneficial, particularly for students with low expectations of success or low academic per- formance (Hulleman & Harackiewicz, 2009; Hulleman et al., 2010; Hulleman, Kosovich, Barron, & Daniel, 2017), as well as for first-generation college students (Harackiewicz, Canning, Tibbetts, Priniski, & Hyde, 2016). These interventions are described as activating “personal relevance,” as students are making connec- tions between themselves and the content they are learning. Hav- ing students evaluate interview quotes describing the usefulness of math has also been found to be an effective approach for enhanc- ing utility value and value beliefs (Gaspard et al., 2015).

Like utility value interventions, context personalization is also theorized to promote perceptions of value. However, the context personalization approach is distinct from utility value interventions in both the number of theorized mechanisms by which it achieves its effects, and the relationship between the intervention and class- room learning activities. Beyond promoting perceptions of value, personalization also enhances the feelings of enjoyment experi- enced by students who solve problems that incorporate the inter- ests they pursue outside of school. Unlike cocurricular utility value interventions, context personalization is embedded in learning: solving personalized problems activates these value and enjoyment mechanisms within a task that is already part of typical classroom instruction.

The personalization intervention we report here involved chang- ing the context of problems to match students’ interests. Thus, we did not explicitly communicate utility value information, nor did we ask learners to self-generate utility value information. The intervention we report may have subtle or implicit effects on utility value or perceptions of relevance, contingent upon the partici- pant’s belief that the interest-based connection in the problem is demonstrating a practically useful application of mathematics. Enhancing utility value was not a central focus of this intervention. However, other approaches to personalization, such as having students pose personalized problems based on their interests (see Walkington & Bernacki, 2015), more explicitly prompt students to connect math to their interests and elicit perceptions of the utility of mathematics.

Dimensions of Context Personalization

To date, theory has focused on three design dimensions of personalized content (Walkington & Bernacki, 2014, in press), and

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

866 BERNACKI AND WALKINGTON

these dimensions have been used to explain the generally mixed research findings for personalization mentioned earlier (Walking- ton & Bernacki, 2014). First is the depth of the intervention: whether the personalization draws upon surface level aspects of a learners’ interest—simply inserting familiar objects or names into an already-designed task (e.g., inserting the name of singer Taylor Swift into a problem that is about albums sold per week)— or whether the personalization involves deep, authentic connections to how students might informally use mathematics within actual experiences the learner has pursuing an interest like music (e.g., problems involving beats per second). Such deep connections may be important for fostering utility value for students who play or write music.

Second is the grain size of the intervention—whether the person- alization is targeted to the specific experiences of an individual, or to the more generic experiences of a group of students who report a general interest in a topic. For example, a finely grained personalized set of problems to a student interested in cycling would consistently provide the cycling enthusiast story problems that involve only cy- cling. In contrast, a cycling enthusiast assigned to a coarser grained personalization condition that provides problems about sports might only receive a few cycling problems among others involving football, baseball, basketball and other sports.

We could further vary the depth of this problem for this cyclist in a way of grounding the problem to the student’s prior knowl- edge about biking. A low depth problem might include cycling as a surface feature of an algebra problem (e.g., selling bike parts like brake pads at a rate of $5 per pair). A higher-depth problem would include a relationship that is authentic to cycling, such as mph pedaled when making a climb during a race stage.

Third is the ownership of the personalization—whether the students themselves take a role in generating the connections between the academic content area and their interests, or if teach- ers or curriculum developers control the personalization. Indeed, similar to the self-generated utility value work, recent research explores the impact of prompting students to author their own personalized problems (Walkington & Bernacki, 2015).

Høgheim and Reber (2015) also differentiate between person- alized tasks that draw upon learners’ interest areas (by differenti- ating instruction based on popular culture interests like music, gaming, or literature) versus their preferences (using familiar ob- jects from everyday situations students engage in). Using an ap- proach that drew upon students’ interests (but not their prefer- ences), they found an effect of personalization on situational interest, but not on performance. They make the case through these results that the performance effects of personalization may not come from situational interest at all, but may instead stem from contextual grounding, which is better evoked via connections to preferences. Similar results for a high-interest low-preference per- sonalization intervention were observed in Walkington, Cooper, Alibali, and Nathan (2015). Thus, the relationship between context personalization and situational interest is an area of controversy.

The Present Study

To deepen our understanding of the way personalization influences the learning process and improves performance, the present study examined effects on interest alongside learning and performance over time. We personalized the math problems that high school students

solved using Cognitive Tutor Algebra software (Carnegie Learning, 2016) and used an embedded, task-specific approach to assess stu- dents’ interest before, after, and periodically during learning (Ber- nacki, Nokes-Malach, & Aleven, 2013). Such a longitudinal approach has not been taken in any prior research and allowed for an exami- nation of the development and maintenance of interest and its long- term effects. Finally, we examined theoretical assumptions about the role of interest by proposing a model where personalization activates the interest development process (Hidi & Renninger, 2006) and ob- served how these different elements of interest influenced learning, performance, and long-term interest in math. Four research questions guided this investigation:

1. What is the immediate impact of a personalization inter- vention on students’ situational interest in algebra in- structional units?

2. What long-term effect does personalization have on stu- dents’ individual interest in algebra?

3. What is the impact of personalization on students’ per- formance and learning of algebra concepts?

4. How does the elicitation of students’ interest relate to performance and learning outcomes?

We addressed these research questions via a path analysis ap- proach and employed a random intercepts only latent growth model to capture the levels of situational interest triggered and maintained by personalization of math problems across four units (Byrne, 2013; Figure 1).

Method

Participants

Total participants included N � 155 ninth Grade Algebra I students in multiple classes1 taught by two different teachers.2

Students attended a suburban/rural Northeastern school that was 96% Caucasian with 21% of students eligible for free/reduced

1 Students from classes covering the whole Algebra I curriculum within the school year (n � 77) and those covering half the curriculum (including linear equations; n � 73) were included in the study. Across courses with different extents of coverage of Algebra I topics, preliminary analyses confirm that students in different course formats did not differ in their individual interest for math, prior achievement, and situational interest in the first unit, ts � 0.99, ps � .05. An inspection of correlation coefficients and a set of Fisher’s R to z comparisons confirmed that the association between variables did not differ between groups. Both the personalized and control conditions were composed of approximately 50% whole curricu- lum and 50% half curriculum students, with no significant differences between conditions, �2 � .004, p � .95.

2 Teacher 1 taught both whole curriculum (n � 48) and half curriculum classes (n � 76); Teacher 2 taught whole curriculum only (n � 28). Preliminary analyses confirmed that students taught by different teachers did not differ in their individual interest for math, prior achievement, or situational interest in the first unit, ts � 1.60, ps � .05. An inspection of correlation coefficients and a set of Fisher’s R to z comparisons confirmed that the association between variables did not differ between groups. In addition, the proportion of students taught by each teacher did not differ in their assignment to condition (personalized or control), �2 � .280, p � .60.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

867SITUATIONAL INTEREST AND PERSONALIZED LEARNING

price lunch. In 2012, 71% of students at the school passed the state standardized test in mathematics, which is administered in the 11th grade. The sample was 51% female, with gender data missing for 13 students. Of the original sample, five students were removed because their data from the intelligent tutoring system logs could not be matched to the data provided by classroom teachers and administrators. Within the final sample of 150 students, the num- ber of units completed varied as a function of the self-paced nature of learning with intelligent tutoring systems. This effectively var- ied the dosage of experimental units completed where seven stu- dents completed one unit, 13 completed two, 24 completed three, and the remainder completed all four. No differences in dosage were found between the Personalization and Control groups,3

t(148) � 0.59, p � .05.

Materials

Students completed computer-based math units that included surveys of students’ out-of-school interests, situational interest for math tasks, and their individual interest in math. Students also completed paper-based post- Algebra tests in the classroom. An overview of the survey measures and units completed by partici- pants in this study is provided in Figure 2.

Cognitive Tutor Algebra. The school at which the study took place used the Cognitive Tutor Algebra curriculum (Carnegie Learning, 2016). Cognitive Tutor Algebra is an intelligent tutoring system for Algebra I that uses model-tracing approaches to relate the students’ actions back to the domain model to provide indi- vidualized error feedback. Cognitive Tutor Algebra also uses knowledge-tracing approaches to track learning from one problem to the next, using this information to examine strengths and weak- ness in terms of production rules. Cognitive Tutor Algebra pres- ents learners with algebra story problems where they must navi- gate tabular, graphical, and symbolic representations of functions (see Figure 3). Students in schools that use Cognitive Tutor Alge- bra typically use the software 2 days per week. Prior to the study,

3 As might be expected when students progress through a self-paced curriculum, the amount of units a student completed in the intervention was associated with their prior math achievement (r[150] � .358, p � .05), and their individual interest in mathematics (r[150] � .232, p � .05). To further ensure that the associations between dosage and these variables— which are accounted for in our structural models— did not differ, we employed a fisher’s R to z analysis and confirmed that associations with dosage were not significantly different across the two groups, Fisher’s R to zs � .69, ps �.49.

Figure 1. Hypothesized model of personalization and effects on situational interest in mathematics, individual interest in math, and math performance. Model displays random intercepts and omits slopes fixed to 0.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

868 BERNACKI AND WALKINGTON

members of the research team adapted a set of problems in four Cognitive Tutor Algebra units on linear expressions to reflect students’ interests (Walkington, Sherman, & Howell, 2014). In these problems, a story is first given so that learners can practice writing and solving linear equations. Standard versions of stories such as “A machine called the Crawler which moves space shuttles travels at the rate of 2.9 feet per second. The Crawler is currently 100 feet from the hanger moving toward the launching pad.” were personalized to students’ out-of-school interests in topics like games: “A racing game has a train which weaves through tracks and tunnels and travels at a rate of 2.9 feet per second. The train is currently 100 feet from the start of the course and moving toward the finish line.” As can be seen in this example, the structure of the story and relationships are held constant to pre- serve the integrity of the problem-solving task, but this modifica- tion allows us to compare tasks involving personalized and stan- dard problems. Although in this example the units (feet and seconds) remained the same in each version, sometimes the units would need to be changed to match the interest category. An example is the problem, “Rainforests are being cut down at the rate of seven square miles per hour,” which had a personalized version, “Carl posts on Facebook at a rate of seven times per hour.” Readability factors like Flesh-Kincaid readability, concreteness, imageability, word count, number of sentences, sentence structure, punctuation, and mathematical vocabulary (e.g., “profit,” “around- the-clock”) were held constant across all problem versions. The linguistic structure of the problems was closely matched in this way to isolate the effect of the interest-based connection, rather than conflating personalization with factors like elaboration, clar- ity, easability, or coherence.

Personalization was implemented in Units 1, 3, 7, and 9 of Cognitive Tutor Algebra, as these were the first four units in the

software that were composed of story problems (rather than com- pletely abstract algebraic equation-solving). As can be seen from Figure 3, students would be given a story context that described a linear function that may or may not have an intercept term. They would be asked to fill in the independent and dependent quantities and units in the story, write a general algebraic expression, and then solve the expression for particular known x and y values. We restrict our analysis here to the step of writing of the algebraic equation, as students are interpreting and working directly with the story context in this portion of the problem, rather than simply performing calculations from a known formula.

Individual and situational interest surveys. Prior to entering Unit 1 (i.e., preintervention) in Cognitive Tutor Algebra, the software presented students with a survey asking them to rate their individual interest in mathematics (Linnenbrink-Garcia et al., 2010). The scale is composed of eight Likert scaled items, ranging from 1 (strongly disagree) to 5 (strongly agree) such as “Thinking mathematically is an important part of who I am.” With this sample, Cronbach’s alpha � .93.

After each unit impacted by the personalization intervention (Figure 2; Units 1, 3, 7, and 9), participants were also given a unit-level survey that assessed the degree to which problems in that unit triggered and maintained their situational interest. Each scale was adapted based on measures from Linnenbrink-Garcia et al. (2010) with the unit as the referent. Scales were composed of four Likert scaled items, ranging from 1 (strongly disagree) to 5 (strongly agree). The triggered interest scale (� � .84, sample item “The topics in this unit grabbed my attention”) and maintained situational interest scales (value, � � .90, sample item “The math in this unit is useful for me to know;” and enjoyment, � � .84, “In this unit, I really enjoyed the math”) were also found to possess sufficient internal reliability.

Manipulation check. Because the first activity students com- plete within Cognitive Tutor Algebra is to identify their out-of- school interests, we anticipated that students would recognize when problems were personalized to reflect these interests (which we assumed would make them more relevant to the student). In each unit where personalized problems could be presented, we thus added the single item, “The problems in this unit were relevant to my out-of-school interests”—responses ranged from 1(not at all true) to 5 (very true) on a 5-point Likert scale.

Prior math achievement. Students’ grade in their prior math course was used as a measure of their prior achievement. Grades were normalized and included in analyses as a correlate of indi- vidual interest and as a predictor of other performance measures.

Paper-based assessment. A paper-based, classroom math posttest was administered to all students around February of their year spent taking Algebra I (i.e., 4 to 5 months later). At this juncture, many of students had made it through the 4 experimental units.4 Using monthly updates from the software, we waited to give the exam until most students whose pace suggested they would progress through the units had progressed through them. The test contained fourstory problems where a linear function was

4 As of the administration of the classroom exam, 139 students (93%) had completed two of the four experimental units, and 110 (73%) had completed all units. Unit completion rates did not differ between groups (�M � 1.7%).

Figure 2. Study design.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

869SITUATIONAL INTEREST AND PERSONALIZED LEARNING

described that either had a slope and intercept (two problems) or had only a slope (two problems). Participants first were given an x-value in the linear function and asked to solve for y, then they were given a y-value in the linear function and asked to solve for x. Finally, they were asked to write the linear function using algebra symbols. Students’ responses to each part of each problem were scored as correct or incorrect.

Procedures

Before entering the first unit in Cognitive Tutor Algebra (Unit 1), all participants were given a survey where they would rate their level of interest in 10 topic areas—music, art, cell phones, food, computers, games, stores, TV, movies, and sports. These 10 topics were chosen based on open-ended surveys (n � 105) and interviews (n � 52) with prior classes of Algebra I students at the school site where students discussed topics of interest (see Walkington, Sherman, & Howell, 2014, for a more in-depth discussion of the surveys and interviews). Participants were then randomly assigned to one of two conditions: (a) a control condition (n � 51) that received the standard algebra story problems, or (b) a personalized condition (n � 99) that received versions of these same problems with the same underlying mathemat- ical structure that were matched to the interests they indicated on the interest survey. These problems were written based on interests past students had discussed during the prior round of open-ended surveys and interviews. Twice as many students were assigned to the person- alized condition in order to create a within-personalized-condition sample of sufficient size that the effect of features of personalization could be examined in one unit (Unit 9). These analyses fall outside the scope of this study (see Walkington & Bernacki, 2018). For each original story problem in Cognitive Tutor Algebra Units 1, 3, and 7, four personalized variations were written to correspond to four of the 10 interest areas (e.g., sports, food, games, shopping). In Unit 9, six different versions of each of the original problems were written, with two versions related to sports, two related to video games, and two related to food. This unit was different because we were testing two slightly different approaches to personalized learning; for a discus- sion, see Walkington and Bernacki (2018).

It was determined that four versions of each problem was an appropriate number based on prior research suggesting that most

students tend to express interest in many of the 10 interest areas, rather than just a few (see Walkington, Sherman, & Howell, 2014). For example, in the present study, there were only four of 150 participants who did not express interest (by rating a 3 or 4) in at least four of our 10 topics. Thus, by having four versions of each problem, there was a very high probability that participants would have at least one problem version available that matched their interest. Ultimately, on each problem opportunity within the personalized units, the tutor selected the problem version (from the four available) that best re- flected a positively endorsed out-of-school interest for each student in the personalized condition. For example, a student who endorsed a stronger interest in sports compared to video games, and stronger interest in video games compared to food or shopping would be presented with a sports version of a problem when one was available for that problem type. When the numbers or relations in the problem precluded a sports version (but allowed for video game, food, and shopping versions), the student would receive a video game version of the problem based on this being their next highest rated out-of-school interest, and their most strongly endorsed topic among topics in problem versions available.

Returning to our design features for personalization, it is important to note that all personalized versions of problems had a similar level of ownership (given that they were selected via survey) and grain size (given that they were each written to match one of 10 discrete, semibroad interest categories). The depth of the problems in the intervention, however, was variable; some were written to correspond to students’ actual quantitative experiences more closely than others. An exploration of how differences in depth of personalization impact student outcomes is beyond the scope of this study; however, we examine this issue separately (Walkington & Bernacki, 2018).

Analyses

Preliminary analyses. To use data from the entire sample of 150 who completed intelligent tutoring system units, we used a robust maximum likelihood estimation approach using Mplus 7.11 software (Muthén & Muthén, 2010). A hypothesized model testing all research questions appears as Figure 1. The model includes initial level of individual interest in mathematics and prior math

Figure 3. Screenshot of Cognitive Tutor Algebra environment. See the online article for the color version of this figure.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

870 BERNACKI AND WALKINGTON

achievement, as these are known correlates of situational interest and performance in intelligent tutoring system units, respectively.

Main analyses. Because situational interest was sampled in four units students completed on a longitudinal basis, and because we hypothesized that personalization would have a similar effect across all time points, we model the effect of personalization on situational interested using a random-intercepts-only model. This is a latent growth curve model that fixes slope to zero and the covariance of intercept and slope to zero. This allows us to constrain the slope (which we hypothesize not to vary significantly across personalized units) so we might focus in on the typical level of situational interest induced by personalization (1; compared to those with the categorical value of 0 for control) depicted as differences in intercepts across the four units, while also capturing random variance around that mean on a unit by unit basis. Effects of personalization on levels of situational interest (i.e., Research Question 1) can be observed by parameters originating at Person- alization and the total effects observed on situational interest variables. Additional paths tested direct and indirect predictive effects of personalization on individual interest (i.e., Research Question 2) and students’ accuracy and learning efficiency when solving algebra problems within an intelligent tutoring system and performance on a classroom test (i.e., Research Question 3), as well as paths examining how elicitation of students’ interest influ- enced learning outcomes (Research Question 4). This estimation of direct and indirect paths enables us to examine the extent that situational interest triggered by personalization contributed to the total effects observed on an outcome variable (i.e., indirect effect), and how much of the total effect cannot be attributed to indirect relationship involving paths through interest variables.

Results

Preliminary Analyses

Behavioral data were extracted from log-files generated by the Cognitive Tutor Algebra software. Self-reports of interest were collected from embedded questionnaires. These data were aligned to performance data from paper tests administered by the class- room instructors and achievement data obtained from the district. Means and standard deviations for all variables involved in infer- ential analyses and a correlation matrix containing these variables appears in Table 1.

Manipulation check: Relevance item. In each unit, data were collected to determine whether or not students were aware that problems were being selected for them based on their indication of their out-of-school interests (i.e., “The problems in this unit were relevant to my out-of-school interests”). Students’ mean ratings for this item across the four units, on a Likert scale ranging from 1 (not at all true) to 5 (very true) revealed no significant differences between students who received personalized problems and stu- dents who received typical problems, t(148) � �0.179, p � .86 (MPersonalized � 2.70, SDPersonalized � 1.32; MControl � 2.74, SDControl � 1.28; d � .03). Both groups’ means reflect that students generally found it to be “somewhat true” that problems were relevant to their out-of-school interests, but also that person- alized problems were not perceived to be more relevant than standard Cognitive Tutor Algebra problems. We return to this perception in the discussion section.

Confirmatory factor analyses. The four-phase model of in- terest development (Hidi & Renninger, 2006) specifies two phases of situational interest: triggering and maintaining of interest in a task.

Linnenbrink-Garcia and colleagues’ (2010) scale measures trig- gered situational interest and two different dimensions of main- tained situational interest: situational interest due to feelings of enjoyment and situational interest due to perceptions of value. However, given the strong correlation found in this study between triggered situational interest and maintained situational interest due to enjoyment (r � .91), we conducted confirmatory factor analysis to test whether a two- or three-factor model best fit the data.

Fit statistics for two separate two-factor and one three-factor solutions appear in Table 2. Based on a key fit index and the information criteria for the models (i.e., on comparative fit index [CFI] across all four units and on each of 12 comparisons of unit-specific Akaike information criteria, Bayesian information criteria, and sample-size-adjusted Bayesian information criteria used for direct comparison), the three-factor solution that separates situational interest into triggered, maintained due to feelings of enjoyment and maintained due to perceptions of value obtained superior fit for these data in each of four units compared to two-factor models that combined maintained interest items into a single latent variable or triggered interest and maintained interest due to feelings of enjoyment into a latent variable. Because this solution possessed superior fit and preserved the original concep-

Table 1 Means, Standard Deviations, and Bivariate Correlations Among Variables

Variable M SD 1 2 3 4 5 6 7 8

1. Prior math achievement (grades, normalized) .00 .99 — 2. Individual interest in mathematics (pretest) 2.91 .97 .30� — 3. Triggered situational interest 2.62 1.07 .15 .48� — 4. Maintained situational interest—enjoyment 2.63 1.07 .17� .56� .91� — 5. Maintained situational interest—value 2.89 1.13 .23� .54� .66� .74� — 6. Percent correct on first attempts in units .63 .15 .15 .17� .31� .22� .21� — 7. Corrects per minute in units 2.51 1.23 .07 .19� .32� .25� .15 .56� — 8. Score on classroom math posttest .83 .18 .13 .21� .26� .23� .11� .37� .35� — 9. Individual interest in mathematics (posttest) 2.99 1.22 .30� .71� .65� .75� .77� .28� .17� .17�

� p � .05.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

871SITUATIONAL INTEREST AND PERSONALIZED LEARNING

tualization of the measures of situational interest employed in this study (i.e., Linnenbrink-Garcia et al., 2010), and because the further articulation of situational interest into enjoyment and value components enables a more precise modeling of the features of maintained situational interest in models, we modeled using the three-factor solution. Solutions for each unit appear in Figure 4.

Measurement invariance. Having confirmed the superior fit of a three-factor model of situational interest, longitudinal invari- ance analyses were conducted to establish measurement invariance over the four units in which situational interest was observed. Maximum likelihood estimation was used for all analyses; accord- ingly, nested model comparisons were conducted using the differ- ence in the model �2 values as a function of the difference in model degrees of freedom. Two invariance models were specified in which observations of a type of situational interest (i.e., trig- gered or maintained) were each observed across four units (i.e., Personalized Unit 1, 2, 3, and 4) estimated simultaneously and clustered by student. The first indicator’s loading was fixed to 1 and its intercept was fixed to 0 for each factor to identify the model; all factor variances, covariances, and means were then estimated. Residual covariances between the same indicators across occasions were estimated as well.

Configural-, metric-, and scalar-invariance models were examined using chi-square and a recommended complementary set of fit indices (i.e., CFI, root mean square error of approximation [RMSEA], standardized root square mean residual; Vandenberg & Lance, 2000). Model statistics appear in Table 3 and indicate acceptable fit on most metrics for the configural model (Hu & Bentler, 1999; Chen, Curran, Bollen, Kirby, & Paxton, 2008), especially given the limited sample size (which can induce higher but still accept- able levels of RMSEA; Kenny, Kaniskan, & McCoach, 2015). Further, the additional constraints placed by the metric model did not induce significantly worse fit compared to the configural model, and additional constraints imposed in the scalar model also failed to induce significantly worse fit. We thus concluded strong invariance of this model of situational interest and moved to test our hypothesized model observing personalization and situational

interest and their influences on individual interest and performance and learning in math.

Potential interaction with individual interest in math. Prior research suggested that one interaction term might be particularly important when considering the effect of personalization: the in- teraction between personalization and individual interest in math- ematics. Høgheim and Reber (2015) found that although person- alization had a significant positive effect on situational interest, utility value, and task effort, this effect faded as participants’ individual interest in mathematics increased. In addition, Durik and Harackiewicz (2007) found that adding decorative elements to math learning materials was beneficial for students with low individual interest in mathematics, while an intervention relating the value of learning math to everyday life was beneficial for high individual interest students. In our preliminary analyses, we ex- amined whether such an interaction term was present in our data. It was not, so we do not consider it further.

Main Analyses

Research Question 1: Effects of personalization on situa- tional interest in algebra units. To examine the effect of per- sonalization on situational interest in math units, we assessed students’ interest immediately after each unit. Parameter estimates in the solution depicted in Figure 5 and Tables 4 and 5 can be used to address all four research questions. Accounting for initial indi- vidual interest in mathematics, personalization was found to have a significant impact on the latent mean level of triggered situa- tional interest across the four units, � .169, p � .025. There were also indirect effects of personalization on maintained situa- tional interest related to enjoyment ( � .145, p � .026) and value ( � .098, p � .032).

Research Question 2: Personalization and the development of interest in mathematics. The effects of personalization on the development of individual interest can be observed in the same model by totaling effects from personalization through measures of situational interest that precede individual interest in Hidi and

Table 2 Model Fit Comparison Across Four Units of Situational Interest Measures

Model and unit �2 df p CFI RMSEA 90% CI SRMR AIC BIC SABIC

One triggered, one maintained situational interest factor 1 251.37 53 .000 .78 .19 [.17, .22] .11 3211.18 3307.94 3191.08 2 365.42 53 .000 .80 .21 [.19, .23] .10 4200.46 4307.68 4190.64 3 341.30 53 .000 .79 .23 [.21, .25] .09 3175.27 3273.47 3156.58 4 267.00 53 .000 .84 .21 [.18, .23] .08 2734.94 2829.61 2712.98

One triggered Maintained situational interest (enjoyment) factor, one maintained situational (value) factor

1 104.13 53 .000 .94 .10 [.07, .13] .06 3063.94 3160.70 3043.84 2 167.58 53 .000 .93 .13 [.11, .15] .06 4002.63 4109.85 3992.81 3 206.99 53 .000 .89 .17 [.14, .19] .07 3040.96 3139.16 3022.27 4 18.64 53 .000 .91 .16 [.13, .18] .06 2648.57 2743.45 2626.62

One triggered, two maintained situational interest factors 1 97.10 51 .000 .95 .10 [.07, .12] .06 3060.91 3162.90 3039.72 2 16.75 51 .000 .94 .13 [.11, .15] .06 3999.79 4112.80 3989.44 3 17.68 51 .000 .91 .15 [.13, .18] .07 3008.65 3112.16 2988.95 4 145.60 51 .000 .93 .14 [.11, .17] .06 2617.52 2717.53 2594.39

Note. CFI � comparative fit index; RMSEA � root mean square error of approximation; CI � confidence interval; SRMR � standardized root mean square residual; AIC � Akaike information criteria; BIC � Bayesian information criteria; SABIC � sample-size-adjusted Bayesian Information Criterion.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

872 BERNACKI AND WALKINGTON

Renninger’s (2006) interest development model. Results indicate support for each link in the model. Positive predictive relationships were observed from triggered situational interest to maintained situational interest (enjoyment: � .858, p � .001; value: � .579, p � .001) and from maintained situational interest to indi- vidual interest (enjoyment: � .273, p � .01; value: � .426, p � .001), controlling for initial individual interest ( � .300, p � .001). The significant path from personalization to triggered situ- ational interest that precedes the interest development process observed in the model confirms an effect of personalization on individual interest ( TOTAL � .081, p � .033).

Research Questions 3 and 4: Personalization, problem- solving performance, and algebra learning. We next exam- ined paths from Personalization to measures of problem solving performance in Cognitive Tutor Algebra units, and indirect effects from Personalization to classroom performance through situational interest and in-tutor performance.

Personalization and performance in tutor units. When direct paths from personalization to performance were modeled simulta- neously with indirect paths through triggered interest, the direct path from personalization to percent correct ( � .080, p � .289,) and correct problems per minute ( � .122, p � .107) were

Figure 4. Confirmatory factor analysis solutions for three-factor model of situation interest across four units of algebra learning.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

873SITUATIONAL INTEREST AND PERSONALIZED LEARNING

positive but not statistically significant. As reported above, assign- ment to the personalization condition significantly predicted trig- gered situational interest. Triggered situational interest in units was a significant predictor of percent correct in units ( � .239, p � .006) and corrects per minute ( � .247, p � .003). The total effects of personalization on tutor performance were found to be statistically significant for the efficiency metric, correct problems per minute ( � .163, p � .033) but not for accuracy on first attempts at problems ( � .120, p � .115; Table 5).

Personalization and performance on classroom exams. An analysis of the indirect paths from personalization to performance on a teacher-administered algebra exam revealed a significant total effect ( � .062, p � .045). This indicates that completing units

personalized to individual interests was sufficient to improve stu- dents’ performance on a summative exam that tested linear ex- pressions (i.e., the topic of personalized units).

Discussion

In this study, we aimed to clarify the role of situational interest in personalization and further examined how personalization might impact individual interest in math, problem-solving performance, and classroom achievement through triggering situational interest. This is the first longitudinal study of a context personalization intervention that simultaneously finds effects on motivational and learning outcomes. Moreover, it is the first to further examine the

Table 3 Longitudinal Measurement Invariance Model of Situational Interest in Four Algebra Units

Model Parameters �2 df p CFI RMSEA [90% CI] SRMR ��2 �df p

Triggered Situational interest

Configural 48 1.478 8 .233 .998 .053 [.000, .098] .014 — — — Metric 39 15.691 17 .546 1.000 .000 [.000, .080] .036 5.213 9 .815 Scalar 30 33.984 26 .135 .994 .053 [.000, .098] .051 23.505 18 .171

Maintained Situational interest

Configural 100 132.145 76 .001 .980 .082 [.058, .105] .030 — — — Metric 82 141.741 94 .001 .983 .068 [.044, .090] .040 9.596 18 .944 Scalar 64 163.832 112 .001 .982 .065 [.042, .086] .044 31.687 36 .674

Note. CFI � comparative fit index; RMSEA � root mean square error of approximation; CI � confidence interval; SRMR � standardized root mean square residual.

Figure 5. Model solution showing only paths significant at p � .05. Model fit: �2(141) � 173, p � .032; comparative fit index � .983; root mean square error of approximation � .039; 90% confidence interval � .013, .058; standardized root mean square residual � .082.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

874 BERNACKI AND WALKINGTON

mechanisms by which these outcomes are achieved, refining our understanding of the role of situational interest first posed by Høgheim and Reber (2015). By using measures of multiple phases of interest development, our path model identifies the central, intermediate role that triggering of situational interest plays in increasing downstream motivational and learning outcomes. Per- sonalization triggered greater situational interest, which predicted the maintenance of situational interest and, ultimately, increases in students’ individual interest in math. Personalization also achieved a significant portion of its overall effects on learning efficiency through the triggering of situational interest. Learning efficiency in turn contributed to prediction of variance in achievement on a classroom test. We expand upon these contributions in the sections that follow.

Manipulation Check: Effect of Personalization on Perception of Relevance

Because students declared their out-of-school interests when they began to use Cognitive Tutor Algebra, we anticipated that they would recognize when units included problems personalized to these out-of-school interests. This did not seem to be the case, however, as students in personalized and control groups indicated similar levels of agreement that the problems they solved were relevant to their out-of-school interests. One interpretation of this null finding is that explicitly perceived relevance is not the mech- anism by which personalization achieved its effects. In other words, given the observed effects on interest and performance, personalization may be more of a “stealth” intervention than was originally anticipated (Yeager & Walton, 2011). Indeed, if such effects were obtained without the conscious awareness of the learner, it would draw further distinctions between context person- alization and explicit, self-generation methods such as those em- ployed in utility value interventions (e.g., Hulleman & Harackie- wicz, 2009).

Another interpretation of our null manipulation check item concerns the validity of our one-item “relevance” measure and the extent to which students interpreted it as expected. Indeed, an examination of other instruments used by researchers investigating perceptions of relevance (Hulleman & Harackiewicz, 2009; Hul- leman et al., 2017; Jensen, King, Carcioppolo, & Davis, 2012;

Jensen, King, Carcioppolo, Krakow, Samadder, & Morgan, 2014; Kosovich, Hulleman, Barron, & Getty, 2015) reveals that research- ers have used scales with multiple items, and these items tended to probe features of relevance (Albrecht & Karabenick, 2016; Hul- leman & Barron, 2015) by including the terms usefulness, per- sonal, customized, and applicable, but seldom employed the term relevant itself. Moreover, education researchers who investigate relevance refrain from including the term relevance in items posed to middle school students (e.g., Hulleman & Harackiewicz, 2009; Kosovich et al., 2015), and include it only with older samples (e.g., Hulleman et al., 2017).

Effects of Personalization on Situational Interest

The findings presented here add to the emerging study of the effects of personalization on interest. Høgheim and Reber (2015) observed that personalization increased both the triggered and maintained forms of situational interest. Students in their sample who solved personalized problems reported means 0.9 and 1.2 points higher than the control group, compared to mean differences of 0.3 and 0.2 points for triggered and maintained situational interest, respectively, in the current study (all on a 5-point Likert scale). A number of factors may explain the differences across these studies. Though the ages of participants were similar (i.e., 8th through 10th grade), samples differed in geographical origin (i.e., Scandinavian vs. United States), tasks differed in content area (i.e., probability vs. linear algebra), and personalization dosage was much larger in the current study (four units of problems) than in the Høgheim and Reber study (three of seven problems).

The geographic and topical differences between studies are unlikely to be responsible for differences in interest unless some cultural elements that have yet to be theorized differ across sam- ples. In terms of the theoretically important features of the context personalization intervention (Walkington & Bernacki, 2014), the intervention approaches differ in the depth, ownership, and grain size of personalization. Høgheim and Reber (2015) described their intervention as a shallow context personalization approach, as students supplied surface features of problems like names of music artists and locations. This differs from the problems students solved in the Cognitive Tutor Algebra interventions, which typi- cally develop a story around the students’ reported interest as

Table 4 Standardized Parameter Estimates From Hypothesized Model of Relations Among Personalization, Interest and Achievement

Independent variables

Endogenous (dependent variables)

Triggered situational

interest

Maintained situational

interest (enjoyment)

Maintained situational

interest value

Posttest individual interest in

math

Percent correct on first attempts

Correct answers per

minute

Classroom posttest score

Personalization .17� — — — .08 .12 — Initial interest in math .53��� .16��� .29��� .30��� — — .01 Prior math achievement — — — — .28�� .21�� .26 Triggered situational interest — .86��� .58��� — .24�� .25�� .09 Maintained situational interest (enjoyment) — — — .27�� — — — Maintained situational interest (value) — — — .43��� — — — Percent correct in units (% Correct) — — — — — — .16 Correct answers per minute — — — — — — .17

� p � .05. �� p � .01. ��� p � .001.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

875SITUATIONAL INTEREST AND PERSONALIZED LEARNING

opposed to inserting the interest into a problem template, as can be seen in the sample problems in the introduction. Under Høgheim and Reber’s (2015) classification system, the problems in the present study would have been high interest, high preference (as opposed to the high interest low preference problems in their study). However, because their more superficial problems obtained greater effects on situational interest, we dismiss this difference in depth as an explanation for differences in situational interest.

In Høgheim and Reber’s intervention, students were asked to supply an interest topic and a second level of detail about the interest to be used in the problems (i.e., subject and object in the story), as well as a location the student has visited. The process of providing specific details that are incorporated into the problem produce two phenomena that are theorized to impact the quality of personalization. First, students themselves take part in personaliz- ing the problem, giving them at least partial ownership of the personalized learning experience (Walkington & Bernacki, 2014), and increasing both their awareness of personalization, and their degree of engagement with the problem. This enhanced sense of ownership might explain the greater levels of interest they report. Second, the more specific details students provided in the Høgheim and Reber study, although unrelated to the math, also refine the grain size of the context personalization to students’ very

precise interests (Walkington & Bernacki, 2014). This level of detail maps more specifically to students’ personal interests (e.g., football) than does a problem personalized to a more general interest category like “sports,” and may also be responsible for greater effects on interest.

Finally, it is important to note that unlike Høgheim and Reber (2015), our results did not vary based on students’ initial level of individual interest in mathematics. Høgheim and Reber’s interven- tion was mainly effective for students with low individual interest in math. Their intervention was quite shallow (although it had a fine grain size), and it is therefore not surprising that students with higher interest in math would not be impacted much by it. Durik and Harackiewicz’s (2007) use of a utility value intervention for math learning was, on the other hand, beneficial only for students with high individual interest in math. Their intervention, which communicated everyday life applications of multiplication, could be seen as having high depth, but a coarse grain size. The Cogni- tive Tutor Algebra intervention we report here could be concep- tualized as falling right in the middle—with a medium depth and grain size. Thus, it makes sense that we did not see differential effects between learners with high or low individual interest in mathematics.

Table 5 Direct, Indirect, and Total Effects on Outcome Variables

Dependent variable Direct Indirect Total effects

Personalization on interest Triggered situational interest .169� — .169�

Maintained situational interest (enjoyment) — .145� .145�

Through triggered .145�

Maintained situational interest (value) — .098� .098�

Through triggered .098�

Individual interest in math (posttest) — .081� .081�

Through triggered, maintained (enjoyment) .039 Through triggered, maintained (value) .042

Initial individual interest on interest Individual Interest in math (posttest) .300��� .419��� .719���

Through triggered, maintained (enjoyment) .123��

Through triggered, maintained (value) .130�

Through maintained (enjoyment) .044 Through maintained (value) .122��

Personalization on performance Percent correct in algebra units .080 .040 .120

Through triggered .040 Correct answers per minute in units .122 .042 .163�

Through triggered .042 Score on classroom math posttest — .062� .062�

Through percent correct in algebra units .013 Through correct answers per minute in units .021 Through triggered .015 Through triggered, percent correct .007 Through triggered, correct per minute .007

Initial individual interest on performance Score on classroom math posttest .006 .090 .096

Through triggered .047 Through triggered, percent correct .020 Through triggered, correct per minute .022

Prior achievement on performance Score on classroom math posttest .263�� .081 .344���

Through percent correct .044 Through correct per minute .036

� p � .05. �� p � .01. ��� p � .001.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

876 BERNACKI AND WALKINGTON

Effects of Personalization on Individual Interest in Mathematics

Through the triggering of situational interest, personalization also achieved a significant effect on students’ individual interest in mathematics at the end of the intervention. Although the effect on individual interest is small, we would expect that students’ indi- vidual interest in algebra would be resistant to change. It is thus notable that modifications to story contexts in four units of a 44-unit curriculum were able to achieve an effect on students’ individual interest in mathematics, especially since this reporting sometimes occurred months after students completed personalized units. This is a notable finding because individual interest is known to drive subsequent enrollment and career choice (Harackiewicz, Barron, Tauer, Carter, & Elliot, 2000; Lent, Lopez, & Bieschke, 1991). It is important to note that the study was conducted within a predominantly Caucasian, middle class school. In current work, we are exploring personalization interventions with a more diverse student population.

Effects of Personalization on Performance

In contrast to sizable effects of context personalization on situ- ational interest, Høgheim and Reber (2015) found no effects of personalization on performance. In the current study, we do find that triggered interest—induced by personalization—predicted ef- ficiency in the tutor and performance on classroom exams. Whereas grain size and ownership may be responsible for effects on interest, other design differences may explain different effects on efficiency. For example, the greater dosage of personalized problem-solving practice in this study (i.e., four units) might have been sufficient to induce performance differences. The greater depth at which problems were personalized to interests may also be responsible for these effects.

Surprisingly, however, we did not see significant effects for personalization on accuracy in the units, measured by percentage of correct first attempts. This runs counter to Walkington (2013) who observed significant differences for a similar 1-unit person- alization intervention in Cognitive Tutor Algebra, with an effect size of d � 0.28. One explanation for this difference is that the present study was simply underpowered for this particular out- come measure—indeed, to have an 80% chance of replicating such an effect ( � 0.8, � � .05), a sample size of 120 per group (240 total) would have been needed. There also might have been slight differences in the interventions themselves that contributed to this difference (e.g., differences in depth of problems or the wording of problems). Given that other recent studies of personalization have consistently shown no effect for “low-depth” personalization on immediate performance (e.g., Fancsali & Ritter, 2014; Høgheim & Reber, 2015, 2017; Kosh, 2017), and few quantitative studies have examined higher depth interventions, it is still an open question whether context personalization can reliably impact short-term performance and accuracy.

We did observe that personalized problem solving in the intel- ligent tutoring system significantly affected performance on exams administered by teachers in classrooms. Prior evidence of transfer effects from personalized learning were observed (Walkington, 2013) but were constrained to transfer within the intelligent tutor- ing system.5 The transfer finding obtained in this study—that

solving units of problems personalized to students’ out-of-school interests improved performance on an exam completed in the classroom—indicates that transfer effects may indeed expand over time, and also extend from technology-based to classroom math tasks. This is, however, qualified by a small effect size, which is an important consideration for future researchers in this area, as well as those applying these findings to practice.

Future research will be needed to confirm the consistency of the role of situational interest in the effects personalization achieved in this study. Additional studies can determine whether additional attentional processes and engagement might result from increased situational interest and more precisely explain how context per- sonalization obtains its effects.

Theoretical Implications

This study confirms a previously observed effect of personal- ization on learning efficiency (but not accuracy; Walkington, 2013) in an expanded test of personalized problem solving. We also observe an additional, albeit small, effect on classroom achievement, which further indicates that personalization of intel- ligent tutoring systems can have effects outside the environment.

The most salient theoretical contributions relate to the role of personalization in interest development. The results obtained pro- vide support for the four-phase model of interest development (Hidi & Renninger, 2006; Renninger & Hidi, 2016), and provide important insight into the number, length, and quality of situational interest-triggering events that may be required to show effects on individual interest. Indeed, Renninger and Hidi (2016) observed that it can potentially take years to move through the phases of interest development— but here, we saw an effect for individual interest in math with a four-unit intervention. Responding to calls for more research on variables like personalization (Renninger & Hidi, 2016), the present study demonstrates the interrelationship of personalization and interest development. We also empirically distinguish the unique ways that maintained interest due to feelings of enjoyment and perception of value contribute to increases in individual interest.

Practical Implications

Findings provide evidence that personalization is an emerging design principle that can be employed to increase interest in learning tasks. By leveraging students’ out-of-school interests— activities they pursue of their own volition—and incorporating them into math tasks, context personalization presents an oppor- tunity to develop students’ individual interest in mathematics. This produces a more enduring potential impact where interest gener- ated through personalization can be transferred to mathematics itself, allowing for a transfer of interest and additional iterations of interest development processes as students reengage in math tasks in future years of schooling.

5 Differences in the subsequent unit were examined in this study as well, but a full reporting falls outside the scope of the article. In brief, significant differences in learning efficiency favoring the personalization condition were again detected, but differences in accuracy (first attempt correct) were not.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

877SITUATIONAL INTEREST AND PERSONALIZED LEARNING

Limitations

The sample size (150 students) and composition of this study limited opportunities to investigate moderating factors and indi- vidual differences that might influence how personalization affects interest and performance. The combination of sample size and assignment precluded examination of teacher effects,6 which is a shortcoming that should be addressed with larger samples in future work. Because the intervention was conducted entirely in the software environment, the impact of the instructor on the student experience should be limited. Nevertheless, accounting for nesting effects would improve the precision of model solutions. In addi- tion, limited sample size and insufficient data on gender precluded multigroup analyses across males and females, who can differ in math interest and achievement (e.g., Gaspard et al., 2015).

A second limitation was the instrumentation available to study interest development in the individual interest phase. The instru- ments used here were sufficient to capture both triggered and maintained situational interest, but failed to discriminate between emerging and well-developed forms of individual interest. In ad- dition, we encountered a potential limitation to the observation of triggered and maintained situational interest due to enjoyment, which were correlated at r � .91. More research is undoubtedly needed to discern the nature of maintained situational interest, the orthogonality of its dimensions, and their respective influence on interest development and in-task engagement.

Future Considerations

The potential impacts context personalization can have on stu- dents’ task interest and individual interest in mathematics are promising, but the prospects of context personalization are not without limitation. It will be important to explore whether context personalization can be applied more broadly across a math curric- ulum that does not always possess features that lend themselves to adaptation. For example, whereas the story problems that frame linear equations in algebra and probability tasks provide a rich context that allow students’ interests to be infused, other topics, like factoring polynomials, are devoid of such features.

Another important practical consideration is the magnitude of effects obtained on outcomes and the effort required to produce such effects. Across multiple studies, effect sizes for performance, learning, and learning efficiency outcomes are typically in the small to medium range, and effects on interest are only slightly larger. Context personalization is a work-intensive intervention approach where multiple versions of problems may need to be designed to accommodate the diverse (and at times, changing) interests of learners. Designers would be wise to consider the balance of costs and benefits when the effects obtained through personalization are likely to be modest. Curriculum and software developers might also consider ways to increase the effects per- sonalization obtains by looking to theory. Larger effects might be achieved by deepening personalization. Indeed, a number of recent investigations of shallow personalization have seen no positive impact on performance or learning (Fancsali & Ritter, 2014; Kosh, 2017; Høgheim & Reber, 2015, 2017) suggesting this is a risky approach for curriculum developers to invest in. Evidence is emerging that a better alignment between math content and student interest areas can activate funds of knowledge to help ground the mathematical features to well-known phenomena (Walkington &

Bernacki, 2018). This too comes with a caveat as learners with different depth of knowledge and quantitative engagement with their own interests can influence the ways they learn with person- alized problems.

A second approach to improving context personalization might be to refine the grain size of personalization by incorporating more specific levels of student interest into the problem context. For example, students interested in sports tend to benefit from receiv- ing problems that incorporate sports elements, but a more precise mapping to a student’s interest in a specific sport, like track and field, might elicit greater levels of situational interest. However, this presents an additional production challenge: many more prob- lems may be needed to accommodate many fine-grained interests.

A third potential opportunity is to increase students’ ownership of personalization. At present, curriculum developers or research- ers often author personalized problems for students, and additional work is constantly needed to ensure that the supply of personalized problems can keep up with demand as new out-of-school interests arise and become prevalent—and problems that incorporate now- passé interests are retired. An emerging model to refine the gran- ularity and increase the volume of problems is to enlist students as authors of personalized problems. This “personalized problem- posing” activity is an educative experience for students who, in the process of authoring a math problem that incorporates their out- of-school interests, develop their own understanding of the math- ematical features of their interests and their grasp of the sociom- athematical norms that govern the structure and language of math problems. Students enrolled in courses as early as prealgebra have demonstrated the ability to author mathematically valid personal- ized algebra problems (Walkington & Bernacki, 2015). If class- room activities and technology tools can be developed to support students as problem authors, this can provide many important benefits for math learning and context personalization. Students can develop a greater ownership of their math learning by con- tributing to the development of math tasks and increase their situational interest in math as observed in Høgheim and Reber (2015). In addition, a broad implementation of the student-as- problem-poser production model can increase the number of per- sonalized problems available for others to solve, as well as the granularity of the specific interests incorporated in these problems.

Conclusion

Evidence that context personalization positively affects situa- tional interest and performance in math tasks continues to accrue, and new evidence includes an observed effect on individual inter- est in math. This study replicated past effects where personaliza- tion improved efficiency of learning, and demonstrated personal- ization also improved classroom test performance. This increases confidence that context personalization is a beneficial principle for

6 One teacher taught two small sections totaling only 28 of the 150 examined here, while the second teacher taught the remaining 122. This imbalance in group sizes by teacher precluded the use of a multigroup model to examine different pathways between tracks of students, as does use of classroom as a clustering variable (doing so induces an error in the model due to insufficient number of clusters). Further, sample size of the two class types (i.e., covering half (n � 73) and whole Algebra I curric- ulum (n � 77) were insufficient to test our model across groups (i.e., more parameters than members of a sample).

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

878 BERNACKI AND WALKINGTON

the design of math tasks. In addition, the situational interest that arises when students engage in personalized learning appears to largely explain effects on learning outcomes. This confirms theo- retical assumptions about personalization and draws connections to theory on interest.

References

Albrecht, J., & Karabenick, S. (2016, August 24 –27). Conceptual expli- cation of relevance in motivational theory. Symposium at the 15th International Conference of Motivation, Research Dissemination Center, Aristotle University of Thessaloniki, Thessaloniki, Greece.

Barton, A. C., & Tan, E. (2009). Funds of knowledge and discourses and hybrid space. Journal of Research in Science Teaching, 46, 50 –73.

Bates, E., & Wiest, L. (2004). The impact of personalization of mathemat- ical word problems on student performance. The Mathematics Educator, 14, 17–26.

Bernacki, M. L., Byrnes, J. P., & Cromley, J. G. (2012). The effects of achievement goals and self-regulated learning behaviors on reading comprehension in technology-enhanced learning environments. Contem- porary Educational Psychology, 37, 148 –161. http://dx.doi.org/10.1016/ j.cedpsych.2011.12.001

Bernacki, M. L., Nokes-Malach, T. J., & Aleven, V. (2013). Fine-grained assessment of motivation over long periods of learning with an intelli- gent tutoring system: Methodology, advantages, and preliminary results. In R. Azevedo & V. Aleven (Eds.), International handbook of metacog- nition and learning technologies (pp. 629 – 644). New York, NY: Springer New York. http://dx.doi.org/10.1007/978-1-4419-5546-3_41

Byrne, B. M. (2013). Structural equation modeling with Mplus: Basic concepts, applications, and programming. New York, NY: Routledge.

Cakir, O., & Simsek, N. (2010). A comparative analysis of computer and paper-based personalization on student achievement. Computers & Educa- tion, 55, 1524 –1531. http://dx.doi.org/10.1016/j.compedu.2010.06.018

Canning, E. A., & Harackiewicz, J. M. (2015). Teach it, don’t preach it: The differential effects of directly-communicated and self-generated utility– value information. Motivation Science, 1, 47–71. http://dx.doi.org/10.1037/ mot0000015

Carnegie Learning. (2016). Cognitive tutor algebra [software]. Pittsburgh, PA: Author.

Chen, F., Curran, P. J., Bollen, K. A., Kirby, J., & Paxton, P. (2008). An empirical evaluation of the use of fixed cutoff points in RMSEA test statistic in structural equation models. Sociological Methods & Re- search, 36, 462– 494. http://dx.doi.org/10.1177/0049124108314720

Civil, M. (2007). Building on community knowledge: An avenue to equity in mathematics education. In N. Nassir. and P. Cobb (Eds.) Improving access to mathematics: Diversity and equity in the classroom (pp. 105–117). Teachers College Press.

Cordova, D., & Lepper, M. (1996). Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice. Journal of Educational Psychology, 88, 715–730. http://dx.doi .org/10.1037/0022-0663.88.4.715

Durik, A. M., & Harackiewicz, J. M. (2007). Different strokes for different folks: How individual interest moderates the effects of situational factors on task interest. Journal of Educational Psychology, 99, 597– 610. http://dx.doi.org/10.1037/0022-0663.99.3.597

Durik, A. M., Shechter, O. G., Noh, M., Rozek, C. S., & Harackiewicz, J. M. (2015). What if I can’t? Success expectancies moderate the effects of utility value information on situational interest and performance. Motivation and Emotion, 39, 104 –118.

Eccles, J. (1983). Expectancies, values and academic behaviors. In R. C. Atkinson, G. Lindzey, & R. F. Thompson (Eds.), Achievement and achievement motives: Psychological and sociological approaches (pp. 75–146). San Francisco: Freeman & Co.

Ellis, G. (2008). Grand challenges for engineering. IEEE Engineering Management Review, 37. National Academy Press. Retrieved from www.engineeringchallenges.org/cms/8996/9127.aspx

Fancsali, S. E., & Ritter, S. (2014). Context personalization, preferences, and performance in an intelligent tutoring system for middle school mathematics. In Proceedings of the fourth international conference on learning analytics and knowledge (pp. 73–77). New York, NY: ACM.

Gaspard, H., Dicke, A. L., Flunger, B., Schreier, B., Häfner, I., Trautwein, U., & Nagengast, B. (2015). More value through greater differentiation: Gender differences in value beliefs about math. Journal of Educational Psychology, 107, 663– 677. http://dx.doi.org/10.1037/edu0000003

Goldstone, R., & Son, J. (2005). The transfer of scientific principles using concrete and idealized simulations. Journal of the Learning Sciences, 14, 69 –110. http://dx.doi.org/10.1207/s15327809jls1401_4

Harackiewicz, J. M., Barron, K. E., Tauer, J. M., Carter, S. M., & Elliot, A. J. (2000). Short-term and long-term consequences of achievement goals: Predicting interest and performance over time. Journal of Edu- cational Psychology, 92, 316 –330.

Harackiewicz, J. M., Canning, E. A., Tibbetts, Y., Priniski, S. J., & Hyde, J. S. (2016). Closing achievement gaps with a utility-value intervention: Disentangling race and social class. Journal of Personality and Social Psychology, 111, 745–765. http://dx.doi.org/10.1037/pspp0000075

Harackiewicz, J. M., Durik, A. M., Barron, K. E., Linnenbrink-Garcia, L., & Tauer, J. M. (2008). The role of achievement goals in the development of interest: Reciprocal relations between achievement goals, interest, and performance. Journal of Educational Psychology, 100, 105–122. http:// dx.doi.org/10.1037/0022-0663.100.1.105

Hidi, S., & Anderson, V. (1992). Situational interest and its impact on reading and expository writing. In K. A. Renninger & A. Krapp (Eds.), The role of interest in learning and development (pp. 215–238). New York, NY: Taylor & Francis.

Hidi, S., & Renninger, K. (2006). The four-phase model of interest devel- opment. Educational Psychologist, 41, 111–127. http://dx.doi.org/10 .1207/s15326985ep4102_4

Høgheim, S., & Reber, R. (2015). Supporting interest of middle school students in mathematics through context personalization and example choice. Contemporary Educational Psychology, 42, 17–25. http://dx.doi .org/10.1016/j.cedpsych.2015.03.006

Høgheim, S., & Reber, R. (2017). Eliciting mathematics interest: New directions for context personalization and example choice. The Journal of Experimental Education, 85, 1–17. http://dx.doi.org/10.1080/ 00220973.2016.1268085

Hu, L., & Bentler, P. M. (1999). Cutoff Criteria for fit indexes in covari- ance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. http://dx.doi.org/10.1080/1070 5519909540118

Hulleman, C. S., & Barron, K. E. (2015). Motivation interventions in education. In L. Corno & E. M. Anderman (Eds.), Handbook of educa- tional psychology (pp. 160). New York, NY: Routledge.

Hulleman, C. S., Godes, O., Hendricks, B. L., & Harackiewicz, J. M. (2010). Enhancing interest and performance with a utility value inter- vention. Journal of Educational Psychology, 102, 880 – 895.

Hulleman, C. S., & Harackiewicz, J. M. (2009). Promoting interest and performance in high school science classes. Science, 326, 1410 –1412. http://dx.doi.org/10.1126/science.1177067

Hulleman, C. S., Kosovich, J. J., Barron, K. E., & Daniel, D. B. (2017). Making connections: Replicating and extending the utility value inter- vention in the classroom. Journal of Educational Psychology, 109, 387– 404. http://dx.doi.org/10.1037/edu0000146

Jensen, J. D., King, A. J., Carcioppolo, N., & Davis, L. (2012). Why are tailored messages more effective? A multiple mediation analysis of a breast cancer screening intervention. Journal of Communication, 62, 851– 868. http://dx.doi.org/10.1111/j.1460-2466.2012.01668.x

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

879SITUATIONAL INTEREST AND PERSONALIZED LEARNING

Jensen, J. D., King, A. J., Carcioppolo, N., Krakow, M., Samadder, N. J., & Morgan, S. (2014). Comparing tailored and narrative worksite inter- ventions at increasing colonoscopy adherence in adults 50 –75: A ran- domized controlled trial. Social Science & Medicine, 104, 31– 40. http:// dx.doi.org/10.1016/j.socscimed.2013.12.003

Kenny, D. A., Kaniskan, B., & McCoach, D. B. (2015). The performance of RMSEA in models with small degrees of freedom. Sociological Methods & Research, 44, 486 –507. http://dx.doi.org/10.1177/00491 24114543236

Khan Academy. (2016). Math. Retrieved from www.khanacademy.org/ math

Koedinger, K. R., Alibali, M. W., & Nathan, M. J. (2008). Trade-offs between grounded and abstract representations: Evidence from algebra problem solving. Cognitive Science, 32, 366 –397. http://dx.doi.org/10 .1080/03640210701863933

Koedinger, K. R., & Corbett, A. (2006). Cognitive tutors. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 61–77). New York, NY: Cambridge Press.

Koedinger, K. R., Corbett, A. T., & Perfetti, C. (2012). The knowledge- learning-instruction framework: Bridging the science-practice chasm to enhance robust student learning. Cognitive Science, 36, 757–798. http:// dx.doi.org/10.1111/j.1551-6709.2012.01245.x

Kosh, A. E. (2017). The effects on mathematics performance of personal- izing word problems to students’ interests (Unpublished doctoral disser- tation). University of North Carolina at Chapel Hill, Chapel Hill, North Carolina.

Kosovich, J. J., Hulleman, C. S., Barron, K. E., & Getty, S. (2015). A practical measure of student motivation: Establishing validity evidence for the expectancy-value-cost scale in middle school. The Journal of Early Adolescence, 35, 790 – 816. http://dx.doi.org/10.1177/02724 31614556890

Ku, H., & Sullivan, H. (2000). Personalization of mathematics word problems in Taiwan. Educational Technology Research and Develop- ment, 48, 49 – 60. http://dx.doi.org/10.1007/BF02319857

Lent, R. W., Lopez, F. G., & Bieschke, K. J. (1991). Mathematics self- efficacy: Sources and relation to science-based career choice. Journal of Counseling Psychology, 38, 424 – 430. http://dx.doi.org/10.1037/0022- 0167.38.4.424

Linnenbrink-Garcia, L., Durik, A. M., Conley, A. M., Barron, K. E., Tauer, J. M., Karabenick, S. A., & Harackiewicz, J. M. (2010). Measuring situational interest in academic domains. Educational and Psychological Measurement, 70, 647– 671. http://dx.doi.org/10.1177/001316440 9355699

Lipstein, R., & Renninger, K. A. (2006). “Putting things into words”: 12–15-year-old students’ interest for writing. In P. Boscolo & S. Hidi (Eds.), Motivation and writing: Research and school practice (pp. 113– 140). New York, NY: Kluwer Academic/Plenum Press.

López, C., & Sullivan, H. (1992). Effects of personalization of instructional context on the achievement and attitudes of Hispanic students. Educa- tional Technology Research and Development, 40, 5–14. http://dx.doi .org/10.1007/BF02296895

Mitchell, M. (1993). Situational interest: Its multifaceted structure in the secondary school mathematics classroom. Journal of Educational Psy- chology, 85, 424 – 436. http://dx.doi.org/10.1037/0022-0663.85.3.424

Mitchell, T. J., Chen, S. Y., & Macredie, R. D. (2005). Hypermedia learning and prior knowledge: Domain expertise vs. system expertise. Journal of Computer Assisted Learning, 21, 53– 64. http://dx.doi.org/10 .1111/j.1365-2729.2005.00113.x

Moll, L. C., Amanti, C., Neff, D., & Gonzalez, N. (1992). Funds of knowledge for teaching: Using a qualitative approach to connect homes and classrooms. Theory into Practice, 31, 132–141. http://dx.doi.org/10 .1080/00405849209543534

Moos, D. C., & Azevedo, R. (2009). Learning with computer-based learn- ing environments: A literature review of computer self-efficacy. Review

of Educational Research, 79, 576 – 600. http://dx.doi.org/10.3102/ 0034654308326083

Muthén, L. K., & Muthén, B. O. (2010). Mplus software (Version 6). Los Angeles, CA: Author.

Nunes, T., Schliemann, A. D., & Carraher, D. W. (1993). Street mathe- matics and school mathematics. New York, NY: Cambridge University Press.

Renninger, A., Hidi, S., & Krapp, A. (2014). The role of interest in learning and development (2nd ed.). New York, NY: Psychology Press.

Renninger, K. A., & Hidi, S. (2002). Student interest and achievement: Developmental issues raised by a case study. In A. Wingfield & J. S. Eccles (Eds.), Development of achievement motivation (pp. 173–195). New York, NY: Academic. http://dx.doi.org/10.1016/B978-012750053- 9/50009-7

Renninger, K. A., & Hidi, S. (2016). The power of interest for motivation and engagement. New York, NY: Routledge.

Renninger, K. A., & Shumar, W. (2002). Community building with and for teachers: The Math Forum as a resource for teacher professional devel- opment. In K. A. Renninger & W. Shumar (Eds.), Building virtual communities: Learning and change in cyberspace (pp. 60 –95). New York, NY: Cambridge University Press. http://dx.doi.org/10.1017/ CBO9780511606373.008

Renninger, K. A., & Su, S. (2012). Interest and its development. In R. M. Ryan (Ed.), The Oxford handbook of human motivation (pp. 167–187). Oxford, UK: University Press.

Schraw, G., Flowerday, T., & Lehman, S. (2001). Increasing situational interest in the classroom. Educational Psychology Review, 13, 211–224.

Simsek, N., & Cakir, O. (2009). Effect of personalization on students’ achievement and gender factor in mathematics education. The Interna- tional Journal of Social Sciences, 4, 278 –282.

U.S. Department of Education. (2010). National education technology plan. Retrieved from http://tech.ed.gov/netp/

U.S. Department of Education. (2016). National education technology plan. Retrieved http://tech.ed.gov/netp/

Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recom- mendations for organizational research. Organizational Research Meth- ods, 3, 4 –70.

Walkington, C., (in press). Design research on personalized problem- posing. In E. Galindo & J. Newton (Eds.), In Proceedings of the 39th annual meeting of the North American chapter of the international group for the psychology of mathematics education. Indianapolis, IN: Hoosier Association of Mathematics Teacher Educators.

Walkington, C. A. (2013). Using adaptive learning technologies to person- alize instruction to student interests: The impact of relevant contexts on performance and learning outcomes. Journal of Educational Psychol- ogy, 105, 932–945. http://dx.doi.org/10.1037/a0031882

Walkington, C., & Bernacki, M. L. (2014). Motivating students by “per- sonalizing” learning around individual interests: A consideration of theory, design, and implementation issues. In S. Karabenick & T. Urdan (Eds.), Advances in motivation and achievement (Vol. 18, pp. 139 –176). Bingley, UK: Emerald. http://dx.doi.org/10.1108/S0749-74232014 0000018004

Walkington, C., & Bernacki, M. L. (2015). Students authoring personalized “algebra stories”: Problem-posing in the context of out-of-school inter- ests. Journal of Mathematical Behavior, 40(B), 171–191. http://dx.doi .org/10.1016/j.jmathb.2015.08.001

Walkington, C., & Bernacki, M. L. (in press). Personalization of instruc- tion: Design dimensions and implications for cognition. Journal of Experimental Education.

Walkington, C., Cooper, J., Alibali, M. A., & Nathan, M. J. (2015, November). The effects of visual representations and interest-based personalization on solving mathematics story problems. Presentation at The 37th annual meeting of the North American Chapter of the Inter-

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

880 BERNACKI AND WALKINGTON

national Group for the Psychology of Mathematics Education. East Lansing, MI: Michigan State University.

Walkington, C., Sherman, M., & Howell, E. (2014). Personalized learning in algebra. Mathematics Teacher, 108, 272–279. http://dx.doi.org/10 .5951/mathteacher.108.4.0272

Walkington, C., & Bernacki, M. L. (2018). Personalizing algebra to students’ interests: How student funds of knowledge moderate outcomes. Manuscript in preparation.

Yeager, D. S., & Walton, G. M. (2011). Social-psychological interventions in education: They’re not magic. Review of Educational Research, 81, 267–301. http://dx.doi.org/10.3102/0034654311405999

Received June 30, 2016 Revision received October 15, 2017

Accepted October 18, 2017 �

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

881SITUATIONAL INTEREST AND PERSONALIZED LEARNING