Rhetorical Analysis Essay（1200_1500words）

Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=nile20

Interactive Learning Environments

ISSN: 1049-4820 (Print) 1744-5191 (Online) Journal homepage: https://www.tandfonline.com/loi/nile20

Gamification and student motivation

Patrick Buckley & Elaine Doyle

To cite this article: Patrick Buckley & Elaine Doyle (2016) Gamification and student motivation, Interactive Learning Environments, 24:6, 1162-1175, DOI: 10.1080/10494820.2014.964263

To link to this article: https://doi.org/10.1080/10494820.2014.964263

Published online: 09 Oct 2014.

Submit your article to this journal

Article views: 10123

View related articles

View Crossmark data

Citing articles: 51 View citing articles

Gamification and student motivation

Patrick Buckleya and Elaine Doyleb*

aDepartment of Management and Marketing, University of Limerick, Limerick, Ireland; bDepartment of Accounting and Finance, Kemmy Business School, University of Limerick, Limerick, Ireland

(Received 7 April 2014; final version received 8 September 2014)

The literature suggests that gamified learning interventions may increase student engagement and enhance learning. We empirically investigate this by exploring the impact of intrinsic and extrinsic motivation on the participation and performance of over 100 undergraduate students in an online gamified learning intervention. The paper makes a number of contributions. First, by synthesizing the literature the central concepts required for a learning intervention to be considered gamified are mapped and the development of an online gamified learning intervention is described. Second, the effect of gamification on learning outcomes is examined using a pre- and post-intervention survey. We find that gamified learning interventions have a positive impact on student learning. Third, our results show that while generally positive, the impact of gamified intervention*ns on student participation varies depending on whether the student is motivated intrinsically or extrinsically. These findings will be of practical interest to teaching and learning practitioners working in a range of educational contexts, and at all levels of education, who wish to increase student engagement and enhance learning.

Keywords: gamification; intrinsic motivation; extrinsic motivation; prediction market

Introduction

Gamification has generated increased attention recently across a range of contexts. At the 2011 Gamification Summit, M2 Research forecasted that the gamification market would reach €2.8 billion in direct spending in the USA by 2016 (M2 Research – Gamification, 2013). The Gartner Group suggests that by 2015 more than 50% of organizations managing innovation processes will gamify them (Gartner, 2011). The interest in gamification arises from the idea that it influences behaviour. Games provoke powerful emotional responses, such as curiosity, frustration and joy (McGonigal, 2011). People are posited to be more engaged and more productive when playing games (Kim, 2012). Games and play are imbedded in our cultural history and identity. However, we are also beginning to understand that we are hardwired to play, with researchers increasingly discovering complex relation- ships between our brains, neural systems and game play (Zichermann & Cunningham, 2011). Done effectively, gamification can align the interests of the game designer with the interests and motivations of players. Product designers and marketers are leveraging this alignment in business contexts to “make them [consumers] come in, bring friends

© 2014 Informa UK Limited, trading as Taylor & Francis Group

*Corresponding author. Email: [email protected]

Interactive Learning Environments, 2016 Vol. 24, No. 6, 1162–1175, http://dx.doi.org/10.1080/10494820.2014.964263

and keep coming back” (Zichermann & Cunningham, 2011, p. ix). As educators, this is what we strive for: capturing the attention and interest of our students, and having them engage in a manner that sustains their interest and keeps them coming back for more. The merits and disadvantages of using gamification in a higher level education context are the subject of recent debate in the literature (Domínguez et al., 2013; Lee & Hammer, 2011). In this paper, we seek to move the debate forward by empirically investi- gating the impact of gamification in a web-based educational context.

The paper makes a number of contributions. First, by synthesizing the literature the central concepts required for a learning intervention to be considered gamified are mapped and the development of an online gamified learning intervention is outlined. Second, we examine the impact of gamification on learning outcomes. One of the advan- tages ascribed to gamification is the positive impact it can have on engagement, partici- pation and learning behaviours. However, learner engagement needs to be understood in terms of both internal and external motivation (Deci, Koestner, & Ryan, 2001). Our final contribution is to investigate the relationship between motivation and participation in online gamified learning interventions.

In the remainder of this paper we review the literature on gamification and motivation, thus deriving our research questions and our hypotheses. Of particular note is our synthesis of the literature to develop a schedule of the key design elements associated with gamifica- tion. In the methodology section, we describe our online gamified learning intervention and the data collection instruments used. The next section presents our results, while we con- clude with a discussion of the implications of our research and suggestions for further work.

Literature review

Gamification

Gamification is “using game-based mechanics, aesthetics and game thinking to engage people, motivate action, promote learning, and solve problems” (Kapp, 2012, p. 10). Gami- fication applies characteristics associated with video games, such as game mechanics and game dynamics, to non-game applications (Simões, Redondo, & Vilas, 2013). It is impor- tant to distinguish gamification from the well-established use of computer games in edu- cation (Squire, 2003). As well as a plethora of business simulation games, a range of commercial games such as Civilization, Railroad Tycoon and World of Warcraft have been used as learning tools. However, as a pedagogical concept gamification does not necessarily involve the use of an actual game or information technology. Rather, it involves the integration of design elements or activity patterns traditionally found in games into edu- cational contexts.

The literature identifies several design elements of games that can be integrated into educational contexts. Both traditional and video games tend to have objective, specific rules (Smith-Robbins, 2011). Salen and Zimmerman (2004, p. 81) define a game as “a system in which players engage in an artificial conflict defined by rules that results in a quantifiable outcome.” In a gamified learning intervention, rules structure the learning activity, placing clear limits on the actions a learner can take. This makes it fundamentally different from free-form learning activities, such as essays, projects or presentations. For example, when writing an essay, a student can use an infinite variety of sentences to con- struct a narrative.

Games have reward systems. Individuals receive rewards for achieving a goal or over- coming an obstacle. Examples are badges or prizes (Glover, 2013). One term used to

Interactive Learning Environments 1163

identify different types of rewards is SAPS – Status, Access, Power and Stuff (Zichermann & Cunningham, 2011). The reward is often not directly related to the goal achieved but serves as notice to the player and others that a level of competence has been achieved. Pro- gress tracking is often enabled and guided by reward systems; progress towards an overall objective is mapped out by a sequence of intermediate goals.

Game playing is associated with trial, error, failure and eventual success through prac- tice, experience, reflection and learning. A key objective of most games is not to forbid failure but to develop a positive relationship with it. Failure is not seen as an end, but as a step on the journey to mastery. Gamified learning interventions seek to maintain a positive relationship with failure by creating rapid feedback cycles and keeping the stakes for indi- vidual learning episodes low (Lee & Hammer, 2011).

In many ways, the paradigm that governs current educational systems has many game- like elements. Most assessments strive for objectivity and continuous assessment is seen as desirable. Students earn points for completing assignments correctly. These translate into comparable rewards – grades. If they perform well, students “level up” by proceeding to a more advanced course of study at the end of every academic year (Lee & Hammer, 2011). What distinguishes gamification most distinctly from more traditional approaches is the explicit use of competition as a motivational tool. This competitive element is a source of motivation (Nicholson, 2012). It is often operationalized in the form of a leader board ranking players on the basis of performance in the game (Deterding, Dixon, Khaled, & Nacke, 2011). These ranking systems serve as motivators because participants see their efforts publicly and instantly recognized (Domínguez et al., 2013) (Table 1).

Motivation

Motivation is a theoretical construct used to explain the initiation, direction, intensity, per- sistence and quality of behaviour (Maehr & Meyer, 1997). It is multi-dimensional and is presented in the literature as being variable in both level (i.e. the magnitude of motivation an individual has) and orientation (i.e. the type of motivation an individual experiences) (Ryan & Deci, 2000). It is used as a dependent or mediating variable to explain a vast range of human behaviours across contexts and environments. In education, motivation is considered a key determinant of learning. It is used to explain the attention and effort stu- dents dedicate to particular learning activities (Brophy, 2013). For this reason, part of the role of the teacher is managing learner motivation. In most circumstances, the objective is to increase motivation levels with a view to engendering positive outcomes, such as increased effort, persistence and enhanced performance.

Table 1. List of common gamification design elements.

Gamification element Description

Objective, specific rules

Gamified activities have rules which predetermine the actions a player can or cannot take

Rewards Gamified learning activities have a reward system that provides participants with SAPS for interacting with the game successfully

Rapid feedback cycles

Gamified activities focus on providing rapid feedback so that participants can quickly learn how to improve at the game

Competitive element Gamified activities present participants with a challenge, while the objective outcome associated with games allows for ranking of participants

1164 P. Buckley and E. Doyle

The two main categories of student motivation are intrinsic and extrinsic (Deci et al., 2001). Intrinsic motivation involves learners being interested in what they learn and in the learning process itself (Harlen & Deakin Crick, 2003). It is an “innate psychological need for competence and self-determination” (Deci et al., 2001, p. 3). Conceptually, it is closely associated with cognitive behavioural theories and the work of Piaget who posits that when individuals experience discrepancy between their experienced knowledge of the world and their private, internally held knowledge, they are driven to eliminate this dis- crepancy (see, e.g. Piaget & Cook, 1952).

Extrinsic motivation is associated with individuals who engage in learning because it is a means to an end, relatively disassociated from the content and subject of learning (Harlen & Deakin Crick, 2003). Extrinsic motivation is associated with B.F. Skinners’ behavioural theories of human learning and focuses on the provision of rewards to direct and control learning behaviour (Skinner, 1976). This philosophical perspective assumes learning is best directed through report cards, conduct cards and award ceremonies (Brophy, 2013).

The division of motivation into intrinsic and extrinsic is further refined in the literature. Turning first to intrinsic motivation, it is divided into a tripartite taxonomy (Vallerand et al., 1992). This division is based on the nature of the internalized utility of the behaviour. Intrin- sic motivation to know is the construct best known in the educational arena. It involves the desire to perform a learning activity for the pleasure one experiences while learning (i.e. the utility to an individual is the learning in and of itself). Intrinsic motivation towards accom- plishment is a second type, involving the desire to engage in an activity for the pleasure and satisfaction experienced when accomplishing a difficult feat (Vallerand et al., 1992). Finally, intrinsic motivation to experience stimulation is operative when an individual engages in an activity to be stimulated. Stimulation can take a range of forms such as sensory pleasure, aesthetic pleasure or emotional sensations such as fear or excitement.

Extrinsic motivation has also been refined into more precise constructs (Deci et al., 2001). It has been presented as a continuum running from external regulation through intro- jected regulation to identification. While the stimulation prompting behaviour is always external to the participant, the factor that distinguishes these forms of extrinsic motivation is the participant’s autonomy. The least autonomous form of extrinsic motivation is external regulation. This refers to behaviours performed to satisfy an external demand, meet an externally set standard or avoid an externally imposed penalty. Typically such behaviours are seen as being externally imposed. Introjected regulation, a second form of extrinsic regulation, describes when activities are performed to attain ego enhancement or avoid guilt (i.e. an individual performs an act to maintain or enhance self-esteem). The regulation is internal to the person in this sense, but the stimulus is still external. The final, most auton- omous form of extrinsic motivation is regulation through identification. In this form of motivation, an individual’s identity is linked with an externally proscribed behaviour and he/she performs an action to instantiate that identity.

There is considerable discussion as to the impact differing types of motivation have on learning (e.g. is learning better enabled by intrinsic motivation or extrinsic motivation? (Maehr & Meyer, 1997)). The relationship between types of motivation is also an issue, with some authors suggesting that the provision of external rewards damages intrinsic motivation, while others suggest that there is no evidence to support this (Deci et al., 2001). These debates are beyond the scope of this paper but highlight that any examination of motivation must be nuanced and sensitive to differing motivation types.

Interactive Learning Environments 1165

Research questions

As befits a relatively novel concept, there are a range of open research questions regarding gamification in education (Deterding et al., 2011). Prompting and mediating positive learn- ing behaviours is seen as the key advantage of gamification. However, individuals have varying learning motivations. Some learn for pleasure or to satisfy curiosity, whereas others learn to obtain rewards (e.g. a high-status job and/or financial rewards). Glover (2013) suggests that student motivation is an important determinant of how students react to learning activities. While some may be motivated by having their learning gamified, it may demotivate others. If gamified learning impacts on individuals differently, that does not necessarily deny its utility, rather, it calls for the inclusion of gamified learning interven- tions as part of a range of learning interventions, chosen in a manner that ensures no type of learner is systematically disadvantaged (e.g. it is argued that common forms of academic assessment, such as essay and reflective pieces, favour learners with relatively high intrinsic motivation).

Thus, a key research question raised in the literature is how individuals with different motivations for learning are impacted by gamified learning activities (Glover, 2013). Empirically investigating this will place gamified learning interventions on a more solid pedagogical footing and will pave the way for the effective deployment of gamification in educational contexts in a manner that utilizes their strengths and complements other learning activities. This research aims to commence this agenda by investigating how indi- viduals with differing learning motivations interact with a gamified learning intervention.

This high-level research agenda is operationalized into a number of hypotheses for empirical testing. One of the learning outcomes of the module in which the gamified learn- ing intervention was deployed is that students’ general knowledge of the national tax system will be enhanced (more detail on the relevant module and student cohort is outlined below). Our first hypothesis is that an online gamified learning intervention will have a positive effect on the relevant learning outcome. We operationalized this using hypothesis H1:

H1: Students’ general knowledge of the national tax system will be improved as a result of the gamified learning intervention.

Our remaining hypotheses investigate how students who are motivated differently inter- act with an online gamified learning intervention. We hypothesize that each type of motiv- ation identified in the literature (i.e. intrinsic to know, intrinsic to accomplishment, etc.) is positively correlated with participation in the gamified learning activity. This presents a further six hypotheses for testing.

H2: There is a positive correlation between intrinsic motivation to know and participation H3: There is a positive correlation between intrinsic motivation towards accomplishment and participation H4: There is a positive correlation between intrinsic motivation towards stimulation and participation H5: There is a positive correlation between identified motivation and participation H6: There is a positive correlation between introjected motivation and participation H7: There is a positive correlation between external regulation and participation

1166 P. Buckley and E. Doyle

Methodology

As mentioned, in this paper we examine how learners’ motivation type affects their inter- action with an online gamified learning environment. We operationalize this by designing an online learning intervention built around a prediction market (PM). This learning inter- vention contains all the key game design elements associated with gamification and is described below. Also outlined below are the research instruments used to collect data.

Gamified learning intervention

The gamified learning intervention used was a group decision-making system called a PM, which was adapted to an educational setting. A PM is “designed and run for the primary purpose of mining and aggregating information scattered among traders and subsequently using this information in the form of market values in order to make predictions about specific future events” (Tziralis & Tatsiopoulos, 2007, p. 75). Conceptually, PMs create group forecasts by allowing participants to buy and sell contracts in uncertain, future events. In the simplest form of a PM, a contract is created whose value depends on a future uncertain event. For example, a manager may wish to evaluate whether a project will be completed on time. A contract is created which returns €100 (virtual currency) if the project is completed on time and €0 otherwise. This contract is offered for sale, typically on an electronic market delivered via a website. If market participants believe the project is likely to be completed on time, they will buy the contract, causing its price to rise. If they believe the contrary, they will sell, driving the price down. The price of the contract can therefore be used as an estimate of the group’s collective estimation as to the probability of the project being completed on schedule.

PMs have been positioned as educational tools with applications in a range of domains (see, for example, Buckley, Garvey, & McGrath, 2011; Ellis & Sami, 2012; Evans, 2012; Garvey & Buckley, 2010; Raban & Geifman, 2010). This study uses a carefully designed online PM learning intervention that instantiates all the key elements of gamification.PMs have objective rules. While the cognitive processes that prompt trading decisions are essen- tially unknowable and infinite, at an operational level, a student has a limited set of options to select from. They can buy or sell contracts. The complexity of the system as a whole arises from information aggregation and the repeated interactions of large numbers of traders but is all derived from the limited subset of atomic operations available.

Implicit in the concept of market-driven group decision-making systems is the concept of a reward, the second element of gamified learning interventions. When participants cor- rectly forecast future events by buying contracts in them, they receive virtual cash, increas- ing the value of their portfolio. Equally, when a participant invests in a contract that does not occur, the virtual currency they have invested is lost, reducing the overall portfolio value.

PMs provide continuous feedback, a key enabler of the rapid feedback cycles; the third key element of gamified learning interventions. At any point, the current market price rep- resents the consensus of all market participants as to the likelihood of the event in question occurring. At any point, a participant can compare their personal estimates to the estimates of the group as a whole. Unlike a poll or similar mechanism for information aggregation, a participant is not limited to making one estimate at a point in time. At any stage throughout the operation of the PM a participant can re-evaluate their decisions in response to feedback through the simple expedient of buying/selling more contracts.

Finally, a PM is competitive; individuals can be ranked in terms of performance by comparing the value of their portfolios. From a practical perspective, most PM software

Interactive Learning Environments 1167

makes this competitive element explicit. Most commercial PM platforms use an Automated Market Maker algorithm. These algorithms have a number of beneficial properties, but the important point to note is that they essentially turn a market into a zero-sum game; in order for one participant to make gains, another must suffer losses. Furthermore, most PM plat- forms explicitly rank participants on a leader board.

A full explanation of the strengths and weaknesses of PMs as pedagogical tools is beyond the scope of this paper but can be found in the previous references. Our purpose here is to demonstrate that PM learning interventions possess the essential characteristics of gamified learning interventions.

Research instruments

In order to investigate the research questions previously outlined, an online gamified learn- ing intervention utilizing a PM was designed and deployed as part of an undergraduate module in taxation. The key focus of this module is the development of quantitative tech- nical skills in the calculation of tax liabilities. However, it also aims to develop students’ general knowledge of the national taxation system. In order to meet these learning out- comes, a tax-specific PM learning intervention was developed, with 10% of a student’s final module grade being determined by their performance in the PM. Annually, the Min- ister for Finance announces a range of tax policy decisions as part of the National Budget. The National Budget Forecasting Project (NBFP) required students to forecast the measures that might be introduced as part of budget 2014. This was operationalized by providing stu- dents with a question (e.g. “The national budget will alter capital gains tax as follows”), with a range of options to choose from, for example:

. No change to the current operation of capital gains tax;

. rate changes to between 25% and 28%;

. rate change to between 29% and 32%;

. rate change to between 33% and 35%;

. rate change to between 36% and 40%;

. capital gains taxed at tiered rates of between 25% and 40%.

Students were provided with €5000 virtual cash when the market opened. They used this to invest in the outcome they considered most likely for each question. Over the course of the NBFP, students were asked 14 different questions. Detailed information on how to deploy a PM can be found in the case study described in Buckley et al. (2011). The specific platform used in this study is Inkling Markets (www.inkling.com), which is a web-based platform which can be bought for a monthly subscription fee.

The gamified learning intervention is designed to prompt students to search for infor- mation about the National Budget. Relevant sources would include the news media, govern- mental and Non-Governmental Organisation (NGO) reports and position papers and recommendations from consultancy firms. These sources would provide students with the information they required to make informed and accurate forecasts. However, reading and analysing this information should also improve students’ general knowledge of the national taxation system, thereby tying the learning activity back to the relevant learning outcome.

Testing the hypotheses required the creation of a number of variables which were gath- ered using a pre- and post-intervention survey. In order to test H1, we required a measure of pre- and post-intervention general tax knowledge. We created this measure by asking stu- dents 10 general knowledge questions about the national tax system. These questions were

1168 P. Buckley and E. Doyle

free form, for example, “What is the name of the current chairman of the revenue commis- sioners?” The questions were specifically designed so their answers were not revealed as part of the taught element of the module. Students would have had to acquire the knowledge needed to answer the questions correctly outside the classroom. Each incorrect answer was assigned a value of zero while correct answers were assigned a value of one. To calculate an overall measure for each student, the values assigned to each question were summed up to give a pre- and post-intervention score.

In order to investigate H2–7, we required measures for each type of motivation, and a participation metric. The instrument used to collect data on student motivation was the Aca- demic Motivation Scale (AMS), developed by Vallerand et al. (1992). This is a validated scale that has been used in over 900 studies to date (Cokley, Bernard, Cunningham, & Motoike, 2001; Fairchild, Horst, Finney, & Barron, 2005). The AMS is based on the tenets of self-determination theory and consists of 28 items subdivided into seven sub- scales measuring three types of intrinsic motivation (to know, to accomplish things and to experience stimulation), three types of extrinsic motivation (external, introjected and identified regulation) and amotivation. The AMS has demonstrated satisfactory levels of internal consistency and temporal stability over a one-month period (Vallerand et al., 1992).

In order to measure participation, we drew on information stored in the PM itself. The NBFP ran continuously for 19 days. Students were free to trade whenever suited. We defined participation as the number of unique days that a student made at least one trade on the PM.

Data collection

156 students traded on the NBFP market which commenced on Monday 23 September 2013 and remained open for a three-week period. None of the students participating in this study had any prior experience of gamified learning interventions. Data were collected from two major sources. The first source was the PM software. Among other data concerned with the operation of the PM, the software recorded the time and date of every trade made by every student. This was used to calculate the number of trading days for each student.

The second data source was a pre- and post-survey of participants. The pre-survey con- sisted of the general knowledge instrument and the AMS. This was distributed and com- pleted by students in class directly before the commencement of the NBFP. A total of 122 responses were received, representing a response rate of 78.02%. The post-survey con- sisted of the general knowledge instrument. This was distributed and completed by students directly after the conclusion of the NBFP, but before the final results were announced to reduce possible bias. A total of 112 responses were received to the post intervention survey (71.79% response rate).

Table 2. Cronbach’s alpha scores for items on the AMS.

Psychometric variable Chronbach’s alpha

Intrinsic motivation to know 0.820 Intrinsic motivation towards accomplishment 0.849 Intrinsic motivation to experience stimulation 0.836 Extrinsic motivation identified 0.800 Extrinsic motivation introjected 0.848 Extrinsic motivation external regulation 0.788

Interactive Learning Environments 1169

Standard protocols were used to calculate the scores of the various items measured by the AMS. The Cronbach’s alpha scores of all items were within acceptable tolerances and are reported in Table 2.

After compiling the pre- and post-survey results, the data from the individual surveys were cross referenced using a unique identifier assigned to each student. After removing data which did not include identifiers or which could not be matched with a corresponding survey, a total of 81 paired responses remained. Those responses were used in the following analysis and represent a final response rate of 51.92%.

Results

H1 stated that students’ general knowledge of the national tax system will have been improved by the online gamified learning intervention. In order to test this, a paired samples t-test was conducted to evaluate the impact of the gamified learning intervention on students’ scores on general knowledge of the national taxation system. There was a stat- istically significant increase in students’ general knowledge from Time 1 (M = 0.763, SD = 1.128) to Time 2 (M = 1.888, SD = 1.64), t(79) = −8.007, p < .005. The mean increase in general knowledge scores was 1.125 with a 95% confidence interval ranging from 0.845 to 1.404. The eta squared statistic (0.44) indicated a large effect size (Tables 3–5).

H2–7 suggest that there is a positive correlation between the various types of motivation and participation. The data revealed that the participation variable had violated parametric assumptions due to non-normally distributed data, so a Spearman’s rho test was utilized. The results of this test are given in Table 6.

H2 is supported. There was a small, positive correlation between intrinsic motivation to know and participation, r = 0.194, n = 75, p< = .05. H3 suggests that there is a positive cor- relation between intrinsic motivation towards accomplishment and participation. This hypothesis is not supported at the 95% confidence level. H4 suggests that there is a positive correlation between intrinsic motivation towards stimulation and participation. There is a

Table 3. Paired samples statistics.

M SD Std. error mean

TaxAfter (N-80) TaxBefore (N = 80) 1.8875 1.64581 0.18401

.7625 1.12783 0.12609

Table 4. Paired samples correlations.

Correlation Sig.

TaxAfter and TaxBefore (N = 80) 0.647 0.000

Table 5. Paired samples test.

M N SD t Sig.

TaxAfter – Tax Before 1.12500 80 1.25663 8.007 0.000

1170 P. Buckley and E. Doyle

small, positive correlation between these two variables, r = 0.201, n = 71, p < = .05. H5, that extrinsic motivation of identification is correlated with participation, is also supported with a small positive effect between the two variables, r = 0.0208, n = 75, p < = .05. The data provide no support for either H6 or H7 at the 95% confidence level, although it should be noted that the relationship between external regulation and participation approached sig- nificance, r = 0.180, n = 77, p = .058.

Discussion and conclusions

Practice in education does not remain static. The desire for continuous improvement on the part of practitioners, allied to the continually changing educational context, means that new tools, techniques and methodologies are constantly evolving. The pace of context change has never been faster than it is now. For example, the meteoric rise to ubiquity of infor- mation technology has fundamentally altered how individuals find, evaluate and use infor- mation and knowledge (Kozma, 2003). The advent of the millennial generation, who have radically different learning styles and requirements from previous generations, has pre- sented further challenges (Elam, Stratton, & Gibson, 2007; Howe & Strauss, 2003). In this context, the development of gamification can be seen as an example of the continual renewal of educational practice. While a promising technique, much work must be under- taken before gamification can be considered a mature pedagogical technique. This study begins that work by making two contributions. First, it synthesizes the literature to identify the key elements of gamification. Second, it presents data that begin the process of empiri- cally validating the claims of proponents of gamification.

The empirical data collected by this study raise a number of interesting points. First, H1 is supported. The data show a statistically significant increase in students’ general knowl- edge of the national taxation system after the gamified learning intervention. As mentioned previously, we were careful to ensure that the questions asked were not addressed in class therefore students must have acquired new knowledge from elsewhere. It must be noted that the nature of the intervention is such that we cannot rule out the possibility of confounding variables causing the observed effect. Definitively ascribing an observed effect to an in- class action research intervention is impossible, particularly as this study does not utilize a control group due to practical and ethical constraints. None the less, by prompting stu- dents to engage with the debate on national tax policy by asking them to forecast policy decisions, we believe that it is reasonable to suggest that students’ general literacy in national tax policy is increased. Thus, we feel confident in ascribing some of the observed effect to the gamified learning intervention.

Table 6. Correlations.

Motivation Correlation coefficients between motivation and participation

Intrinsic motivation to knowa 0.194 Intrinsic motivation towards accomplishment 0.108 Intrinsic motivation towards stimulationa 0.201 Extrinsic motivation identificationa 0.0208 Extrinsic motivation introjected 0.122 Intrinsic motivation external regulation 0.180 Amotivation 0.041

aStatistically significant at the 0.05 level.

Interactive Learning Environments 1171

The results of the remaining hypotheses are varied. H2, H4 and H5 are supported, while H3, H6 and H7 are not. Our results show that intrinsic motivation is positively correlated with participation. In general, one would expect students with an inherent desire to learn to be enthusiastic participants in learning activities. The support for H4 demonstrates the power of the gamification. The specific elements of game design included in our gamified learning intervention, such as leaderboards and ranking systems, as well as the inherent uncertainty of forecasting, provide a form of stimulation similar to gambling. Individuals seeking stimulation are likely to find the competitive, uncertain nature of gamified learning interventions exciting and motivating. H5 suggests that students who are extrinsically motivated by identification are also more likely to participate in the market. As a teacher-suggested activity, students who self-identify as good students are likely to engage in behaviours suggested as being appropriate by mentors.

H3, which suggests that intrinsic motivation towards accomplishment is positively cor- related with participation, is not supported. Neither are H6 and H7, both of which suggest that certain types of external motivation are not positively correlated with participation. We are unsure about why the data do not support these hypotheses and believe that further research is needed to explore this in detail. We suggest that it is caused by the dichotomy between the gamified learning environment and more traditional methods of measuring, rewarding and prompting learning. The majority of students are familiar with their edu- cational behaviour being measured and rewarded using “traditional” assessment tools such as projects, essays and end-of-term assessments (Sambell, McDowell, & Montgomery, 2012). Individuals who are motivated intrinsically towards accomplishment are accustomed to deriving satisfaction from these forms of assessment. Similarly, individuals who are motivated extrinsically are accustomed to mediating their behaviour to match the require- ments of such instruments. From anecdotal verbal and written feedback from students, some students had difficulty with some aspects of gamification. Some did not like the com- petitive aspect of gamification. Others felt that it was unfair of teaching faculty to present a problem where there was no “right” answer or correct process to follow. We suggest that gamified learning interventions such as ours diverge too radically from traditional assess- ment mechanisms to actively engage the types of motivations in H3, H6 and H7.

In summary, we believe that this research demonstrates a number of important points which guide the deployment of online gamified learning interventions and suggest further research in the area. First, online gamified learning interventions have a positive impact on learning outcomes. While this is caveated by the acknowledged limitations of this study and the fact that positive results require careful design to ensure that the learning activities prompted by gamification are tied to learning outcomes, this study nevertheless presents a positive picture of the utility of gamification. This study positions gamification as a powerful tool for educators teaching at all levels within the education system. As with the specific example presented in this paper, most gamified learning interventions tend to be web-based, scalable and asynchronous. This makes them particularly useful in educational contexts, such as online learning. Gamified learning activities could become an integral part of flipped teaching environments (Bergmann & Sams, 2012). Their social, asynchronous nature can be used to prompt students to engage with pre-prepared content, while gamified learning activities can be used in the classroom to prompt student interaction and participation.

A key selling point of gamification is that game design elements can be used as a tool to control and generally increase student engagement and participation (Kapp, 2012). Our study provides evidence supporting this notion, with the important caveat that it is not a simple “gamification increases engagement” relationship. Gamification impacts students

1172 P. Buckley and E. Doyle

with different types of motivation differently. Our results demonstrate that it is particularly effective for students who are intrinsically motivated, particularly either by a motivation to know or a motivation towards stimulation. The effect on students who are extrinsically motivated appears to be confined to students who are motivated by identification. This is the form of extrinsic motivation described as being most closely aligned with intrinsic motivation. Overall, the results suggest that gamified learning interventions have a larger impact on students who are intrinsically motivated. This result is not an argument against gamification. In general, it is the best practice for a module to encompass a variety of learning interventions designed to engage a range of learning types, and crucially, to ensure that no-one is disadvantaged in terms of assessment mechanism (Race, 2010). In this context, our study provides empirical evidence that educators can use to inform how and when gamified learning interventions should be used as part of a module.

We aim with this study to prompt further research in this emerging pedagogical area. As befits a relatively novel research domain, confirmatory studies validating our results would be welcome. While we have demonstrated that gamification impacts differently motivated students in different ways, our work does not explore the cause of this effect. Investigating this fertile research domain, perhaps using qualitative approaches, offers great promise in enhancing our understanding of how effective gamified learning interventions should be designed.

Notes on contributors Dr. Patrick Buckley lectures in the area of Information Management in the Department of Manage- ment and Marketing in the Kemmy Business School, University of Limerick, Ireland. Patrick’s research interests include the use of technology to address the challenges of large group teaching. He has received national research funding to pursue this interest. His work in this area has been pub- lished in books such as ‘Technologies for Enhancing Pedagogy, Engagement and Empowerment in Education: Creating Learning-Friendly Environments’ and peer-reviewed journals such as ‘Journal of Teaching in International Business’ and ‘Computers and Education’. He was awarded the national Jennifer Burke award for innovation in Teaching and Learning in 2013.

Dr. Elaine Doyle lectures in taxation in the Kemmy Business School, University of Limerick in Ireland. She designed and is the course director for the Master of Taxation program and was awarded the University of Limerick’s Teaching Excellence award in 2010. In 2012, she spent 3 months as an International Research Scholar with the Open University Business School in Milton Keynes, UK. Her research interests include inter alia, professional ethics and risk management in tax practice, the tax aggression of tax practitioners, research ethics, procedural justice, ethical reason- ing and ethics education. She has published both nationally and internationally in these areas in jour- nals such as ‘Journal of Business Ethics’ and ‘Innovations in Education and Teaching International’ and has secured national and international funding to support her research activities.

References Bergmann, J., & Sams, A. (2012). Flip your classroom: Reach every student in every class every day.

Eugene, Or: International Society for Technology in New York, NY Education. Brophy, J. E. (2013). Motivating students to learn. New York, NY: Routledge. Buckley, P., Garvey, J., & McGrath, F. (2011). A case study on using prediction markets as a rich

environment for active learning. Computers & Education, 56(2), 418–428. doi:10.1016/j. compedu.2010.09.001

Cokley, K. O., Bernard, N., Cunningham, D., & Motoike, J. (2001). A psychometric investigation of the academic motivation scale using a United States sample. Measurement and Evaluation in Counseling and Development, 34(2), 109–119.

Deci, E. L., Koestner, R., & Ryan, R. M. (2001). Extrinsic rewards and intrinsic motivation in edu- cation: Reconsidered once again. Review of Educational Research, 71(1), 1–27. doi:10.3102/ 00346543071001001

Interactive Learning Environments 1173

Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to gameful- ness: defining “gamification.” Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments (pp. 9–15). New York, NY: ACM. doi:10.1145/2181037.2181040

Domínguez, A., Saenz-de-Navarrete, J., de-Marcos, L., Fernández-Sanz, L., Pagés, C., & Martínez- Herráiz, J.-J. (2013). Gamifying learning experiences: Practical implications and outcomes. Computers & Education, 63, 380–392. doi:10.1016/j.compedu.2012.12.020

Elam, C., Stratton, T., & Gibson, D. D. (2007). Welcoming a new generation to college: The millen- nial students. Journal of College Admission. Retrieved from http://www.eric.ed.gov/ ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=EJ783953

Ellis, C. M., & Sami, R. (2012). Learning political science with prediction markets: An experimental study. PS: Political Science & Politics, 45(2), 277–284. doi:10.1017/S1049096511002113

Evans, A. J. (2012). Pedagogical synergies between Austrian economics and the case method. The Journal of Private Enterprise, 28(1), 91–103.

Fairchild, A. J., Horst, S. J., Finney, S. J., & Barron, K. E. (2005). Evaluating existing and new val- idity evidence for the academic motivation scale. Contemporary Educational Psychology, 30(3), 331–358. doi:10.1016/j.cedpsych.2004.11.001

Gartner Says By 2015, More Than 50 Percent of Organizations That Manage Innovation Processes Will Gamify Those Processes. (2011). Gartner Newsroom. Retrieved November 7, 2013, from http://www.gartner.com/newsroom/id/1629214

Garvey, J., & Buckley, P. (2010). Teaching the concept of risk: Blended learning using a custom-built prediction market. Journal of Teaching in International Business, 21(4), 346–357. doi:10.1080/ 08975930.2010.526032

Glover, I. (2013). Play as you learn: Gamification as a technique for motivating learners. In: J. Herrington, A. Couros & V. Irvine (Eds.) Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications (Vol. 2013, pp. 1999–2008). Chesapeake, VA: AACE.

Harlen, W., & Deakin Crick, R. (2003). Testing and motivation for learning. Assessment in Education: Principles, Policy & Practice, 10(2), 169–207. doi:10.1080/0969594032000121270

Howe, N., & Strauss, W. (2003). Millennials go to college: Strategies for a new generation on campus. Washington, DC: American Association of Collegiate Registrars and Admissions Officers.

Kapp, K. M. (2012). The gamification of learning and instruction: Game-based methods and strat- egies for training and education. San Francisco, CA: John Wiley & Sons.

Kim, B. (2012). Harnessing the power of game dynamics why, how to, and how not to gamify the library experience. College & Research Libraries News, 73(8), 465–469.

Kozma, R. B. (2003). Technology, innovation, and educational change: A global perspective : A report of the second information technology in education study, Module 2. ISTE (Interntl Soc Tech Educ.

Lee, J., & Hammer, J. (2011). Gamification in education: What, how, why bother? Academic Exchange Quarterly, 15(2), 146.

M2 Research – Gamification. (2013). M2 Research. Retrieved November 7, 2013, from http:// m2research.com/Gamification.htm

Maehr, M. L., & Meyer, H. A. (1997). Understanding motivation and schooling: Where we’ve been, where we are, and where we need to go. Educational Psychology Review, 9(4), 371–409. doi:10. 1023/A:1024750807365

McGonigal, J. (2011). Reality is broken: Why games make us better and how they can change the world. London: Penguin.com.

Nicholson, S. (2012, June). A user-centered theoretical framework for meaningful gamification. Paper presented at Games+Learning+Society 8.0, Madison, WI.

Piaget, J., & Cook, M. (1952). The origins of intelligence in children. New York, NY: W W Norton. Raban, D., & Geifman, D. (2010). Designing online information aggregation and prediction markets

for MBA courses. Interdisciplinary Journal of E-Learning and Learning Objects, 5, 247–262. Race, P. (2010). Making learning happen: a guide for post-compulsory education. London: SAGE. Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic definitions and new

directions. Contemporary Educational Psychology, 25(1), 54–67. doi:10.1006/ceps.1999.1020 Salen, K., & Zimmerman, E. (2004). Rules of play: Game design fundamentals. Boston: MIT Press.

1174 P. Buckley and E. Doyle

Sambell, K., McDowell, L., & Montgomery, C. (2012). Assessment for learning in higher education. Milton Park: Routledge.

Simões, J., Redondo, R. D., & Vilas, A. F. (2013). A social gamification framework for a K-6 learning platform. Computers in Human Behavior, 29(2), 345–353. doi:10.1016/j.chb.2012.06.007

Skinner, B. F. (1976). About behaviorism (1st ed.). New York, NY: Vintage. Smith-Robbins, S. (2011). This game sucks”: How to improve the gamification of education.

Educause Review, 46(1), 58–59. Squire, K. (2003). Video games in education. International Journal Intelligent Games & Simulation,

2(1), 49–62. Tziralis, G., & Tatsiopoulos, I. (2007). Prediction markets: An extended literature review. The Journal

of Prediction Markets, 1(1), 75–91. Vallerand, R. J., Pelletier, L. G., Blais, M. R., Briere, N. M., Senecal, C., & Vallieres, E. F. (1992). The

academic motivation scale: A measure of intrinsic, extrinsic, and amotivation in education. Educational and Psychological Measurement, 52(4), 1003–1017. doi:10.1177/ 0013164492052004025

Zichermann, G., & Cunningham, C. (2011). Gamification by design: Implementing game mechanics in web and mobile apps. Sebastopol, Canada: O’Reilly Media.

Interactive Learning Environments 1175