article summary 3

Journal of School Psychology 51 (2013) 187–199

Contents lists available at SciVerse ScienceDirect

Journal of School Psychology

j ourna l homepage: www.e lsev ie r .com/ locate / j schpsyc

The role of children's on-task behavior in the prevention of aggressivebehavior development and peer rejection: A randomizedcontrolled study of the Good Behavior Game in Belgianelementary classrooms

Geertje Leflot a,⁎, Pol A.C. van Lier b, Patrick Onghena a, Hilde Colpin a

a University of Leuven (K.U.Leuven), Belgium, Faculty of Psychology and Educational Sciences, Tiensestraat 102, 3000 Leuven, Belgium b VU University Amsterdam, the Netherlands, Department of Developmental Psychology, De Boelelaan 1105, 1081 HV Amsterdam, Netherlands

a r t i c l e i n f o

⁎ Corresponding author at: Department of Applied fax: +32 3 241 08 20.

E-mail addresses: [email protected] (G. Leflot), (H. Colpin).

ACTION EDITOR: Renee Hawkins.

0022-4405/$ – see front matter © 2013 Society for the http://dx.doi.org/10.1016/j.jsp.2012.12.006

a b s t r a c t

Article history: Received 1 April 2011 Received in revised form 22 December 2012 Accepted 26 December 2012

The role of children's on-task behavior in the prevention of aggressive behavior was assessed among 570 Dutch speaking children followed from second- to third-grade elementary school in Flanders, Belgium.A first objectivewas to investigatewhether individual level variation of on-task behavior moderated the impact of a universal preventive intervention, the Good Behavior Game (GBG), on aggression development, controlling for classroom levels of on-task behavior. The second goal was to study whether improved on-task behavior or reductions in peer rejection mediated intervention impact on children's aggression. Second-grade classroomswere randomly assigned to the GBG or a control condition. Results showed that intervention impact was found only among children who had low-level on-task behavior at baseline. These children showed a decrease in aggression when in the GBG condition, which was not found among control group children. The reduction in aggression among low on-task children was mediated by reductions in peer rejection. No mediation effect of on-task behavior was found. These results suggest that the effect of a universal preventive intervention may depend upon initial levels of on-task behavior and that improvements in social relations with peers may explain the reductions in aggression among these low-on task children. © 2013 Society for the Study of School Psychology. Published by Elsevier Ltd. All rights reserved.

Keywords: On-task behavior Aggression Peer rejection Prevention Good Behavior Game

1. Introduction

Although many factors may influence student success in elementary school, one of the key factors in children's successful classroom adjustment is their on-task behavior. Children who fail in this central skill are at risk for poor academic achievement (e.g., Ladd, Buhs, & Seid, 2000; Lan et al., 2009), negative interactions with peers and teachers (e.g., Bellanti, Bierman, & Conduct Problems Prevention Research Group, 2000; Hughes & Kwok, 2006; Skinner & Belmont, 1993), and psychopathology, such as aggressive behavior development (e.g., Bellanti et al., 2000; Harachi, Fleming, White, & Ensminger, 2006). Given these findings, children who display low levels of on-task behavior may be an important target group for prevention effort. However, evidence on the role of on-task behavior for children's classroom adjustment has mainly come from observational studies and not from intervention studies. The present study focused on the role of children's on-task behavior in the prevention of aggressive behavior over grades 2 and 3 of elementary school. Two questions were addressed. The first was whether children's baseline levels of on-task behavior acted as a moderator of the effect of a universal preventive intervention, the Good Behavior Game intervention (GBG; Barrish, Saunders, & Wolf, 1969; Dolan, Turkkan, Werthamer-Larsson, & Kellam, 1989; Kellam, Reid, & Balster, 2008), on

Psychology, Lessius Antwerp, Jozef De Bomstraat 11, 2018 Antwerp, Belgium. Tel.: +32 3 241 08 29;

[email protected] (P.A.C. van Lier), [email protected] (P.Onghena),[email protected]

Study of School Psychology. Published by Elsevier Ltd. All rights reserved.

188 G. Leflot et al. / Journal of School Psychology 51 (2013) 187–199

aggressive behavior development. The second question focused on the processes through which children with low baseline levels of on-task behavior who received the GBG showed reductions in aggression.

1.1. On-task behavior and aggressive behavior development

Classroom on-task behavior, also referred to as academic behavior and classroom participation, concerns involvement in and attention and concentration for academic tasks in the classroom (Skinner, Kindermann, & Furrer, 2009). On-task behavior is observable through active behaviors, such as asking or answering questions, and passive behaviors, such as looking at the teacher who is giving an instruction (see Lan et al., 2009).

The capacity to stay on-task has been shown to be important not only for children's learning (e.g., Ladd et al., 2000; Lan et al., 2009) but also for children's social and behavioral adjustment. More specifically, children with low on-task behavior in kindergarten and elementary school have been shown to have higher average levels of aggressive behavior (Bellanti et al., 2000; Harachi et al., 2006; Morrison, Robertson, Laurie, & Kelly, 2002) and to follow a developmental trajectory with the average level of aggression increasing over time (Schaeffer, Petras, Ialongo, Poduska, & Kellam, 2003). Thus, children with low levels of classroom on-task behavior are at risk for increased levels and further development of aggression.

1.2. On-task behavior as a moderator of the Good Behavior Game impact

A moderator is a variable that “affects the direction and/or the strength of the relationship between an independent variable and the dependent variable” (Baron & Kenny, 1986, p. 1174). It is a variable that specifies under which conditions an effect, such as an intervention effect, occurs (Holmbeck, 1997). The Good Behavior Game (GBG) is a universal classroom team-based prevention program designed to improve elementary school children's behavior. The GBG aims to promote prosocial behavior while discouraging antisocial behavior in the classroom (e.g., Dolan et al., 1989). Randomized controlled intervention trials have shown that the GBG is effective in reducing children's disruptive behavior problems, including aggression (e.g., Dolan et al., 1993; Kellam, Rebok, Ialongo, & Mayer, 1994; van Lier, Muthén, van der Sar, & Crijnen, 2004; van Lier, Vuijk, & Crijnen, 2005; Witvliet, van Lier, Cuijpers, & Koot, 2009) and associated problems such as alcohol abuse and depression (Kellam et al., 2008; Vuijk, van Lier, Crijnen, & Huizink, 2007). However, as with other universal interventions, the GBG impact on reductions in aggression has been shown to be stronger for, or limited to, children who were characterized by risk factors for the development of aggression at baseline (e.g., Dolan et al., 1993; Kellam et al., 1994; van Lier et al., 2005; see also Coie et al., 1993; Flay et al., 2005). In particular, the study by Rebok, Hawkins, Krener, Mayer, and Kellam (1996), showed that after one year of intervention, the GBG was most effective in reducing aggression among those children who had difficulty in on-task behavior, such as concentration problems. However, the study of Rebok et al. (1996) did not take the classroom level of on-task behavior into account. Classes as a whole may be off-task, and therefore the effects of intervention on individual students could be accounted for by classroom effects.

Although not focusing on on-task behavior, there is some evidence that the classroom level of aggression moderated the effect of the GBG on children's aggressive behavior development (Kellam, Ling, Merisca, Brown, & Ialongo, 1998). For the control group, results showed that if initially aggressive boys were in a highly aggressive first-grade classroom, they had a much higher chance of being highly aggressive at age 12 than initially aggressive boys in a less aggressive classroom. This finding was less marked among GBG classrooms. Specifically, if aggressive boys had been in an initially highly aggressive GBG classroom, as compared to a highly aggressive control classroom, their chances of being highly aggressive at age 12 years were lower. Although the statistical power to attribute these results to the GBG was low, the findings suggested that classroom levels of aggression acted as a moderator of the impact of the GBG in reducing children's risks for aggressive behavior at age 12. As on-task behavior is linked to aggression (Bellanti et al., 2000; Harachi et al., 2006), it may well be that classroom levels of on-task behavior, and not individual variation of on-task behavior, act as the moderator of intervention impact.

When studying the possible moderating role of on-task behavior, it is important to determine if it is individual level variation in on-task behavior that moderates intervention impact when controlling for the classroom level variation. If individual level variation in on-task behavior moderates the GBG impact on aggression, it implies that changes at the individual level, regardless of group or classroom level, predicted these improvements. Hence, it is then suggested to search for mechanisms of change at the individual level. The first aim of the present study was therefore to examine whether children's on-task behavior acted as a moderator of the effect of the GBG on the development of aggression over the second and third grade, while controlling for mean classroom levels of initial on-task behavior.

1.3. Mediators of the effect of the Good Behavior Game

The second research question pertains to the pathways through which children with low levels of on-task behavior may reduce their aggressive behavior. Whereas moderation provides information about for whom an intervention is effective, mediation of the moderation effect identifies how the intervention is effective for a specific subgroup (Morgan-Lopez & MacKinnon, 2006). More specifically, a mediator is a variable that “accounts for the relation between the independent variable and the dependent variable” (Baron & Kenny, 1986, p. 1176). In other words is specifies how (or the mechanism by which) an effect such as an intervention effect occurs (Holmbeck, 1997). The GBG is a behavioral management intervention that, via structuring of the classroom, and by facilitating positive peer interactions, aims to reduce children's behavioral problems. Given this, the present study focused on two pathways through which the GBG may reduce aggressive behavior.

189G. Leflot et al. / Journal of School Psychology 51 (2013) 187–199

The first possible pathway is through improving on-task behavior. TheGBG focuses on improving children's classroombehavior by explicitly defining appropriate on-task and prosocial behaviors, which arewritten into positively formulated classroom rules (e.g., “In the classroom, we work quietly.”), and accompanied by pictograms. Children's compliance to these rules is praised by the teacher (e.g., “You are doing a great jobworking quietly today!”). Rulesmay differ according to the specific tasks children have to complete or the lessons being taught. Via the resulting improved classroom structure, and in order to comply with the classroom rules, it is expected that GBG children will improve their classroom behavior, and especially increase their on-task behavior (van der Sar & Goudswaard, 2001). Some small-scaled studies showed that the GBG indeed improved children's on-task behavior (e.g., Fishbein & Wasik, 1981; Lannie &McCurdy, 2007). However, no studies to date have shown that improved on-task behaviorsmediated the effect of the GBG on aggression. Thus, in addition to acting as a moderator, the improvement in on-task behavior during the intervention may act as mediator of the effect of the GBG on aggression among children.

A second pathway through which the GBG may decrease children's aggression is by improving children's peer relations (van Lier et al., 2005; Witvliet et al., 2009). Children with low on-task behavior are at risk for poor relations with peers (Bellanti et al., 2000). When children enter elementary school, classmates immediately start to evaluate their peers (see Dishion, Patterson, & Griesler, 1994; Dodge, Coie, & Brakke, 1982). Children who do not live up to the standards of the mainstream peer group, such as low on-task children, have a high risk of becoming disliked (Boivin, Vitaro, & Poulin, 2005; Deater-Deckard, 2001; van Lier & Koot, 2008). When children are disliked by the majority of classmates, these children are seen as actively rejected by peers (Cillessen, 2009). Rejected children are likely to be deprived of contact with mainstream peers. As a result of the limited social interactions with mainstream peers, these children receive little social correction and guidelines for their behavior, which may facilitate, maintain, or exacerbate aggression over time (Keiley, Bates, Dodge, & Pettit, 2000; Ladd, 2006; Sturaro, van Lier, Cuijpers, & Koot, 2011; see overviews by Deater-Deckard, 2001; Parker, Rubin, Erath, Wojslawowicz, & Buskirk, 2006; Rubin, Bukowski, & Parker, 2006).

In the GBG intervention, children are assigned to mixed teams of children who either engage or do not engage in disruptive classroom behaviors. Team members are encouraged to work together and to reinforce each other in behaving appropriately. Teams as a whole are rewarded for complying with the classroom rules, and teachers praise teams and individual children for their good behavior. Because of the focus on teams and as teams change regularly, it is likely that the GBG will promote a sense of community within the teams and classrooms while teaching and providing for positive, prosocial peer interactions in a context that is beneficial for the self and others. A recent study showed that the reductions in boys' oppositional and conduct problems were mediated by improved peer acceptance as found among GBG children (Witvliet et al., 2009). In sum, the second aim of the present study was to examine whether improvements in on-task behavior, improvements in peer relations, or both types of improvements acted as mediators in the effect of the GBG on aggressive behavior development among those children with low baseline levels of on-task behavior.

1.4. The present study

The present study investigated the role of on-task behavior in the effect of the GBG on aggressive behavior development. It was hypothesized that initially low on-task GBG children would show a greater reduction in their aggression, compared to GBG children who did not have difficulty in on-task behavior at baseline and compared to control children. It was also hypothesized that this effect would hold even when controlling for the mean classroom level of initial on-task behavior. With regard to the mediation pathways, it was hypothesized that among GBG children with low baseline levels of on-task behavior, improvements in on-task behavior and peer relations would explain why these children showed reductions in aggressive behavior, as compared to low on-task control group children. All analyses will be controlled for possible influences of sex, as boys have been found to have higher levels of aggressive behavior (Moffitt, Caspi, Rutter, & Silva, 2001), lower levels of on-task behavior (Birch & Ladd, 1997; Zimmer-Gembeck, Geiger, & Crick, 2005), and more social problems than girls (Rubin et al., 2006).

2. Method

2.1. Participants

Fifteen schools, all located in rural to moderately urban communities (with populations ranged from about 9000 to 90,000) in the Flemish speaking part of Belgium (General Direction Statistics and Economical Information, 2004) participated in the study. Children were followed from the start of the second grade (September 2006) until the end of the third grade (June 2008). Each school had two second-grade classes, for a total of 30 classes. All children in these classrooms were eligible for inclusion. Written parental permission was obtained for 570 children (97%). Approximately half of the children (49%) were boys. At the beginning of the second grade (wave 1), children's mean age was 7 years and 5 months (SD=4.6 months). The majority of the children and their parents were White (Caucasian) and were of Belgian nationality (>95%). Most parents completed higher education (63% of mothers, 57% of fathers). The remaining parents finished high school (28% of mothers, 30% of fathers) or completed primary school (9% of the mothers and 13% of the fathers).

With regard to the teachers, two teachers in the second grade and two teachers in the third grade were men. All other 26 teachers were women. The mean age and the mean teaching experience of the teachers of the second grade was 35.91 years old (SD=9.41; range 21–52 years) and 13.20 years (SD=9.14; range: 0–33 years), respectively. The mean age and mean teaching

190 G. Leflot et al. / Journal of School Psychology 51 (2013) 187–199

experience of the teachers in third grade was 41.00 years old (SD=8.15; range 25–55 years) and 17.82 years (SD=7.34; range 3–28 years), respectively.

Over second and third grade, the implementation period of the GBG, classroom composition stayed intact. Only the teachers changed from the second to the third grade, which is usual in the Flemish educational system. During the course of the study, 41 children were dropped from the study because of grade retention, or because of moving away from school. These children had higher initial levels of peer-rated aggression, t(568)=−3.09, p=.002, and peer rejection, t(568)=4.31, pb .0001. They did not differ in initial level of observed on-task behavior, t(568)=−0.74, p=.46. A flow-chart of the sample over the intervention period is given in Fig. 1.

2.2. Design

An intervention design with randomized intervention and control groups and repeated measures was developed to answer the research questions. A randomized block design was employed, with classes being randomly assigned to the condition. Before intervention implementation, within each school, the two classes of the second grade (with teacher and students) were randomly assigned to the GBG (n=287) or the control condition (n=283), resulting in one GBG and one control class per school. No statistically significant differenceswere found between students in the control and GBG conditions based on sex, nationality, parental education level, and number of children lost during the course of the study. For the 2006−2007 school year, the GBG was implemented in the second grade intervention classes, according to the Dutch version of the program (cf. van Lier et al., 2004). With the transition to third grade, school directors were asked to maintain the composition of the control and intervention classrooms, which was checked (through class composition lists) and approved by the research team at the start of the third grade. During the 2007−2008 school year, the GBG continued to be implemented in the same intervention classes (now in the third grade).

Data collection occurred in the GBG and control classes at four points in time: prior to intervention implementation at the beginning of the second grade (wave 1, pre-test, September/October 2006), at the end of the second grade (wave 2, May/June 2007), at the beginning of the third grade (wave 3, September/October 2007), and at the end of the third grade, after terminating the intervention (wave 4, post-test, May/June 2008). Data were collected in the school setting. Behavioral observations of children's levels of on-task behavior at baseline (wave 1) and wave 4 were used. Peer rejection was assessed at baseline and wave 4 as well, and children's aggression was assessed at each of the four data collection waves. For the assessment of peer rejection and aggression (and other issues not relevant for the present article) interviews were conducted according to a standardized script, including standardized instruments (e.g., peer nominations), by the researchers who conducted the observations. The interviews started when the observation procedure had been completed. Each child was interviewed individually by one of the research team members in a quiet room on the school grounds during the school hours. To this aim, each child was taken out of the classroom for 10 min on average. The behavioral observations and the interviews lasted up to a day and a half per classroom.

This study was approved by the ethical review board of the Faculty of Psychology and Educational Sciences of the KU Leuven, University of Leuven and by the commission on medical ethics of the Academic Hospitals in Leuven.

2.3. Measures

2.3.1. On-task behavior Children's on-task behavior was observed by two trained observers using an instrument developed by van der Sar (2004). The

observations were scheduled in the morning, during non-GBG moments when children were in a mathematics lesson, a language

September 2006 Second grade, 30 classes (2 classes/school), N = 587

September 2006 Second grade, 30 classes randomly assigned to the GBG- or control-group,

N (signed parental permission) = 570

Control-group 15 2nd grade classes, n = 283

GBG-group 15 2nd grade classes, n = 287

Control-group 15 3rd grade classes, n = 264

Control-group 15 3rd grade classes, n = 265

Lost over 2nd (2006-2007) and 3rd grade (2007-2008), n = 41

Fig. 1. Flow-chart of classes and participants in the randomized controlled trial. GBG = Good Behavior Game.

191G. Leflot et al. / Journal of School Psychology 51 (2013) 187–199

lesson, or across both types of lessons, making it possible to observe generalization effects. They were conducted by two trained members of the research team. Prior to the observation, the teacher made a map of the classroom, indicating where each child was seated. In addition the teacher was given an observation design, so that all teachers would structure their lesson in the same way during the observation period. Observers recorded the behavior of each child six times during one observation session. At each of the observation sessions, the on-task behavior of the child was observed during exactly 20 s, followed by a 10 s recording interval. The time was kept by using a stopwatch. Then, the next child was observed until all children in the classroom were observed. Then, the next round of observations started (total of six rounds over the morning). For on-task behavior (e.g., listening to the teacher or doing assignments, not looking around, yawning, making grand gestures, or fidgeting), the child received a score of 0 (not on-task at all during the whole interval of 20 s), 1 (not on-task during 10 s or more), 2 (not on-task during less than 10 s) or 3 (on-task during the whole interval of 20 s).

Before the start of the study, the two observers were trained during multiple training sessions. During these training sessions the observation procedure was studied and discussed. The observation procedure was practiced using videotapes of classroom situations. These videotapes contained video recordings of morning classroom situations when children were in mathematics and language lessons. They were especially made for the purpose of training observers in this observation procedure and determining the interobserver agreement. After each training session, interobserver agreement was calculated. This agreement score was the ratio of the two observers giving exactly the same score for one child during one observation interval (e.g., both 3) divided by the total number of observation intervals for all children and multiplied by 100. The criterion for percentage agreement was 80%, which was reached by the observers during these training sessions. Prior to data collection, the two observers also rated children's classroom behavior simultaneously during live classroom situations. The mean percentage of agreement was 71% for on-task behavior across all live training sessions. Before the beginning of each data collection wave, re-training of the observers occurred using videotapes of classroom sessions and interobserver agreement was calculated during exercise sessions. Retraining exercises were concluded when interobserver agreement levels of 80% or higher were reached.

Based on the observation of their baseline on-task behavior at wave 1, children were identified as eithermoderate/high on-task or low on-task. The grand mean score and standard deviation of on-task behavior in the studied sample was calculated. Children who had an observed individual score that was at least 1 standard deviation lower than the grand mean were assigned to the low on-task group (dummy coded as 1). All other children were assigned to the moderate/high on-task group (dummy coded 0). This resulted into 87 (55.2% boys) and 475 (48.4% boys) children being assigned to the low on-task group and moderate/high on-task group, respectively. Eight children could not be assigned to either group as they were absent at wave 1. The cut-off score used is similar to that of other studies investigating the role of high and low engagement and behavior problems in the development of aggression (e.g., Bellanti et al., 2000).

2.3.2. Aggression Children's aggression was rated by peers (see van Lier, 2002). Children were asked to nominate all children in the classroom

who met the behavioral description “Sometimes hits children” (measuring aggression). The answering procedure was facilitated by a list of all the names of the children in the classroom, which children were encouraged to use during the interview. For each child, the number of nominations on the question was added up and divided by the number of children in the class minus one (nominating oneself was not allowed). A nomination score of 1 means that all classmates nominated this child. The intraclass correlations of the aggression scores were .02, .01, .02, and .03 over waves 1 to 4 respectively. The internal consistency reliability coefficient (using Kuder–Richardson formula 20; KR-20) was .85 for aggression (see also Cillessen, 2009; Terry, 2000). Test–retest reliability coefficient of the control group over measurement wave 1 and 2 for aggression was .75.

2.3.3. Peer rejection Peer rejection was assessed by asking children to nominate all classmates that they liked least, using the protocol delineated

by Coie, Dodge, and Coppotelli (1982). The answering procedure was facilitated by a list of all the names of the children in the classroom, which children were encouraged to use during the interview. The total number of negative nominations (LL) was divided by the number of children in the class minus one (nominating oneself was not allowed). This score indicated the level of peer rejection (Cillessen, 2009). The higher the child's peer rejection score, the more the child was rejected by classmates. A nomination score of 1 means that all classmates nominated this child. The internal consistency reliability coefficient (using KR-20) was .81 for peer rejection (see also Cillessen, 2009; Terry, 2000). Test-retest reliability coefficient of the control group over measurement wave 1 and 4 for peer rejection was .60.

2.3.4. Sex and intervention status Sex and intervention status were dummy coded (0 = girl, 1 = boy and 0 = control group, 1 = GBG, respectively).

2.4. Intervention procedures

The Dutch adaptation of the GBG (van der Sar & Goudswaard, 2001) was implemented, as Dutch is the official language in Flanders, Belgium. It is also the mother tongue of the vast majority of the children in this study. The GBG is a classroom team-based preventive intervention aimed at improving prosocial and reducing antisocial behavior. Before implementing the GBG, children's behavior is observed by the classroom teacher and rules for appropriate behavior are formulated. Based on the behavioral observations, children are seated in teams of four to five members, including a mix of students who display relatively

192 G. Leflot et al. / Journal of School Psychology 51 (2013) 187–199

higher levels of disruptive behavior and students who display appropriate classroom behavior. Together, teacher and children design positively formulated rules for desired classroom behaviors (e.g., “We let our classmates work in peace” and “During work we are quiet”), which are visualized in the classroom by pictograms. During the GBG session, on-task and prosocial behavior is reinforced through verbal praise (e.g., “Wow, you are doing a great job sitting quietly!”) of teams and individual children who follow predefined rules. Each team receives a number of cards, and one of the team's cards is removed if a team member violates one of the predefined rules. No further attention is given to the disruptive behavior. Instead, attention is given to the prosocial behavior of teammates via praise. In addition to verbal praise, the teacher rewards the teams again, when at the end of the session, at least one card remains.

The Dutch version of the GBG is different on some aspects from the United States version of the GBG (Dolan, Turkkan, Werthamer-Larsson, & Kellam, 1989). First, the classroom rules in the Dutch version are always positively formulated. Second, teachers do not mention the children who violated GBG rules, nor do the teams get penalty points when a teammember violates a rule. Third, in the Dutch version, teams do not compete for rewards. All teams having cards left at the end of the game receive a reward. Finally, in the Dutch version, children in teams are encouraged to actively support each other in behaving appropriately.

The GBG was implemented during two consecutive grades (second and third grade of elementary school). In each of the grades, the GBG was implemented frommid-October to mid-May, leaving enough time for data collection in the beginning and at the end of the school year (September and June). The GBG has three implementation phases, an introduction, an expansion, and a generalization stage. The three phases are implemented within a school year, according to the Dutch implementation manual of the game (van der Sar & Goudswaard, 2001). In each of these phases, the GBG is played during three sessions per week. In the introduction phase (3 months), the playtime of the GBG is gradually increased from 10 min per session, up to 45 min per session. The goal is to allow teacher and pupils become acquainted with the GBG. In the expansion phase (next 3 months), the teacher further expands the duration of the game up to half a day per session. Finally, during the generalization phase (last 2 months of the school year), the GBG is still played three times a week; however, attention is focused on promoting generalization of prosocial behavior outside GBG sessions by encouraging the teacher to praise positive behavior and pay less attention to negative behavior outside GBG sessions as well. In the second and the third grade, all three phases were implemented.

2.4.1. Teacher training and adherence At the start of each phase, teachers receive a half-day group training by a trained school consultant. Manuals are available for

teachers and school consultants. Moreover, teachers are supervised and supported by the same school consultant during ten 60-min standardized classroom observations during each implementation year (van der Sar & Goudswaard, 2001). In our study, however, teacherswere visited by the school consultant on 8 instead of 10 occasions on average due to practical circumstances. One second grade teacher was ill for several periods of time during the school year; in this classroom, the expansion phase was not completed in second grade. All other second- and third-grade GBG teachers implemented the three implementation phases as planned: the introduction phase from the beginning of October to the end of December, the expansion phase from the beginning of January to the end of March, and the generalization phase from the beginning of April to the end of May. Via standardized observations, the school consultant rated the implementation quality on six categories indicating crucial GBG components. The findings on these standardized observationswere also used to provide the teacherwith data-based feedback concerning their implementation quality. Each category is given a score from 0 to 2, with higher scores indicating better implementation (range total score=0 to12). Examples of categories are “Does the teacher consistently remove a card when a child violates a rule?” (“Yes”=2 points; “Yes, but also for rules that were not explicitly mentioned before the start of the game”=1 point; “No, not consistently”=0 points) and “Does the teacher compliment individual children and teams?” (“Yes, teams and individual children”=2 points; “Yes, but only to teams or to individual children”=1point; “No”=0points). The mean score over the two school years and over the classrooms was 9.21 (SD=1.38). This score indicates that our teachers, on average, did not implement the GBG perfectly, which may be expected in naturalistic settings (Durlak & DuPre, 2008). More importantly, it reflects the frequently found fluctuation of the quality of implementation of programs in a naturalistic setting (Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005). An intent-to-treat approach was used in the further analyses, meaning that data were treated to conform to the intervention status as reflected by the randomization, regardless of actual levels of implementation of the program. An intent-to-treat approachwill give a pragmatic estimate of the effect of theGBGon aggression in a “real-life” situation rather than of the potential effect of the GBG on aggression in children who received the GBG perfectly as planned (Hollis & Cambell, 1999).

To minimize treatment confounding between GBG and control classes, the research team repeatedly explained why control teachers could not be informed about the GBG and frequently encouraged GBG teachers to withhold all GBG information from control teachers. It was clear from conversation with control teachers that, although they were aware of the goals of the program, they did not have specific knowledge about the GBG instructions, nor did they implement the GBG or parts of the program.

2.5. Overview of data analyses

The analyses were conducted in two stages. We first tested the role of on-task behavior as a possible moderator of intervention impact. Specifically we tested whether individual variation in baseline levels of on-task behavior moderated the effect of the GBG on the development of aggression, while controlling for the mean classroom level of on-task behavior at baseline. A multilevel growth model was fitted in Mplus 5.21 (Muthén & Muthén, 1998-2009). An MLR estimator was used to control for possible non-normality of the data. Three levels were identified with variation across time at level 1, variation across individuals at level 2, and variation across classes at level 3. The development of children's aggression over the four assessments was captured by latent growth parameters, in which the intercept reflects initial level differences (at baseline) and the slope reflects change over time.

193G. Leflot et al. / Journal of School Psychology 51 (2013) 187–199

The growth parameters were regressed on sex and the individual level of on-task behavior at the individual level (within level or level 2) and regressed on intervention status and the latent classroommean of children's baseline level of on-task behavior at the classroom level (between level or level 3). A graphical representation of the tested model is given in Fig. 2.

To test whether individual level variation in on-task behavior (level 2 variable) moderated GBG impact on growth of aggression, a random slope parameter was considered to reflect a cross level (classroom-to-individual level) interaction variable. Modeling this random slope provides a test of whether the effect of children's on-task behavior on the individual level growth parameters could vary across classrooms. The random slope parameter was regressed on intervention status and classroom level of on-task behavior (level 3). A significant effect of intervention status on this random slope parameter indicates that the effect of individual level variation in children's on-task behavior on the growth parameters of aggression depends on GBG status, regardless of possible classroom level effects of on-task behavior.

In the second stage, we focused on the possible mediation of improvements in on-task behavior and reductions in peer rejection of the impact of the intervention on the development of aggression. We tested for individual level mediation among initially low on-task children, given that the GBG impact on the growth of aggression is dependent on individual level variation of on-task behavior at baseline. To test for the role of improvements in on-task behavior and reductions in peer rejection as possible mediators of the effect of the GBG on aggressive behavior development among low on-task children (n=87), the wave 1 and 4 scores of on-task behavior (peer rejection) were added to the latent growth model of aggression. In these models, the wave 4 value of the mediator was regressed on its wave 1 value and sex. Mediation was tested for (1) by regressing the growth factors of aggression on intervention status, (2) by regressing the wave 4 value of on-task behavior (peer rejection) on intervention status, and (3) by regressing the slope of aggression on on-task behavior (peer rejection) at wave 4, controlling for intervention status. Mediation was tested for by calculating the significance of the pathway that comprised the indirect effect (Mackinnon, Lockwood, & Williams, 2004). To control for the multilevel design of the data, all standard errors were adjusted for classroom level variation by using a sandwich estimator (Williams, 2000). To test whether the models represented the associations well in the observed data, the standardized root mean square residual (SRMR; Browne & Cudeck, 1993), comparative fit index, and the Tucker Lewis index (TLI; Bentler, 1990) were used. A good fit is indicated by SRMR≤ .10, CFI≥ .90, and TLI≥ .90

All models were analyzed with Mplus 5.21 (Muthén & Muthén, 1998-2009). The Mplus missing data module was used to handle missing data.

3. Results

3.1. Descriptive statistics

Means and standard deviations for on-task behavior, aggression, and peer rejection for children with moderate/high and low levels of on-task behavior at baseline in GBG and control classes are presented in Table 1. An alpha of .05 was used for all tests of significance. No baseline differences between the GBG and control classes were found on any of the study variables. However, at

First grade Second grade

Within

Between Agg1 Agg2 Agg3

Ibetw Sbetw

GBG Classroom on-task

Agg4

Agg1 Agg2 Agg3

Iwithin Swithin

Sex Individual on-task

Agg4

S

s s

Fig. 2. Graphical representation of themulti-level model with random slope to test formoderation of the GBGon the influence of individual level on-task behavior on the development of aggression. At thewithin level the intercept and slope of aggression are regressed on sex and individual level of on-task behavior. At the between level the intercept and slope are regressed on intervention status and the latent classroom mean of children's baseline level of on-task behavior. The random slope (S) denotes a cross level (classroom-to-individual level) interaction variable. Within level = variation across individuals; between level = variation across classes. GBG = Good Behavior Game. Iwithin = intercept at within level. Swithin = slope at within level. Ibetw = intercept at between level. Sbetw = slope at between level. S = random slope.

194 G. Leflot et al. / Journal of School Psychology 51 (2013) 187–199

baseline, low on-task children in control and GBG classes had significantly higher aggression (control: t(277)=3.00, p=.03, d=0.48 and GBG: t(281)=1.96, p=.05, d=0.30) and peer rejection scores (control: t(277)=2.32, p=.02, d=0.33 and GBG: t(281)=1.78, p=.08; d=0.29) compared to moderate/high on-task children.

At wave 4, low on-task GBG children had statistically significantly lower levels of aggression than students in the control group who also had low levels of on-task behavior at baseline (see Table 1). Also at wave 4, low on-task GBG children had slightly lower levels of peer rejection (although differenceswere not statistically significant, p=.08) and higher levels of on-task behavior (p=.05) than low on-task control group students. No statistically significant differences on any of the study variables were found between moderate/high on-task GBG children and their control counterparts at wave 1 and wave 4. However, at wave 4, moderate/high on-task GBG children had slightly higher levels of on-task behavior than moderate/high on-task control children.

3.1.1. The moderating role of on-task behavior in the impact of the GBG on aggressive behavior development Results in Table 2 show a significant negative effect of intervention status on the random slope (B=-0.04, p=.02), supporting

that the effect of children's individual variation in on-task behavior on the growth parameters of aggression was different between control group and GBG children, while controlling for classroom level of on-task behavior. Note that we had a specific hypothesis that the GBG would affect the development of aggression especially among initially low on-task children. We therefore conducted a number of follow-up analyses to breakdown the interaction effect between on-task behavior and GBG status on the growth parameters of individual level aggression.

First, we tested the effect of the interaction on each of the individual level growth parameters of aggression by including an interaction term between individual level on-task behavior and GBG status in themodel. In order to include this interaction term, we specified a conventional growth model in which standard errors are adjusted for clustering of data at the classroom level. The interaction termof on-task andGBG significantly predicted the slope parameter, after entering themain effects, B=−0.07, SE=0.03, β=−0.28, suggesting that the influence of on-task behavior on the change (i.e., slope) parameter of aggression depended upon GBG status.We then broke down this interaction term according to ourmain hypothesis (high/moderate vs. low on-task by GBG/control). This breakdown showed that the development of aggression among high/moderate on-task in either GBG or control and among low on-task in the control condition was similar, Δχ2 (2)=0.98, p=.61. However, the development of aggression among low on-task children in the GBG condition deviated from the other groups, Δχ2 (1)=6.21, p=.01, suggesting that the GBG impacted the link between on-task behavior and the development of aggression only among initially low on-task children.

This moderation of GBG in the link between on-task behavior and aggression development is illustrated in Fig. 3. In fact, it shows that after two intervention years, no differences in levels of aggression were found any longer between low on-task children and moderate/high on-task GBG children, t(260)=0.86, p=.39, d=0.18. Such differences were still found among control students, t(259)=−2.89, p=.004, d=−0.42. Having found support that GBG impact on the growth of aggression is dependent on individual level variation of on-task behavior at baseline, we then tested for individual level mediation processes among these initially low on-task children.

3.1.2. Mediators of the impact of the GBG on aggression among low on-task children The model with on-task behavior as a potential mediator was fitted first. This model had an adequate fit to the data (SRMR=.05;

CFI=.97; TLI=.96). The effect of the GBG onwave 4 on-task behavior was not statistically significant (β=0.23, p=.08). In addition, in this group, no link between wave 4 on-task behavior and the slope of aggression was found (β=0.11, p=.38), making testing for mediation inapplicable.

Table 1 Means and Standard deviations of peer-rated aggression; peer rejection and observed on-task behavior of children in the control and GBG conditions for moderate/high and low on-task subgroups.

Moderate/High on-task Low on-task

Control GBG GBG effect Control GBG GBG effect

Variable M SD M SD F p d M SD M SD F p d

On-task behavior Wave 1 (n=562) 3.10a 0.31 3.07a 0.27 0.75 .39 0.08 2.28b 0.23 2.33b 0.20 1.38 .24 −0.26 Wave 4 (n =514) 3.01a 0.31 3.05a 0.24 2.89 .09 −0.16 2.92a 0.35 3.08a 0.29 3.90 .05 −0.47

Aggression Wave 1 (n=570) 0.09a 0.14 0.10a 0.15 1.60 .21 −0.12 0.16b 0.22 0.15b 0.18 0.02 .89 0.03 Wave 2 (n=563) 0.12 0.19 0.13 0.19 0.69 .41 −0.08 0.20 0.26 0.15 0.20 0.91 .34 0.22 Wave 3 (n=529) 0.12 0.19 0.15 0.20 1.40 .24 −0.11 0.20 0.25 0.16 0.19 0.64 .43 0.19 Wave 4 (n=529) 0.12a 0.19 0.15a 0.19 1.86 .17 −0.13 0.22b 0.26 0.12a 0.15 3.97 .03 0.48

Peer rejection Wave 1 (n=569) 0.17a 0.16 0.17a 0.15 0.25 .62 −0.05 0.23b 0.20 0.22b 0.19 0.01 .91 0.03 Wave 4 (n=529) 0.23a 0.20 0.22a 0.18 0.24 .62 0.05 0.30b 0.23 0.21a 0.18 3.15 .08 0.41

Note. Means within the same measurement wave with the same superscript are not significantly different, whereas means within the same measurement wave with a different superscript are significantly different. d = Cohen's d.

Table 2 Individual and classroom level parameter estimates and standard errors for the multilevel latent growth curve model with a random slope for predicting children's aggression development.

Individual level

Intercept Slope

B SE B SE

Male 0.11⁎⁎ 0.02 0.05⁎⁎ 0.01

Classroom level

Intercept Slope Random slope

B SE B SE B SE

Growth factor mean 0.04⁎ 0.02 0.02 0.01 Low classroom on-task 0.03 0.05 −0.11⁎ 0.05 0.10 0.08 Intervention (GBG) 0.02 0.02 0.00 0.01 −0.04⁎ 0.02

Note. The random slope parameter reflects the cross level (classroom to individual) interaction term. ⁎ pb .05.

⁎⁎ pb .01.

195G. Leflot et al. / Journal of School Psychology 51 (2013) 187–199

The model with peer rejection fitted the data well (SRMR=.05; CFI=.94; TLI=.90). Results are presented in Fig. 4. For low on-task children, a statistically significant and negative link betweenGBG andwave 4 peer rejectionwas found (β=−0.18, p=.001). Peer rejection at wave 4 in turn was statistically significantly linked with the slope of aggression (β=0.70, pb .0001). When accounting for wave 4 peer rejection, the direct effect of the GBG on the development of aggression was no longer statistically significant (β=−0.12, p=.34). The test of mediation of the path from GBG to the development of aggression via peer rejection was statistically significant (β=−0.13, p=.002).

4. Discussion

Examining data from a randomized controlled universal preventive intervention study with the Good Behavior Game (GBG), results showed that the effect of the GBG on the development of aggression varied as a function of individual variation in on-task behavior. A GBG effect on aggression was found for initially low on-task children but not for their initially moderate to high on-task classmates. Whereas low on-task children in the control condition had high and increasing levels of aggression over time, low on-task GBG children showed a decrease in aggression over grades 2 and 3 of elementary school. After two years of intervention, low on-task GBG children had levels of aggression that resembled GBG children with moderate/high baseline levels of on-task behavior.

This finding supports, extends, and presents new insights to the existing literature on the effectiveness of the GBG intervention. First, the finding that control group children with low baseline levels of on-task behavior maintained the highest levels of aggression across the two years of the study underscores that low levels of on-task behavior is a risk condition for concurrent and future aggression (e.g., Bellanti et al., 2000; Harachi et al., 2006; Morrison et al., 2002; Schaeffer et al., 2003). Second, this study showed that the moderation effect of individual baseline level of on-task behavior was independent of the

Low - GBG

M/H - GBG

M/H - controle

Low - controle

Time End grade 2 Beginning grade 3Beginning grade 2 End grade 3

0.1

0.2

0.3

Aggression

Fig. 3. Developmental trajectories of low on-task and moderate/high on-task children in the control and GBG group. Results illustrate the significant difference in aggressive behavior development among initially low on-task children in the GBG condition (low—GBG; solid line)when compared controls (low— control— solid line). Results also showno difference in aggressive behavior development among initiallymoderate/high on-task children in theGBG condition (M/H—GBG; dotted line)when compared to controls (M/H — control; dotted line).

Agg Agg Agg Agg

SaggIagg

GBG

Wave 1 Wave 3 Wave 2 Wave 4

Peer Peer

-0.07

-0.12

0.67***

-0.19**

0.70***

Fig. 4. Path estimates and graphical representation of the mediation model testing possible associations between the Good Behavior Game (GBG), aggressive behavior (Agg), and peer rejection (peer) among children with low levels of on-task behavior (n=87). Values on the single headed arrows reflect standardized regression estimates; values on the double headed arrows are correlations among residual variances of the variables. Iagg = intercept of aggression. Sagg = slope of aggression. * pb .05. ** pb .01. *** pb .001.

196 G. Leflot et al. / Journal of School Psychology 51 (2013) 187–199

mean classroom level of on-task behavior. This finding implies that although the GBG is a classroom-based program, its impact on reductions in aggressive behavior varied as a function of individual children,when controlling for the classroom-level variation. Third, the findings showed that a universal program like the GBGwas effective in reducing the aggression of low on-task children, and they are in line with findings from previous studies showing that the GBGworks at the upper end of the risk continuum (e.g., Dolan et al., 1993; Kellam et al., 1994; Rebok et al., 1996; van Lier et al., 2004, 2005). Fourth, the results support that the GBG “corrected” the development of aggression among initially at-risk children, so that their levels of aggression were comparable to initially low at-risk children after termination of the intervention. Thus, this finding may characterize the GBG as a corrective intervention, reducing the level differences in aggression between initially at-risk and low-risk children.

A second goal of the study was to examine the pathway through which the GBG reduced levels of aggression among children who were initially low on-task. It was shown that these children in the control group were most likely to experience rejection by mainstream peers, which is in line with previous studies (Bellanti et al., 2000; Hughes & Kwok, 2006). However, the GBG resulted in a decrease of peer rejection among low on-task children. This decrease in peer rejection in turn mediated the effect of the GBG on their aggressive behavior development. We found that the results applied to both boys and girls. Thus, initially at-risk children who received the GBG intervention showed a reduction in their aggressive behavior to normative levels, because these children improved their relations with mainstream peers. This finding supports a recent finding that the GBG affects children's behavior through improving relations with peers (van Lier et al., 2005; Witvliet et al., 2009) and extends it to at-risk children.

This study underscores the importance of the role of peer relations in the development of aggression among at-risk children. Children form groups based on similarities, whereas dissimilarities reduce the contact between children (see Dishion et al., 1994). Low on-task children frequently do not live up to the standards of the larger peer group. Peer rejection in turn is a solid predictor of further engagement in aggression, delinquency, and other externalizing behaviors (see overview studies by Deater-Deckard, 2001; Parker et al., 2006; Rubin et al., 2006). This study provided unique support for this described pathway by showing it is possible to reverse the pattern. The results suggest that when peer relations are fostered, such as done by the GBG, it will provide initially at-risk children opportunities to interact with mainstream peers and thereby to improve their peer status. As a consequence, these children's behavior problems can be reduced.

It must also be noted that a second component of the GBG intervention, that is, structuring the classroom environment and thereby possibly improving on-task behavior, did not explain the GBG effect on aggression. GBG children with low on-task behavior at baseline did not show significantly more on-task behavior at the end of the third grade, than their control counterparts. In addition the on-task behavior did not account for the effect of the GBG on the decrease of aggression over second and third grade among low on-task children. On-task behavior was found to be unrelated to aggressive behavior development. It may well be that the pathway through which low on-task behavior is linked to aggressive behavior development is via poor peer relations.

4.1. Limitations

Some limitations should be considered when interpreting these results. The first concerns the sample. Most of the children had a Flemish Belgian background and had well-educated parents, making the sample ethnically and socially homogenous. Studies with more ethnically and socially diverse samples are needed before these findings can be generalized. Furthermore, to handle the missing data we used the Mplus missing data module to optimally use the data available. However, the data might not have

197G. Leflot et al. / Journal of School Psychology 51 (2013) 187–199

been missing at random. These children (N=41) had higher initial levels of peer-rated aggression and peer rejection. Thus, our results generalize only to students who were not kept back a grade or who did not move schools.

Also, although we had a sizeable sample, the number of children identified as being at-risk (children with initial low on-task behavior) in this study was modest. This small number of children at-risk reduced the statistical power of the analyses. Nonetheless, statistically significant effects were found in this subsample. Associated to this aspect, there is a limitation with regard to the moderator, on-task behavior. The moderator was dichotomized (moderate to high on-task behavior vs. low on-task behavior) to simplify the complex analyses. Moreover, and more importantly, the scores of on-task behavior were highly skewed, with the large majority of the children having a mean on-task score of 2 out of three 3 (mean=1.96, median=2) with a small range (SD=0.40). Although, this is one of the only situations in which dichotomization may be justified (see MacCallum, Zhang, Preacher, & Rucker, 2002), dichotomizing the moderator may have resulted into a loss of power and a distortion of the results (see MacCallum et al., 2002).

Second, some limitations concerning the instruments should be considered. With regard to aggression and peer rejection, we used peer ratings for both measurements, whichmay have inflated the associations between the variables. As for the observations of on-task behavior, the observers were not always blind to the intervention status of the classroom, mostly because teachers or children revealed the intervention status of their class. Next, although the observers of on-task behavior reached and maintained an interobserver agreement of 80% and more using videotaped classrooms situations, the score of the interobserver agreement assessed in real classroom settings was lower. Many studies observing on-task behavior score the child or classroom as either on-task or not on-task (e.g., Ferguson & Houghton, 1992; Lan et al., 2009; Sutherland, Wehby, & Copeland, 2000). In this study, however, four different scores of on-task behavior were used, indicating the amount of time a child was on-task during the interval, making it more difficult to reach high interobserver agreement. Perhaps training more frequently in real-life classroom settings would have resulted in a better level of interobserver agreement. Also with regard to the observation of on-task behavior, the present study included a limited number of observation sessions. As all the children in the participating classrooms had to be observed, it was not feasible to organize more and longer observations of each child. Given the assumed natural variability in students' on-task behavior, this would have been preferable in terms of reliability.

Third, a directional path from peer relations to the development of aggression is assumed in this study as the GBG aims to facilitate, among others, positive peer relations in order to reduce behavior problems. Although Ladd (2006) showed a directional effect of peer rejection on the development of aggressive behavior, when variables are measured concurrently it is difficult to draw causal inferences (Kline, 2005). Therefore, causal conclusions on the role of peer relations in aggressive behavior development should not be drawn from this study. Thus, future experimental research should examine the time sensitive mediated effect of the GBG on aggression in that it should establish that the effect of the program on peer rejection should in time precede andmediate reductions in aggression. In addition to studying a time sensitive mediation path, future research should also focus on the micro-level processes that may explain why low on-task children who become increasingly accepted by peers respondwith lower levels of aggression. Such processes could include fewer impulsive responses to the rejection experiences—for instance, through less coercive exchanges with mainstream peers (Snyder et al., 2008) and fewer victimization experiences by the rejected child frommainstream peers that may enhance aggressive responses (van Lier & Koot, 2010). Also, increasingly accepted low on-task children may respond with less aggression because of improvements in prosocial behavior (Haselager, et al., 2002). The threatening of their “need to belong” may even decrease (Baumeister & Leary, 1995), which has been shown to impair self-regulation (Baumeister, DeWall, Ciarocco, & Twenge, 2005), especially likely among the low-on task, hence, at-risk children.

Fourth, some limitations should be stated with regard to the measurement of the quality of implementation of the GBG. The implementation quality was measured using a procedure especially developed to assess the quality of the implementation of the GBG (van der Sar, 2004). This instrument intends to evaluate the crucial components of the GBG. However, although these components seem important parts of the GBG, no research has been conducted on these crucial components to date. It would be interesting for future research to focus on this aspect. Furthermore, while the quality of the implementation of the GBG in intervention classrooms was measured, information from conversations with control teachers was used to assess whether they implemented the GBG or parts of the program despite being allocated to the control condition. To the best of our knowledge, this study was the first implementation of the GBG in Flanders, Belgium. None of the teachers participating in the study nor their principals reported to have heard of the program when we introduced the study. No indication was found that control teachers implemented parts of the GBG. However, using an objective instrument to assess possible GBG implementation in the control classrooms would have been a more objective way to assess possible treatment confounding. Furthermore, teachers completed 8 classroom consultations instead of the prescribed 10 consultations (van der Sar & Goudswaard, 2001). Although statistically significant effects of the GBG on the outcome measures were found, despite having two fewer consultations than prescribed, research should be directed at the support that teachers need to implement the GBGwith high quality, in general, and the amount of classroom consultations needed, specifically.

4.2. Conclusions and future directions

The present results provide new and valuable insights on the role of on-task behavior in the development of aggression. Low levels of on-task classroom behavior were shown to be an important risk factor for children's problematic social and behavioral adjustment to the classroom, as has been shown in other educational contexts. More importantly, this study demonstrated that it is possible to reduce aggressive behavior among low on-task children through improving their peer relations. This result implies that both school psychologists and researchers should be aware of individual children with low levels of on-task classroom

198 G. Leflot et al. / Journal of School Psychology 51 (2013) 187–199

behavior as “at risk”. They should also be aware of the harmful effect that peer relations may have in the development of children's aggression and, in a more positive perspective, that it is possible to improve both peer relations and behavior through universal intervention.

The present study also provides insights into the effectiveness of the GBG and universal programs in general. The GBG not only resulted into decreased levels of aggression, but it did so by reducing an underlying cause for aggression—peer rejection. Thereby, this study adds to the evidence on the effectiveness of the GBG and its working mechanisms in different educational contexts. Future research is recommended to focus on the long-term effects of the program, as well as on additional outcomes possibly linked to aggression, such as grade retention and discipline referrals. Furthermore, it is suggested that future researchers evaluate if aggression and peer rejection improve not only in classroom settings, directly influenced by the GBG, but also in non-classroom settings. Only then can it be concluded that the GBG actually influenced the children's developmental trajectory and not just the short-term, classroom-based outcomes. In addition, it is recommended that future researchers evaluate the effects of the GBG in other grades (e.g., upper elementary school, middle school, high school), where children may have more established levels of (higher or lower) on-task behavior.

Furthermore, the findings showed that this universal preventive intervention was able to reduce the aggressive behavior development of at-risk children to levels of aggression that resembled children at low-risk for aggressive behavior. These findings amplify the importance of such universal programs as they can target an important social system in which children function, the classroom, as a whole. Improving the peer relations within this system resulted in the reduced aggression. This finding may imply that, for these children, selective programs targeting only children at-risk may not be necessary. This finding is important because universal interventions generally minimize stigmatization of participants and may be more readily accepted and adopted because they are positive and proactive and fit better in natural (i.e., group-oriented) classroom practices (cf. Greenberg, Domitrivich, & Bumbarger, 2001). The study showed that beyond these rationales, such universal programs like the GBG are also effective.

References

Algemene Directie Statistiek en Economische Informatie (2004). Bevolking en huishoudens. Huishoudens en familiekernen [Population and households. Households and family nuclei]. General Direction Statistics and Economical Information. Brussels, Belgium: Algemene Directie Statistiek en Economische Informatie.

Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic and statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182.

Barrish, H. H., Saunders, M., & Wolf, M. M. (1969). Good behavior game: Effects of individual contingencies for group consequences on disruptive behavior in a classroom. Journal of Applied Behavior Analysis, 2, 119–124.

Baumeister, R. F., DeWall, C. N., Ciarocco, N. J., & Twenge, J. M. (2005). Social exclusion impairs self-regulation. Journal of Personality and Social Psychology, 88, 589–604.

Baumeister, R. F., & Leary, M. R. (1995). The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 117, 497–529.

Bellanti, C. J., & Bierman, K. L.Conduct Problems Prevention Research Group. (2000). Disentangling the impact of low cognitive ability and inattention on social behavior and peer relationships. Journal of Clinical Child Psychology, 29, 66–75.

Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246. Birch, S. H., & Ladd, G. W. (1997). The teacher-child relationship and children's early school adjustment. Journal of School Psychology, 35, 61–79. Boivin, M., Vitaro, F., & Poulin, F. (2005). Peer relationships and the development of aggressive behavior in early childhood. In R. E. Tremblay, W. W. Hartup, & J.

Archer (Eds.), Developmental origins of aggression (pp. 376–397). New York, NY: Guilford Press. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen, & J. S. Long (Eds.), Testing structural equation models (pp. 136–162).

Newburry Park, CA: Sage. Cillessen, A. H. N. (2009). Sociometric methods. In K. H. Rubin, W. M. Bukowski, & B. Laursen (Eds.), Handbook of peer interactions, relationships, and groups

(pp. 82–99). New York, NY: Guilford Press. Coie, J. D., Dodge, K. A., & Coppotelli, H. (1982). Dimensions and types of social status: A cross-age perspective. Developmental Psychology, 18, 557–570. Coie, J. D., Watt, N. F., West, S. G., Hawkins, J. D., Asarnow, J. R., Markman, H. J., et al. (1993). The science of prevention. A conceptual framework and some

directions for a national research program. American Psychologist, 48, 1013–1022. Deater-Deckard, K. (2001). Annotation: Recent research examining the role of peer relationships in the development of psychopathology. Journal of Child

Psychology and Psychiatry, 42, 565–579. Dishion, T. J., Patterson, G. R., & Griesler, P. C. (1994). Peer adaptations in the development of antisocial behavior: A confluence model. In G. R. Huesmann (Ed.),

Aggressive behavior: Current perspectives (pp. 61–95). New York, NY: Plenum Press. Dodge, K. A., Coie, J. D., & Brakke, N. P. (1982). Behavior patterns of socially rejected and neglected preadolescents: The role of social approach and aggression.

Journal of Abnormal Child Psychology, 10, 389–410. Dolan, L. J., Kellam, S. G., Brown, C. H., Werthamer-Larsson, L., Rebok, G. W., Mayer, L. S., et al. (1993). The short-term impact of two classroom-based preventive

interventions on aggressive and shy behaviors and poor achievement. Journal of Applied Developmental Psychology, 14, 317–345. Dolan, L. J., Turkkan, J. S., Werthamer-Larsson, L., & Kellam, S. G. (1989). The Good Behavior Game manual. City of Baltimore: Department of Education and the

Prevention Centre, The John Hopkins University. Durlak, J. A., & Dupre, E. P. (2008). Implementation matters: A review of the research on the influence of implementation on program outcomes and the factors

affecting implementation. American Journal of Community Psychology, 41, 327–350. Ferguson, E., & Houghton, S. (1992). The effects of contingent teacher praise, as specified by Canter’s assertive discipline program, on children’s ontask behavior.

Educational Studies, 18, 83–93. Fishbein, J. E., & Wasik, B. H. (1981). Effect of the Good Behavior Game on disruptive library behavior. Journal of Applied Behavior Analysis, 14, 89–93. Fixsen, D. L., Naoom, S. F., Blasé, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. FMHI Publications, 231, Tampa, FL:

University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (Retrieved September 30, 2011, from http://ctndisseminationlibrary.org/PDF/nirnmonograph.pdf)

Flay, B. R., Biglan, A., Boruch, R. F., González, Castro F., Gottfredson, D., Kellam, S., et al. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissimilation. Prevention Science, 6, 151–175.

Greenberg, M. T., Domitrivich, C., & Bumbarger, B. (2001). The prevention of mental disorders in school-aged children: Current state of filed. Prevention and Treatment, 4, 1–62.

Harachi, T. W., Fleming, C. B., White, H. R., & Ensminger, M. E. (2006). Aggressive behavior among girls and boys during middle childhood: Predictors and sequelae of trajectory group membership. Aggressive Behavior, 32, 279–293.

199G. Leflot et al. / Journal of School Psychology 51 (2013) 187–199

Haselager, G. J., Cillessen, A. H., Van Lieshout, C. F., Riksen-Walraven, J. M., Hartup, W. W., & Bukowski, W. M. (2002). Heterogeneity among peer-rejected boys across middle childhood: Developmental pathways of social behavior. Developmental Psychology, 38, 446–456.

Hollis, S., & Cambell, F. (1999). What is meant by intention to treat analysis? Survey of published randomized controlled trials. British Medical Journal, 319, 670–674.

Holmbeck, G. (1997). Toward terminological, conceptual, and statistical clarity in the study of mediators and moderators: Examples from the child-clinical and pediatric psychology literatures. Journal of Consulting and Clinical Psychology, 65, 599–610.

Hughes, J. N., & Kwok, O. -M. (2006). Classroom engagement mediates the effect of teacher-student support on elementary students' peer acceptance: A prospective analysis. Journal of School Psychology, 43, 465–480.

Keiley, M. K., Bates, J. E., Dodge, K. A., & Pettit, G. S. (2000). A cross-domain growth analysis: Externalizing and internalizing behaviors during 8 years of childhood. Journal of Abnormal Child Psychology, 28, 161–179.

Kellam, S. G., Ling, X., Merisca, R., Brown, C. H., & Ialongo, N. (1998). The effect of the level of aggression in the first grade classroom on the course and the malleability of aggressive behavior into middle school. Development and Psychopathology, 10, 165–185.

Kellam, S. G., Rebok, G. W., Ialongo, N., & Mayer, L. S. (1994). The course and malleability of aggressive behavior from early first grade into middle school: Results of a developmental epidemiologically-based preventive trial. Journal of Child Psychology and Psychiatry, 35, 259–281.

Kellam, S. G., Reid, J., & Balster, R. L. (Eds.). (2008). Effects of a universal classroom behavior program in first and second grade in young adult outcomes. Drug and Alcohol Dependence, 95(1) (Special issue).

Kline, R. B. (2005). Principles and practice of structural equation modeling (2nd ed.). New York, NY: Guilford Press. Ladd, G. W. (2006). Peer rejection, aggressive or withdrawn behavior, and psychological maladjustment from ages 5 to 12: An examination of four predictive

models. Child Development, 77, 822–846. Ladd, G. W., Buhs, E. S., & Seid, M. (2000). Children's initial sentiments about kindergarten: Is school liking an antecedent of early classroom participation and

achievement? Merrill-Palmer Quarterly, 46, 255–279. Lan, X., Ponitz, C. C., Miller, K., Li, S., Cortina, K., Perry, M., et al. (2009). Keeping their attention: Classroom practices associated with behavioural engagement in

first grade mathematics classes in China and the United States. Early Childhood Research Quarterly, 24, 198–211. Lannie, A. L., & McCurdy, B. L. (2007). Preventing disruptive behavior in the urban classroom: Effects of the Good Behavior Game on student and teacher behavior.

Education and Treatment of Children, 30, 85–98. MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002). On the practice of dichotomization of quantitative variables. Psychological Methods, 7, 19–40. MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods.

Multivariate Behavioral Research, 39, 99–128. Moffitt, T. E., Caspi, A., Rutter, M., & Silva, P. A. (2001). Sex differences in antisocial conduct disorder, delinquency, and violence in the Dunedin longitudinal study. New

York, NY: Cambridge University Press. Morgan-Lopez, A. A., & Mackinnon, D. P. (2006). Demonstration and evaluation of a method assessing mediated moderation. Behavior Research Methods, 38,

77–87. Morrison, G. M., Robertson, L., Laurie, B., & Kelly, J. (2002). Protective factors related to antisocial behavior trajectories. Journal of Clinical Psychology, 58, 277–290. Muthén, L. K., & Muthén, B. O. (1998-2009). Mplus user's guide (5th ed.). Los Angeles, CA: Muthén & Muthén. Parker, J., Rubin, K. H., Erath, S., Wojslawowicz, J. C., & Buskirk, A. A. (2006). Peer relationships and developmental psychopathology. In D. Cicchetti, & D. Cohen

(Eds.), (2nd ed.). Developmental psychopathology: Risk, disorder, and adaptation, Vol. 2. (pp. 419–493)New York, NY: Wiley. Rebok, G. W., Hawkins, W. E., Krener, P., Mayer, L. S., & Kellam, S. G. (1996). Effects of concentration problems on the malleability of children's aggressive and shy

behaviors. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 193–203. Rubin, K. H., Bukowski, W. A. M., & Parker, J. G. (2006). Peer interaction, relationships, and groups. In N. Eisenberg (Ed.), W. Damon, & R.M. Lerner (Series Eds.),

Handbook of Child Psychology, Vol. 3: Social, Emotional, and Personality Development (6th ed., pp. 571-645). New York, NY: Wiley. Schaeffer, C. M., Petras, H., Ialongo, N., Poduska, J., & Kellam, S. (2003). Modeling growth in boys' aggressive behavior across elementary school: Links to later

criminal involvement, conduct disorder, and antisocial personality disorder. Developmental Psychology, 39, 1020–1035. Skinner, E. A., & Belmont, M. J. (1993). Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. Journal

of Educational Psychology, 85, 571–581. Skinner, E. A., Kindermann, T. A., & Furrer, C. J. (2009). A motivational perspective on engagement and disaffection: Conceptualization and assessment of

children's behavioral and emotional participation in academic activities in the classroom. Educational and Psychological Measurement, 69, 493–525. Snyder, J., Schrepferman, L., McEachern, A., Barner, S., Johnson, K., & Provines, J. (2008). Peer deviancy training and peer coercion: Dual processes associated with

early-onset conduct problems. Child Development, 79, 252–268. Sutherland, K. S., Wehby, J. H., & Copeland, S. R. (2000). Effect of varying rates of behavior-specific praise on the on-task behavior of students with EBD. Journal of

Emotional and Behavioral Disorders, 8, 2–8. Sturaro, C., van Lier, P. A. C., Cuijpers, P., & Koot, H. M. (2011). The role of peer relationships in the development of early school-age externalizing problems. Child

Development, 2011(82), 758–765. Terry, R. (2000). Recent advances in measurement theory and the use of sociometric techniques. In A. H. N. Cillessen, & W. M. Bukowski (Eds.), Recent advances in

the measurement of acceptance and rejection in the peer system: New directions for child development, Vol. 88. (pp. 27–53)San Francisco, CA: Jossey-Bass. Van der Sar, A. M. (2004). Met taalspel lukt het wel. Een tussenrapportage over de effecten van Taakspel op taakgericht en regelovertredend gedrag in de klas.

Rotterdam, The Netherlands: Pedologisch Instituut, onderdeel van de CED groep. Van der Sar, A. M., & Goudswaard, M. (2001). Docentenhandleiding Taakspel voor basisonderwijs. Rotterdam, The Netherlands: Pedologisch Instituut, onderdeel van

de CED groep. Van Lier, P. A. C. (2002). Preventing disruptive behavior in early elementary schoolchildren. (unpublished doctoral dissertation). University of Rotterdam, The

Netherlands. van Lier, P. A. C., & Koot, H. M. (2008). Peer relationships. In R. Loeber, W. N. Slot, P. Van der Laan, & M. Hoeve (Eds.), Tomorrow's criminals: The development of child

delinquency and effective interventions. Ashgate PressAshgate, England. van Lier, P. A. C., & Koot, H. M. (2010). Developmental cascades of peer relations and symptoms of externalizing and internalizing problems from kindergarten to

fourth-grade elementary school. Development and Psychopathology, 22(3), 569–582. van Lier, P. A. C., Muthén, B. O., van der Sar, R. M., & Crijnen, A. A. M. (2004). Preventing disruptive behavior in elementary schoolchildren: Impact of a universal,

classroom-based intervention. Journal of Consulting and Clinical Psychology, 72, 467–478. van Lier, P. A. C., Vuijk, P., & Crijnen, A. A. M. (2005). Understanding mechanisms of change in the development of antisocial behavior: Impact of a universal

intervention. Journal of Abnormal Child Psychology, 33, 521–535. Vuijk, P., van Lier, P. A. C., Crijnen, A. A. M., & Huizink, A. C. (2007). Testing sex-specific pathways from peer victimization to anxiety and depression in early

adolescents through a randomized intervention trial. Journal of Affective Disorders, 100, 221–226. Williams, R. L. (2000). A note on robust variance estimation for cluster-corrected data. Biometrics, 56, 645–646. Witvliet, M., van Lier, P. A. C., Cuijpers, P., & Koot, H. M. (2009). Testing links between childhood positive peer relations and externalizing outcomes through a

randomized controlled intervention study. Journal of Consulting and Clinical Psychology, 77, 905–915. Zimmer-Gembeck, M. J., Geiger, T. C., & Crick, N. R. (2005). Relational and physical aggression, prosocial behavior, and peer relations: Gender moderation and

bidirectional association. Journal of Early Adolescence, 25, 421–452.