Wk 5 DQ1 and DQ2
bubblegum
Thequietgamedisruptions.pdf
The Quiet Classroom Game: A Class-Wide Intervention to Increase Academic Engagement and Reduce
Disruptive Behavior
Keith C. Radley, Evan H. Dart, and Roderick D. O’Handley The University of Southern Mississippi
Abstract. The current study investigated the effectiveness of the Quiet Classroom Game, an interdependent group contingency using an iPad loaded with a decibel meter app, for increasing academically engaged behavior. Three first-grade class- rooms in the southeastern United States, identified as displaying high levels of noise and disruptive behavior, were included in the study. A multiple-baseline design with an embedded ABAB condition sequence was used to evaluate the effect of implementation of the Quiet Classroom Game on academically engaged behavior, disruptive behavior, and classroom decibel level. Implementation of the intervention resulted in large increases in academically engaged behavior, mod- erate to large reductions in disruptive behavior, and large decreases in classroom noise. Results of social validity checklists administered to teachers and students indicated acceptability and utility of the intervention. Findings of the study suggest that the Quiet Classroom Game may be an effective method for increasing the academically engaged behavior and decreasing the noise and disruptive behavior of first-grade students in a general-education setting.
Establishing a group contingency within a classroom is one of the most common evi- dence-based classroom management strategies (Simonsen, Fairbanks, Briesch, Myers, & Sugai, 2008). Group contingencies represent a practical way to administer a single conse- quence to multiple individuals in an effort to change the behavior of the entire group (Coo- per, Heron, & Heward, 2007). The school- based intervention literature generally sup- ports the use of group contingencies in the classroom as one of the more effective strate- gies to shape the behavior of students (Stage & Quiroz, 1997). In fact, a systematic review of the school-based group contingency interven-
tion literature targeting challenging behavior conducted by Maggin, Johnson, Chafouleas, Ruberto, and Berggren (2012) suggested that interventions using group contingencies should be considered an evidence-based strategy ac- cording to the criteria set forth by the What Works Clearinghouse (Kratochwill et al., 2010) and that the research supporting inter- dependent group contingencies, specifically, is the largest contributor to this body of evidence.
The review by Maggin et al. (2012) es- tablished interdependent group contingencies as a practical and efficient behavior change strategy that is empirically grounded for use in
Please address correspondence regarding this article to Keith C. Radley, Department of Psychology, University of Southern Mississippi, 118 College Dr, Ste 5025, Hattiesburg, MS, 39406; e-mail: [email protected]
Copyright 2016 by the National Association of School Psychologists, ISSN 0279-6015, eISSN 2372-966x
School Psychology Review, 2016, Volume 45, No. 1, pp. 93–108
93
the schools. However, the review made a vital distinction within this subset of the literature by identifying the relative scarcity of research investigating the effectiveness of interdepen- dent group contingencies on the academically engaged behavior of students. The majority of the studies included in the review targeted nonacademic dimensions of student behavior. This is concerning, considering the literature suggesting the importance of academic en- gagement to achievement (Finn & Zimmer, 2012) and the fact that reducing students’ dis- ruptive behavior will not necessarily increase students’ engagement. Of the 27 articles in- cluded in the review, only three peer-reviewed studies used an interdependent group contin- gency in an attempt to increase the academic engagement of students.
Crouch, Gresham, and Wright (1985) arranged three concurrent group contingencies within a single third-grade classroom in an attempt to increase on-task behavior and de- crease disruptive and off-task behavior. One interdependent group contingency specifically targeted on-task, or academically engaged, be- havior by having the teacher provide the stu- dents with free time if 80% or more of them were on-task during a specific number of in- tervals throughout the day. Using an ABAB withdrawal design and data collected by the teacher during periodic checks as the primary source of data, the researchers were able to demonstrate a functional relationship between intervention implementation and an increase in on-task behavior exhibited by the students. Although the results of this study highlight the effectiveness of group contingencies on on- task behavior, because there were two other concurrent group contingencies in place tar- geting disruptive behavior, it is difficult to determine which component of the interven- tion package was responsible for the increase in on-task behavior.
A second and more recent study exam- ining the effects of interdependent group con- tingencies on the academically engaged be- havior of students was conducted by Christ and Christ (2006) in three high school class- rooms. In this study, teachers used a digital scoreboard to record the number of 2-min
intervals their students went without exhibit- ing any disruptive behavior during a 48-min class period. If the class earned 17 undisrupted instructional intervals, the students received reinforcement with free time. The primary re- sults indicated that the rate of academically engaged behavior increased and the rate of disruptive vocalizations decreased as a result of implementation of the interdependent con- tingency. Furthermore, it is interesting to note that the contingency was based on reinforce- ment of the absence of disruptive behavior yet still produced increases in academically en- gaged behavior without explicitly targeting this behavior; however, the classroom teachers were responsible for identifying periods of undisrupted instructional intervals and ap- peared to differ from the researchers in their judgments about the occurrence of disruptive behavior, causing the authors to question the objectivity of the teachers.
Finally, McKissick, Hawkins, Lentz, Hailley, and McGuire (2010) investigated the effects of a randomized interdependent group contingency on the academically engaged be- havior of a second-grade classroom in a mul- tiple-baseline across-settings design. During each day of the intervention, the target behav- ior, criterion for reinforcement, and reinforcer were randomized and kept hidden from the students to mitigate some of the negative ef- fects that nonrandomized elements can have on the effectiveness of group contingencies (Skinner, Williams, & Neddenriep, 2004). Students were provided with reinforcement if they met the randomly selected criterion for the randomly selected behavior each day; however, because of relatively rigorous data collection procedures, the teacher was not able to implement the intervention independently. Regardless, the authors identified positive inter- vention effects on both academic engagement and disruptive behavior. In addition, similar to the Christ and Christ (2006) study, academic engagement was the primary dependent variable despite the fact that it was not explicitly targeted by the intervention protocol.
A single study published since the Mag- gin et al. (2012) review (Flower, McKenna, Muething, Bryant, & Bryant, 2014) investi-
School Psychology Review, 2016, Volume 45, No. 1
94
gated the effects of a popular interdependent group contingency, the Good Behavior Game, on students’ off-task behavior in two high school special-education classrooms. By split- ting the students within a classroom into teams and having the teacher provide “fouls” to a team when a team member was off-task, the researchers were able to demonstrate a rela- tionship between implementation of the Good Behavior Game and a reduction in students’ off-task behavior using a withdrawal design; however, limitations exist regarding the use of teacher judgments of behavior as the primary contingency management outcome and the lack of data on academically engaged behav- ior. Although a reduction in off-task behavior also suggests an increase in on-task behavior, the study did not provide any data to support an increase in academically engaged behavior.
The previously described studies all used interdependent group contingencies to increase the academically engaged behavior of students; however, there appears to be a tradeoff between the objectivity of the data used to manage these contingencies and the practicality of the procedures. Crouch et al. (1985), Christ and Christ (2006), and Flower et al. (2014) used teacher judgments of on- task, disruptive, and off-task behavior, respec- tively, to manage the reinforcement contin- gencies, but some noted concerns regarding the accuracy of these judgments as a limita- tion. McKissick et al. (2010) used researcher- conducted systematic direct observation to manage the interdependent contingency but were not able to transfer implementation of the intervention to the teacher as a result. There- fore, it would be useful to identify alternative outcomes that are objectively measured and feasible for classroom teachers to assess which are behavioral correlates of academically en- gaged behavior.
One possible outcome that may be mea- sured objectively in the classroom is the level of noise exhibited by the students. Classroom noise can be objectively measured and quan- tified into decibels, a standardized unit of sound intensity. Using the average decibel level emitted by students in a classroom as an objective analogue for academically engaged
behavior may make group contingency inter- ventions that target this behavioral domain more feasible. Furthermore, there is prelimi- nary evidence to suggest that the level of noise exhibited by students within a classroom may affect academic achievement (Shield & Dock- rell, 2006).
Although there is evidence to suggest that excessive classroom noise can hinder ac- ademic performance, relative little research has evaluated strategies for reducing noise in school settings and the effects on academ- ic engagement. Researchers have evaluat- ed noise-reduction interventions in school hallways (Kartub, Taylor-Greene, March, & Horner, 2000; Staub, 1990) and cafeterias (Davey, Alexander, Edmonson, Stenhoff, & West, 2001; LaRowe, Tucker, & McGuire, 1980), yet these studies did not include any additional measures of student behavior other than the noise level. Even fewer studies have investigated such procedures within the class- room, where noise may be more closely re- lated to academic performance. Both Schmidt and Ulrich (1969) and Strang and George (1975) used decibel level-based group contin- gencies to reduce the noise level within class- rooms. Despite successfully reducing the deci- bel level, neither study evaluated the effect of the intervention on the academically engaged or disruptive behavior of students. Although group contingencies have been found to be generally socially valid (e.g., Flower et al., 2014; McKissick et al., 2010), studies that have used decibel level-based group contin- gencies have been limited by a lack of social validity data—despite early concerns ques- tioning the social validity of such procedures (Winett & Winkler, 1972).
PURPOSE
The purpose of the study was to identify the presence of a functional relationship be- tween implementation of the Quiet Classroom Game (QCG), which is a novel universal be- havioral intervention, and increases in the ac- ademically engaged behavior of elementary school students by arranging an interdepen- dent group contingency based on the noise
Quiet Classroom Game
95
level in the classroom. The current study was designed to answer the following research questions:
1. To what extent does implementation of the QCG result in an increase in academically engaged behavior of el- ementary school students?
2. How well does implementation of the QCG result in a reduction in the dis- ruptive behavior of elementary school students?
3. To what extent does implementation of the QCG result in a reduction in the noise level within the classroom?
4. How do teachers describe the accept- ability and social validity of QCG as a class-wide behavioral intervention?
5. How do students describe the accept- ability and social validity of QCG as a class-wide behavioral intervention?
METHOD
Participants included three elementary school teachers and their classrooms, located in the southeast region of the United States, and referred for high levels of classroom dis- ruptive behavior and noise. Teachers A, B, and C were first-grade general-education teachers within the same school district.
Participants and Setting
Teacher A’s school consisted of approx- imately 461 students, pre-K through sixth grade, with 79% of students eligible for free or reduced-price lunch and 94% of the student population identified as African American (6% other). Teacher B and Teacher C’s school consisted of approximately 441 students, pre-K through sixth grade, with approximately 80% of students eligible for free or reduced- price lunch and 96% of the student population identified as African American (4% other). All teachers had previously consulted with their respective Teacher Support Team regarding elevated levels of disruptive behavior and noise within the classroom, with teacher re- ports indicating minimal improvement in be- havior following initial consultation with the Teacher Support Team.
Teachers A, B, and C were each accom- panied by a classroom aide during the course of the study. Teacher A was a 24-year-old African American woman in her second year teaching the first grade. Teacher A obtained a degree in elementary education and, at the time of the study, was working toward a mas- ter’s degree in gifted education. Her classroom consisted of 23 students, 14 girls and 9 boys, all of whom were African American. At the time of the study, three students were receiv- ing individualized behavioral support in the form of a daily behavior report card. Teacher A indicated that the three students had not maintained progress on their daily behavior report cards. Prior to collection of baseline data, Teacher A identified a 15-min period immediately after lunch, during which the teacher provided instruction in math and stu- dents completed math worksheets, for inter- vention implementation.
Teacher B was a 32-year-old White woman with 8 years of teaching experience and a master’s degree in reading. Her class- room consisted of 15 students, 12 girls and 3 boys, all identified as African American. At the time of the study, six students were receiv- ing individualized behavioral support in the form of daily behavior report cards. Teacher B identified a 15-min period at the beginning of each day, during which students completed student-directed language arts centers (e.g., computers, reading, vocabulary bingo), for in- tervention implementation. Teacher C was a 50-year-old White woman with 16 years of teaching experience and a bachelor’s degree in general education. Her classroom consisted of 18 students, 14 boys and 4 girls, with 16 students identified as African American and 2 as Hispanic. At the time of the study, four students were receiving individualized behav- ioral support via daily behavior report cards. Prior to baseline, Teacher C identified a 15- min period at the beginning of the day, during which the teacher provided group instruction and students completed assignments, for inter- vention implementation.
Classroom configurations and materials present were consistent across each classroom. Each classroom measured approximately 7.5
School Psychology Review, 2016, Volume 45, No. 1
96
meters by 9 meters. All students had an indi- vidual desk and chair, with desks placed in groups of five in all classrooms. All classes were equipped with a whiteboard, secured to the wall at the front of the classroom. Other materials present, but not used in the current study, included teacher and student computers, a teacher desk, a projector, and bookcases.
Materials
Several materials were used during inter- vention, including a teacher script, a MotivAider device, an iPad, the classroom whiteboard, and teacher-approved tangible rewards. We describe the classroom materials below.
Teacher Script Teachers were provided with a student-
training script to read to their class. The script did not have to be read verbatim but contained several critical features that teachers were required to mention when introducing the QCG to their class, including the purpose of the QCG, demonstration of how the decibel meter functioned, the classroom’s decibel goal, and the availability of tangible rewards should the students meet reinforcement criteria.
iPad With Decibel 10th App The classrooms’ decibel level was mon-
itored using the Decibel 10th app. The Decibel 10th app was developed by SkyPaw, designed for iPhone and iPad devices, and download- able for free via iTunes. The researchers downloaded the Decibel 10th app on an iPad for use in the current study. The Decibel 10th app records the decibel level every one tenth of a second, automatically generating a down- loadable data sheet for periods of use. Using data sheets generated by the Decibel 10th app, the researchers were able to determine the average decibel level during each observa- tional period. In addition to use of the iPad with the Decibel 10th app to assess the noise level within the classroom, the iPad with the app was used by classroom teachers to deter- mine whether the classrooms had met their noise-level goal. Prior to use of the Decibel 10th app in classrooms, the app was calibrated
using a RadioShack Digital Sound Level me- ter to ensure valid measurement of the class- room decibel level.
MotivAider The MotivAider is a device that pro-
vides a private tactile prompt (i.e., vibration) for a target behavior. The device can be set to vibrate at predetermined intervals based on either fixed or variable schedules. In the cur- rent study, the MotivAider was set at 2-min fixed intervals and was worn by teachers dur- ing the intervention phases to check their classroom’s decibel level on the iPad.
Tangible Rewards Prior to implementation of the interven-
tion, a researcher met with each teacher to iden- tify small tangible rewards to be provided to students for meeting decibel-level goals. Re- wards identified by teachers included small edi- bles (e.g., crackers, M&Ms) and stickers. Re- wards used in the current study represented items teachers were using as rewards in their classroom.
Measures
Three of the dependent measures were observation of student academically engaged behavior (AEB), disruptive behavior (DB), and noise level. The fourth and fifth dependent measures were teacher and student acceptabil- ity of the QCG.
Student Outcomes The primary dependent variable of the
current study was AEB, which was measured using a modified definition from the Behavior Observation of Students in Schools (Shapiro, 2004). AEB was inclusive of both passive and active engagement in the current study, similar to previous research (Briesch, Chafouleas, & Riley-Tillman, 2010; Ferguson, Briesch, Volpe, & Daniels, 2012; Hintze & Matthews, 2004). Passive engagement in AEB was considered to have occurred when the observed students had their eyes oriented toward their teacher or their assignment (e.g., reading, attention given to the teacher or assignment on the desk, silent reading). Active engagement in AEB was con-
Quiet Classroom Game
97
sidered to have occurred when the observed students were actively engaged in an academic activity (e.g., writing on an assignment, asking the teacher questions). For an interval to be coded as AEB, the observed students must have been in their assigned seats unless given permission otherwise by the teacher.
DB was collected as a secondary depen- dent variable. DB included playing with ob- jects (manipulating an object inconsistently with its intended use, manipulating a task- irrelevant object); out of seat (the student re- moves his or her buttocks from his or her seat without teacher permission); and inappropriate vocalizations (student utterances unrelated to the classroom activity). Passive off-task be- havior (e.g., head down, eyes oriented toward items not related to the academic task) was recorded by scoring an interval as neither AEB nor DB. As such, it was possible that the percentages of intervals scored as AEB and DB did not sum to 100%, with the remaining percentage indicating the observed level of passive off-task behavior.
A 10-s momentary time sampling pro- cedure was used to determine levels of AEB and DB during a 15-min observation session. Consistent with momentary time sampling procedures, the occurrence or nonoccurrence of AEB and DB was momentarily (i.e., 1 to 2 s) observed and recorded by observers via pencil and paper. Observers were prompted by an audio device via headphones to observe and record behavior at the end of 10-s intervals. To obtain a classroom composite of AEB and DB, observers used a rotating observation system, in which a different student was observed dur- ing each 10-s interval. During the first 10-s interval, the behavior of the first student in the first group of desks in the classroom was ob- served. During the next interval, the behavior of the second student in the first group of desks was observed. Observation of students continued in this manner, rotating through all students within the classroom by row until the 15-min observational period was complete. Previous research indicated the utility of rotating observation systems in accurately estimating true levels of behavior within classrooms (Bri- esch, Hemphill, Volpe, & Daniels, 2015).
The noise level of each classroom was measured in decibels. To assess the noise level, a researcher placed the iPad loaded with the Decibel 10th app in a predetermined loca- tion within the classroom. To ensure reliable assessment of the decibel level, the location of the iPad within the classroom was consistent across all observations. At the beginning of the intervention session, the researcher started the Decibel 10th app. During the entire 15-min intervention period, the Decibel 10th app col- lected and recorded the decibel level at 1/10-s intervals, compiling decibel-level data in a downloadable data sheet. At the conclusion of the intervention period, the researcher stopped the Decibel 10th app and downloaded the data sheet for analysis. The average decibel level per observation was calculated by averaging the decibel level recorded across all 1/10-s intervals.
Behavior Intervention Rating Scale The Behavior Intervention Rating Scale
(BIRS; Elliott & Von Brock Treuting, 1991) is a 24-item intervention acceptability measure completed by teachers. Items are endorsed on a 6-point Likert scale ranging from strongly disagree (1) to strongly agree (6), with higher scores indicating higher intervention accept- ability. Psychometric evaluations have sug- gested high construct validity and reliability, with an � coefficient of .97 for the entire scale and � coefficients of .97, .92, and .87 for Acceptability, Effectiveness, and Time of Ef- fectiveness factors (Elliott & Von Brock Treuting, 1991). The BIRS has shown moder- ate to high correlations with other measures of treatment acceptability (e.g., Elliott & Von Brock Treuting, 1991), as well as utility in discriminating between interventions (e.g., Von Brock & Elliott, 1987). In the current study, a minor modification was made to the wording of items, replacing “the intervention” with “the Quiet Classroom Game.” Previous research indicated that such modifications do not alter the psychometric properties of the measure (Mautone et al., 2009). Teachers were asked to complete the BIRS following the final intervention observation session.
School Psychology Review, 2016, Volume 45, No. 1
98
Children’s Intervention Rating Profile The Children’s Intervention Rating Pro-
file (CIRP; Witt & Elliott, 1985) is a six-item intervention acceptability measure completed by students. Items on the CIRP are endorsed on a 6-point Likert scale ranging from I agree (1) to I do not agree (6). Psychometric eval- uations suggest a single factor (i.e., acceptabil- ity) with an average � of .89 (Elliott, 1986). Similar to the BIRS, the CIRP has shown the capacity to discriminate between interventions (Turco & Elliott, 1990; Waas & Anderson, 1991). A minor modification was made to the wording of items, replacing “the method” with “the Quiet Classroom Game,” with such mod- ification consistent with previous research using the CIRP (Fiala & Sheridan, 2003; McQuillan, DuPaul, Shapiro, & Cole, 1996; Mong & Mong, 2012). In the current study, the researchers administered the CIRP to each classroom following the final intervention ob- servation. As the CIRP is written at a fifth- grade level (Finn & Sladeczek, 2001), the researchers facilitated completion of the CIRP by reading each item aloud to students.
Conditions
Four conditions were defined: baseline, intervention, withdrawal, and reimplementa- tion. They are described below.
Baseline During baseline, teachers were instructed
to continue to respond to classroom behavior in the manner that they typically would and they were not provided the MotivAider device. Stu- dent AEB and DB were recorded using the class- room observation procedure, as described. The iPad with the Decibel 10th app was placed in a predetermined location within the classroom. The iPad was placed such that teachers were able to see the decibel meter displayed on the app whereas students were unable to see the decibel meter. The researchers instructed the teachers not to provide students with feedback regarding the decibel level displayed by the app. The Decibel 10th app was started when obser- vation of AEB and DB was initiated and re- mained on for the duration of the 15-min obser- vation session.
Teacher Training After collection of baseline data, a one-
on-one meeting was arranged with each teacher to review the procedures of the QCG. The researchers stated their observational findings and discussed appropriate student be- havioral expectations. Training was delivered in three steps. First, the researchers provided teachers with a teacher script and described the procedures in a didactic fashion. Second, the researchers provided correct models of intervention implementation (described later). Finally, teachers were instructed to demonstrate appropriate intervention procedures with 100% accuracy during instances in which students did and did not meet their noise-level goal, with a researcher playing the student role. Teacher training lasted approximately 15 min.
Intervention During intervention, teachers imple-
mented the QCG. At the start of the observa- tion session, teachers recited the teacher script, which included descriptions of game rules, student behavioral expectations, students’ noise-level goal, indication that noise level would be monitored periodically, and the availability of a reward if students met their noise-level goal on at least five of seven noise- level checks. When the teacher completed the teacher script, observation of AEB and DB began, and the Decibel 10th app was started and remained on for the duration of the 15-min observation session. Noise-level goals during intervention were established by determining the median decibel level demonstrated during baseline and subtracting by 5 dB. Five deci- bels was selected as a criterion because re- search suggests this change represents a clearly noticeable difference in noise level (Cavanaugh, Tocci, & Wilkes, 2011).
Once the QCG began, teachers were prompted once every 2 min by the MotivAider to examine students’ noise level. When prompted, teachers quickly viewed the iPad displaying the students’ decibel level and provided positive or corrective feedback. Examples of feedback in- cluded “Class, we made our noise level goal because you were sitting quietly! Great job!” or “Class, we didn’t reach our goal, but don’t
Quiet Classroom Game
99
worry, we have more chances. Remember, to meet your goal you must all be sitting quietly and doing your work.” Teachers also placed a smiley face or a frowning faced within one of the seven boxes located on the classroom white- board, which allowed students to monitor their progress. At the end of the 15-min observation, during which time teachers provided seven in- stances of feedback regarding students’ decibel level, teachers indicated whether students met their goal of obtaining five of seven possible smiley faces. When at least five smiley faces were earned, teachers provided the tangible re- wards. When fewer than five smiley faces were earned, teachers stated that they would try again another day. Student AEB and DB were re- corded using the classroom observation proce- dure, as previously described.
Withdrawal The withdrawal condition was identical
to baseline. Teachers were instructed to con- tinue to respond to classroom behavior in the manner that they typically would, were in- structed to disregard the decibel meter dis- played on the iPad, and were not provided the MotivAider device. The Decibel 10th app was started when observation of AEB and DB was initiated and remained on for the duration of the 15-min observation session. No decibel-level feedback was provided at any point during with- drawal observations. Student AEB and DB were recorded using the classroom observation proce- dure, as previously described.
Reimplementation The procedures involved during the re-
implementation of the QCG were identical to those during the initial intervention implemen- tation condition.
Design
A concurrent multiple-baseline design across three classrooms with an embedded ABAB condition sequence was used to eval- uate the effects of the QCG on students’ AEB, DB, and noise level. Multiple baseline designs with embedded ABAB condition sequences allow for sequential application, comparison of intervention efficacy, and replication of in-
tervention effects both within and across par- ticipants (Cihak, Wright, & Ayers, 2010). Levels of student AEB, DB, and noise level were directly compared between baseline (A), implementation of the QCG (B), withdrawal of the QCG (A), and reimplementation of the QCG (B). Phase change decisions were made based on the stability of AEB and increasing or decreasing trends of AEB. Stability was operationalized as having a range of up to 15% variability across three consecutive observa- tions (Tawney & Gast, 1984). Data were an- alyzed visually by evaluating data trend, level, variability around level and trend, and magni- tude of change between conditions.
Reliability
Secondary observers were trained to ac- curately and reliably record occurrences and nonoccurrences of AEB and DB with a mini- mum of 80% interobserver agreement with primary observers before participating in the data collection process. Secondary observers were trained in observational procedures using video recordings of classrooms not included in the current study. Interobserver agreement was collected during at least 33.3% of observations per teacher. The results of observations com- pleted by the primary researchers were plotted when interobserver agreement was evaluated.
Interobserver agreement was calculated separately for AEB and DB. To calculate in- terobserver agreement, the total number of agreements was divided by the sum of agree- ments plus disagreements and multiplied by 100. Interobserver agreement was collected during 33.3%, 40.9%, and 47.6% of observa- tions for Teachers A, B, and C, respectively. Interobserver agreement data are reported in Table 1 and represent adequate levels of inter- observer agreement.
Intervention Integrity
To determine whether teachers imple- mented the QCG as intended, intervention in- tegrity was assessed in a yes or no manner by the primary observer following 100% of ob- servations. Specifically, teachers were re- quired to (a) announce the target decibel goal;
School Psychology Review, 2016, Volume 45, No. 1
100
(b) state students’ behavioral expectations (e.g., stay in seat, stay on-task, stay quiet); (c) indicate whether students met their decibel goal when prompted to by the MotivAider on seven separate occasions; (d) provide students with positive or corrective feedback during all seven instances when prompted by the MotivAider; and (e) provide a reward to each student, if earned, for obtaining a minimum of five smiley faces or indicate that students did not earn their reward and tell them to try harder the next time.
Intervention integrity was calculated by dividing the number of steps implemented cor- rectly by the number of steps possible and multiplying by 100. Intervention integrity was 100%, 99.1%, and 97.1% for Teachers A, B, and C, respectively. Intervention integrity in- terobserver agreement was obtained via sec- ondary observer and calculated by totaling the number of agreements between observers, di- viding that value by the number of steps pos- sible, and multiplying by 100. Intervention integrity interobserver agreement was calcu- lated for 33.3%, 40.9%, and 47.6% of obser- vations for Teachers A, B, and C, respectively, and was 100% across teachers.
RESULTS
The present study evaluated the effects of implementation of the QCG on student be- havior, as well as teacher perceptions of social validity of the intervention. Results are de- scribed below.
Academic Engagement Behavior
Figure 1 represents the percentage of AEB across classrooms. Prior to intervention, Teacher A’s classroom showed stable levels of AEB (M � 38.9%). After introduction of the QCG, an immediate increase in AEB was ob- served (M � 71.7%). Clear separation be- tween baseline and intervention was observed. During the following phase, withdrawal of the QCG, mean AEB decreased to 48.5%. Al- though variable, an immediate increase in level of AEB (M � 76.6%) was observed after the reintroduction of the QCG in Teacher A’s classroom.
Baseline levels of AEB in Teacher B’s class showed a decreasing trend (M � 41.1%). On introduction of the intervention, an imme- diate increase in mean level of AEB was ob- served (M � 77.6%). Clear separation was detected between the baseline and intervention phases. Withdrawal of the QCG resulted in a decrease in classroom AEB (M � 49.9%). On reintroduction of the intervention, AEB showed an improving trend and mean in- creased level to 70.4%.
Baseline levels of AEB in Teacher C’s class were relatively stable (M � 48.1%). Im- plementation of the QCG resulted in an im- mediate increase in mean level of AEB (M � 70.3%). Withdrawal of the interven- tion resulted in reductions in AEB (M � 45.1%). Reimplementation of the in- tervention produced immediate improve- ments in level and trend of classroom AEB (M � 74.6%). Clear separation was apparent between the withdrawal and reimplementa- tion phases.
Disruptive Behavior
Figure 1 represents the percentage of DB across classrooms. During baseline, DB in Teacher A’s classroom showed a decreasing trend and was observed during an average of 39.6% of intervals. Introduction of the QCG resulted in a further decrease in DB (M � 13.5%), and data showed clear sepa- ration from the baseline phase. Withdrawal of the intervention resulted in an increasing trend of DB (M � 28.9%). Although data
Table 1. Mean Interobserver Agreement for Measures of AEB and DB
Interobserver Agreement, Mean (Range)
AEB DB
Teacher A 90.7% (84.4%–95.6%) 90.2% (77.8%–98.9%) Teacher B 87.6% (78.9%–94.4%) 94.6% (87.8%–97.8%) Teacher C 85.4% (74.4%–95.6%) 92.5% (71.1%–98.9%)
Note. AEB � academically engaged behavior; DB � disruptive behavior.
Quiet Classroom Game
101
showed some variability and data overlap during the reintroduction phase, an immedi- ate decrease in level of DB was observed (M � 15.1%).
Prior to introduction of the QCG, DB in Teacher B’s classroom was observed during an average of 42.2% of intervals and showed an increasing trend. Introduction of the inter- vention was associated with an immediate and substantial reduction in DB (M � 10.2%). On entering the withdrawal phase, an increase in level was observed (M � 27.8%) and showed a decreasing trend. Reimplementation of the
intervention resulted in a return to interven- tion-phase levels of DB (M � 12.2%).
Baseline observations of Teacher C’s classroom indicated DB during an average of 23.0% of intervals. On entering the interven- tion phase, an immediate reduction in DB was observed (M � 5.5%). Although a full return to baseline levels was not observed during the withdrawal phase, removal of the intervention was associated with increased level of DB (M � 17.0%). Reimplementation of the inter- vention resulted in an immediate decrease in DB (M � 7.0%) and was stable throughout the phase.
Figure 1. Percentage of Intervals of Academically Engaged and Disruptive Behavior
Note. AEB � academically engaged behavior; BL � baseline; DB � disruptive behavior; INT � intervention; RI � reimplementation; WD � withdrawal.
School Psychology Review, 2016, Volume 45, No. 1
102
Decibel Level
Figure 2 represents average decibel lev- els per observation for each classroom. During baseline, the mean decibel level per observa- tion in Teacher A’s classroom was 62.6 dB and the decibel levels showed an increasing trend. Implementation of the intervention re- sulted in a reduction in mean level (M � 58.8 dB), with most data points indicating a clearly noticeable reduction (i.e., 5 dB; Cavanaugh, Tocci, & Wilkes, 2011) of classroom noise from median baseline level. Although consid-
erable variability was observed, withdrawal of the QCG resulted in a return to baseline levels of classroom noise (M � 62.2 dB). Reimple- mentation of the intervention resulted in a reduction of noise that exceeded that observed during the initial intervention phase (M � 57.1 dB), with all but one data point indicating a noticeable reduction in classroom noise.
The mean decibel level during baseline in Teacher B’s classroom was 68.2 dB and the decibel levels showed an increasing trend. In- troduction of the QCG resulted in decibel lev-
Figure 2. Average Decibels per Observation
Note. The dotted lines indicate a 5-dB reduction from median of baseline; the dashed lines indicate a 10-dB reduction from mean of baseline. BL � baseline; INT � intervention; RI � reimplementation; WD � withdrawal.
Quiet Classroom Game
103
els being noticeably reduced from median baseline levels during all intervention points (M � 59.0 dB) and clear separation from the baseline phase. Withdrawal of intervention procedures resulted in variable, yet increased, average decibel levels (M � 64.6 dB). Re- implementation of the intervention resulted in mean level reduction in average decibel levels (M � 59.9 dB), with all data points indicating a clearly noticeable reduction in noise.
Prior to intervention, the mean decibel level in Teacher C’s classroom measured 62.8 dB. After introduction of the intervention, an immediate and noticeable reduction in decibel level was observed (M � 53.6 dB). Clear separation between the baseline and interven- tion phases was evidenced. During the with- drawal phase, the mean decibel level increased to 60.0 dB but failed to fully return to baseline levels. Reimplementation of the QCG resulted in clear separation between the withdrawal- phase and reimplementation-phase data points (M � 53.3 dB), with all data points indicating clearly noticeable reductions in noise.
Social Validity
The BIRS (Elliott & Von Brock Treut- ing, 1991) was used to assess teacher accept- ability of the intervention used in the current study. The rating scale was administered to each of the teachers included in the study. Each found the intervention to be socially valid. Teacher A endorsed high ratings on the Acceptability (M � 5.53), Effectiveness (M � 4.71), and Time of Effectiveness (M � 5.00) factors. Teacher B provided similar endorsements for intervention Ac- ceptability (M � 5.20), Effectiveness (M � 4.14), and Time of Effectiveness (M � 5.00). Teacher C indicated high rat- ings on the Acceptability (M � 5.80) and Time of Effectiveness (M � 6.00) factors and slightly positive ratings on the Effec- tiveness factor (M � 3.43). With the excep- tion of two items, “Using the intervention should not only improve the student’s be- havior in the classroom but also in other settings” (M � 3) and “The intervention should produce enough improvement in a
student’s behavior so the behavior no longer is a problem in the classroom” (M � 3), all items were rated positively (i.e., �3) across teachers.
In addition, the CIRP (Witt & Elliott, 1985) was used to assess student acceptability of the QCG. The scale was administered to all students (N � 56) in each classroom. Overall, the intervention was found to be acceptable to students of Teacher A (M � 4.78), Teacher B (M � 5.06), and Teacher C (M � 4.89).
DISCUSSION
The primary purpose of the current study was to identify the presence of func- tional relationships between implementation of the QCG and an increase in students’ AEB. In addition, separate research questions aimed to identify the presence of functional relation- ships between implementation of the QCG and decreases in classroom noise and DB. Results of the study indicate that implementation of the QCG resulted in increases in AEB across all three classrooms during the intervention phases of the study. Similar, but slightly less robust, effects were observed in the reduction of classroom noise and DB. Overall, findings of the current study suggest that the imple- mentation of the intervention is likely to con- tribute to improved outcomes for early ele- mentary school (e.g., first grade) general-edu- cation classrooms that show high levels of inappropriate classroom noise and low levels of academic engagement.
The observed intervention effects are similar to those produced in empirical eval- uations of the Good Behavior Game (e.g., Flower et al., 2014; Tingstrom, Sterling- Turner, & Wilczynski, 2006), which have found implementation of the intervention to result in rapid reductions in off-task behavior. Critiques of the Good Behavior Game, as orig- inally proposed by Barrish, Saunders, and Wolf (1969), have suggested that the interven- tion may result in teachers focusing on nega- tive student behaviors (Babyak, Luze, & Kamps, 2000). Other findings suggest that teachers may be inconsistent in applying rules for earning marks (e.g., Lyon, Frazier, Mehta,
School Psychology Review, 2016, Volume 45, No. 1
104
Atkins, & Weisbach, 2011; Mitchell, 2012). Use of procedures incorporated in the QCG may overcome these critiques, as behavioral expectations are positively worded and receiv- ing marks is based on an objective measure (i.e., decibel reading) rather than relying on teacher evaluation of behavior.
Previous studies have supported the use of decibel level-based interventions for re- ducing noise within classroom settings (e.g., Schmidt & Ulrich, 1969; Strang & George, 1975). However, previous research evaluating decibel level-based interventions has failed to examine whether reductions in decibel level are associated with increases in behaviors pre- dictive of academic success (i.e., AEB; Ponitz, Rimm-Kaufman, Grimm, & Curby, 2009). Re- sults of the current study indicate that inter- ventions targeting classroom noise may not only modify classroom noise level but also promote student academic engagement.
The results of the current study support the utility of decibel level-based behavior management strategies by showing how such interventions may be implemented with mini- mal additional resources. Previous research using decibel level as a criterion for reinforce- ment has relied on materials often not readily available to school personnel (e.g., sound- level meters, stoplights), likely limiting the widespread use of these interventions in ap- plied settings. Given the abundance of smart phones, tablets, and other mobile devices within school settings, the current study sug- gests that no-cost decibel-level apps may be used in place of less readily available materi- als. Teachers without access to these devices may find the QCG too cost prohibitive to implement or may prefer less costly yet equally effective strategies; however, high levels of teacher and student acceptability fur- ther indicate that no-cost decibel-level apps may facilitate the use of decibel level-based group contingencies within school settings.
Limitations
Although results of the study indicated positive effects associated with implementa- tion of the QCG, the results must be consid-
ered in light of several limitations. First, the current study is limited in that only three first- grade classrooms were included, limiting the generalizability of the results to other subjects or age groups. Future investigations may ad- dress this limitation through replicating the current study with other populations.
Second, maintenance of the QCG’s ef- fects on students’ AEB was not formally as- sessed in the present study; however, the large and immediate decreases in class-wide AEB that were observed across all three classrooms during the withdrawal phase of the study sug- gest that a systematic reduction of the inter- vention’s intensity may be necessary to main- tain the intervention effects. In addition, when implementing the QCG, classroom teachers were interrupting their students to provide feedback regarding their noise level every 2 min. This relatively frequent interruption might not be feasible or acceptable to teachers, and fading the feedback component (i.e., fewer and less frequent checks) may be more appealing. Future research should investigate the degree to which systematic fading of the QCG results in maintenance of intervention effects.
Third, although phase changes were based on AEB, it must be noted that two classrooms showed decreasing trends of DB during the baseline phase, limiting conclusions regarding the effect of the program on DB. Future research may consider basing phase changes on stability on DB. It is also important to note that data for DB were collapsed across three different types of DB, and as such, conclusions are limited regarding the types of DB most affected by the intervention. The study is also limited in that the rate of noise-related feedback given by teachers during the baseline phase was not recorded. To more fully demonstrate that the systematic feed- back provided within the context of the QCG is more effective than other types of noise-related feedback, future researchers should consider re- cording rates of noise-related feedback during baseline phases.
Although designed to be administered to individual students, the CIRP has successfully been used in evaluating acceptability of class- wide interventions (Lannie & McCurdy, 2007). In addition, previous studies have used
Quiet Classroom Game
105
a read-aloud strategy to ensure comprehension of items by younger children (Galloway & Sheridan, 1994). However, administration of the measure when read aloud in a group for- mat may affect the psychometric properties of the measure. As such, findings regarding child acceptability should be interpreted with reser- vations. Lastly, it is unknown whether im- provements generalized to periods when the intervention was not in place. Future studies should investigate the generalization of effects to nonintervention periods.
CONCLUSION
The results of the current study demon- strate that the QCG may be used to increase AEB within early elementary school general- education classrooms, as well as decrease both the noise level and DB of students. Given the intervention’s procedural similarity to the Good Behavior Game, it is unsurprising that the group contingency was effective in man- aging behavior. However, procedures used in the current study may have advantages over the Good Behavior Game. Specifically, the objective criteria for earning rewards (i.e., decibel-level reading) may be particularly use- ful in avoiding inconsistency in earning points documented in some evaluations of the Good Behavior Game (Lyon et al., 2011; Mitchell, 2012). Given the negative correlation between classroom noise and academic performance, reductions in classroom noise associated with the QCG may be beneficial in promoting im- proved long-term outcomes in students (Shield & Dockrell, 2006; Shield & Dockrell, 2008). Although results of the current study provide preliminary evidence suggesting that the QCG may be effective for managing classroom be- havior, additional research is required.
REFERENCES
Babyak, A. E., Luze, G. J., & Kamps, D. M. (2000). The good student game: Behavior management for diverse classrooms. Intervention in School and Clinic, 35, 216–223.
Barrish, H. H., Saunders, M., & Wolf, M. M. (1969). Good behavior game: Effects of individual contingen- cies on disruptive behavior in a classroom. Journal of Applied Behavior Analysis, 2, 119–124.
Briesch, A. M., Chafouleas, S. M., & Riley-Tillman, T. C. (2010). Generalizability and dependability of behavior
assessment methods to estimate academic engagement: A comparison of systematic direct observation and Direct Behavior Rating. School Psychology Review, 39, 408–421.
Briesch, A. M., Hemphill, E. M., Volpe, R. J., & Daniels, B. (2015). An evaluation of observational methods for measuring response to classwide intervention. School Psychology Quarterly, 30, 37–49.
Cavanaugh, W. J., Tocci, G. C., & Wilkes, J. A. (2011). Architectural acoustics (2nd ed.). Hoboken, NJ: John Wiley and Sons.
Christ, T. J., & Christ, J. A. (2006). Application of an interdependent group contingency mediated by an au- tomated feedback device: An intervention across three high school classrooms. School Psychology Review, 35, 78–90.
Cooper, J. O., Heron, T. E., & Heward, W. L. (2006). Applied behavior analysis (2nd ed.). Columbus, OH: Merrill.
Crouch, P. L., Gresham, F. M., & Wright, W. R. (1985). Interdependent and independent group contingencies with immediate and delayed reinforcement for control- ling classroom behavior. Journal of School Psychol- ogy, 23, 177–187.
Davey, B., Alexander, M., Edmonson, C., Stenhoff, D., & West, R. P. (2001). The effects of a discriminative stimulus, paired with individual and group reward contingencies, on the decibel levels in an elementary school lunch room. Paper presented at the Growing Partnerships for Rural Special Education conference, San Diego, CA, March 29–31, 2001. ERIC Document Reproduction Service Manuscript Number RC-022-970.
Elliott, S. (1986). Children’s ratings of acceptability of classroom interventions for misbehavior: Findings and methodological considerations. Journal of School Psy- chology, 24, 23–34.
Elliott, S. N., & Von Brock Treuting, M. (1991). The Behavior Intervention Rating Scale: Development and validation of a pretreatment acceptability and effec- tiveness measure. Journal of School Psychology, 29, 43–51.
Ferguson, D., Briesch, A. M., Volpe, R. J., & Daniels, B. (2012). The influence of observation length on the dependability of data. School Psychology Quarterly, 27, 187–197.
Fiala, C. L., & Sheridan, S. M. (2003). Parent involvement and reading: Using curriculum-based measurement to assess the effects of paired reading. Psychology in the Schools, 40, 613–626.
Finn, C. A., & Sladeczek, I. E. (2001). Assessing the social validity of behavioral interventions: A review of treatment acceptability measures. School Psychology Quarterly, 16, 176–206.
Finn, J. D., & Zimmer, K. S. (2012). Student engagement: What is it? Why does it matter? In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds), Handbook of re- search on student engagement (pp. 97–131). New York, NY: Springer Science.
Flower, A., McKenna, J., Muething, C. S., Bryant, D. P., & Bryant, B. R. (2014). Effects of the Good Behavior Game on classwide off-task behavior in a high school basic algebra resource classroom. Behavior Modifica- tion, 38, 45–68.
Galloway, J., & Sheridan, S. M. (1994). Implementing scientific practices through case studies: Examples us-
School Psychology Review, 2016, Volume 45, No. 1
106
ing home-school interventions and consultation. Jour- nal of School Psychology, 32, 385–413.
Hintze, J. M., & Matthews, W. J. (2004). The generaliz- ability of systematic direct observations across time and setting: A preliminary investigation of the psycho- metrics of behavioral observation. School Psychology Review, 33, 258–270.
Kartub, D. T., Taylor-Greene, S., March, R. E., & Horner, R. H. (2000). Reducing hallway noise: A systems approach. Journal of Positive Behavior Interventions, 2, 179–182.
Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010). Single-case designs technical documentation. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf
Lannie, A. L., & McCurdy, B. L. (2007). Preventing disruptive behavior in the urban classroom: Effects of the good behavior game on student and teacher behav- ior. Education and Treatment of Children, 30, 85–98.
LaRowe, L. N., Tucker, R. D., & McGuire, J. M. (1980). Lunchroom noise control using feedback and group contingent reinforcement. Journal of School Psychol- ogy, 18, 51–57.
Lyon, A. R., Frazier, S. L., Mehta, T., Atkins, M. S., & Weisbach, J. (2011). Easier said than done: Interven- tion sustainability in an urban after-school program. Administration and Policy in Mental Health and Men- tal Health Services Research, 38, 504–517.
Maggin, D. M., Johnson, A. H., Chafouleas, S. M., Ru- berto, L. M., & Berggren, M. (2012). A systematic evidence review of school-based group contingency interventions for students with challenging behavior. Journal of School Psychology, 50, 625–654.
Mautone, J. A., DuPaul, G. J., Jitendra, A. K., Tresco, K. E., Junod, R. V., & Volpe, R. J. (2009). The relationship between treatment integrity and accept- ability of reading interventions for children with atten- tion-deficit/hyperactivity disorder. Psychology in the Schools, 46, 919–931.
McKissick, C., Hawkins, R. O., Lentz, F. E., Hailley, J., & McGuire, S. (2010). Randomizing multiple contin- gency components to decrease disruptive behaviors and increase student engagement in an urban second- grade classroom. Psychology in the Schools, 47, 944–959.
McQuillan, K., DuPaul, G. J., Shapiro, E. S., & Cole, C. L. (1996). Classroom performance of students with serious emotional disturbance: A comparative study of evaluation methods for behavior management. Journal of Emotional and Behavioral Disorders, 4, 162–170.
Mitchell, R. R. (2012). The effects of the good behavior game with general education high school students (Un- published master’s thesis). University of Southern Mississippi, Hattiesburg, MS.
Mong, M. D., & Mong, K. W. (2012). The utility of brief experimental analysis and extended intervention anal- ysis in selecting effective mathematics interventions. Journal of Behavioral Intervention, 21, 99–118.
Ponitz, C. C., Rimm-Kaufman, S. E., Grimm, K. J., & Curby, T. W. (2009). Kindergarten classroom quality, behavioral engagement, and reading achievement. School Psychology Review, 38, 102–120.
Schmidt, G. W., & Ulrich, R. E. (1969). Effects of group contingent events upon classroom noise. Journal of Applied Behavior Analysis, 2, 171–179.
Shapiro, E. S. (2004). Academic skills problems work- book. New York, NY: Guilford Press.
Shield, B. M., & Dockrell, J. E. (2006). Acoustical bar- riers in classrooms: The impact of noise on perfor- mance in the classroom. British Educational Research Journal, 32, 509–525.
Shield, B. M., & Dockrell, J. E. (2008). The effects of environmental and classroom noise on the academic attainments of primary school children. The Journal of the Acoustical Society of America, 123, 133–144.
Simonsen, B., Fairbanks, S., Briesch, A., Myers, D., & Sugai, G. (2008). Evidence-based practices in classroom manage- ment: Considerations for research to practice. Education and Treatment of Children, 31, 351–380.
Skinner, C. H., Williams, R. L., & Neddenriep, C. E. (2004). Using interdependent group-oriented rein- forcement to enhance academic performance in general education classrooms. School Psychology Review, 33, 384–397.
Stage, S. A., & Quiroz, D. R. (1997). A meta-analysis of interventions to decrease disruptive classroom behav- ior in public education settings. School Psychology Review, 26, 333–368.
Staub, R. W. (1990). The effects of publicly posted feed- back on middle school students’ disruptive hallway behavior. Education & Treatment of Children, 13, 249–257.
Strang, H. R., & George, J. R. (1975). Clowning around to stop clowning around: A brief report on an automated approach to monitor, record, and control classroom noise. Journal of Applied Behavior Analysis, 8, 471–474.
Tawney, J. W., & Gast, D. L. (1984). Single-subject research in special education. New York, NY: Macmillan.
Tingstrom, D. H., Sterling-Turner, H. E., & Wilczynski, S. M. (2006). The good behavior game: 1969–2002. Behavior Modification, 30, 225–253.
Turco, T. L., & Elliott, S. N. (1990). Acceptability and effectiveness of group contingencies for improving spelling achievement. Journal of School Psychology, 28, 27–37.
Von Brock, M. B., & Elliott, S. N. (1987). The influence of treatment effectiveness information on the accept- ability of classroom interventions. Journal of School Psychology, 25, 131–144.
Waas, G. A., & Anderson, G. P. (1991). Outcome expec- tancy and treatment acceptability: Perception of school-based interventions. Professional Psychology: Research and Practice, 22, 149–154.
Winett, R. A., & Winkler, R. C. (1972). Current behavior modification in the classroom: Be still, be quiet, be docile. Journal of Applied Behavior Analysis, 5, 499– 504.
Witt, J. C., & Elliott, S. N. (1985). Acceptability of classroom intervention strategies. In T. R. Kratochwill (Ed.), Advances in school psychology (Vol. 4, pp. 251–288). Hillsdale, NJ: Lawrence Erlbaum.
Date Received: June 17, 2014 Date Accepted: January 12, 2015
Associate Editor: Amanda VanDerHeyden Article accepted by previous Editor �
Quiet Classroom Game
107
Keith C. Radley is an assistant professor in the School Psychology Program at the University of Southern Mississippi. His research focuses on the application of behavioral interventions to address classroom behavior and social skills.
Evan H. Dart is an assistant professor in the School Psychology Program at the University of Southern Mississippi. His research focuses on peer-mediated behavioral interventions and school-based consultation tactics.
Roderick D. O’Handley is a graduate student in the School Psychology Program at the University of Southern Mississippi. His research interests include compliance training, social skills training, and treatment integrity in applied settings.
School Psychology Review, 2016, Volume 45, No. 1
108
Copyright of School Psychology Review is the property of National Association of School Psychologists and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
