Evidence base practice

OpportunitiestoRespond.pdf

Home >Education homework help >Evidence base practice

A Comparison of Three Types of Opportunities to Respond on Student Academic and Social Behaviors

Todd Haydon University of Cincinnati, Ohio Maureen A. Conroy Virginia Commonwealth University, Richmond Terrance M. Scott University of Louisville, Kentucky Paul T. Sindelar Brian R. Barber Ann-Marie Orlando University of Florida, Gainesville

An alternating treatments design was used to investigate the effects of three types of opportunities to respond (i.e., indi- vidual, choral, and mixed responding) on sight words and syllable practice in six elementary students with behavioral problems. During the mixed responding condition, five out of six students demonstrated a lower rate of disruptive behavior, and four out of six students had fewer intervals of off-task behavior. Results of the three types of opportunities to respond on participants’ active student responding were less clear. A discussion of limitations, implications, and future research directions is included.

Keywords: individual responding; choral responding; active student responding; disruptive behavior

Fletcher, & Hennington, 1996). Researchers have shown that improving the quality and increasing the quantity of learning trials results in higher learning rates (Barbetta & Heward, 1993; Carnine, 1976; Miller, Hall, & Heward, 1995). An example of a learning trial is when a teacher presents a science word on a flash card (i.e., stimulus), the student recites the word aloud (i.e., response), and the teacher then says, “Good answer” (i.e., consequent) (Skinner, Belfiore, Mace, William-Wilson, & Johns, 1997). Researchers have shown that increasing the num- ber of learning trials could increase learning levels during the acquisition, fluency building, and maintenance stages of learning (Skinner, Smith, & McLean, 1994).

Using choral responding is one instructional strategy that increases both learning trial rates and learning rates during teacher-led instruction (Skinner et al., 1996). Choral

Journal of Emotional and Behavioral Disorders Volume 18 Number 1

Disabilities 10.1177/1063426609333448

http://jebd.sagepub.com hosted at

http://online.sagepub.com

Teachers in general education classrooms typically use the lecture format during large group instruction and expect that their students passively watch and listen while course content is presented. The common ques- tioning procedure used with this style of instruction is asking individual students to volunteer by raising their hands (Armendariz & Umbreit, 1999). However, a limi- tation of this instructional method is that only a handful of students, usually higher achievers, actively respond to teachers’ questions (Greenwood, 2001; Greenwood, Delquadri, & Hall, 1984). In the past, Good (1970) found that students, in particular students who are low achiev- ers, were not provided equal opportunities to respond and frequently passively watched and listened as their higher achieving peers answered questions. As a result, low achieving students may often fail to receive the prac- tice and feedback that is necessary for achievement gains.

To increase teacher rates of opportunities to respond, researchers have theorized and conceptualized instruction as having a basic unit of instruction called a learning trial. A learning trial consists of a three-term, stimulus- response-consequent contingency sequence (Skinner,

Authors’ Note: This research and preparation of this article have been supported in part by an OSEP doctoral leadership grant. Please address correspondence to Todd Haydon, ML 0022, 600F Teachers/ Dyer Hall, CECH, University of Cincinnati, Cincinnati, OH 45221-0022; e-mail: [email protected].

28 Journal of Emotional and Behavioral Disorders

responding occurs when all students are asked to respond following the presentation of an instructional stimulus (Heward, 1994). The purpose of using choral responding is to increase the number of active student responses and, as a result, increase the number of correct responses and the amount of time students are engaged during instruction while allowing the teacher to monitor each student’s under- standing of each question (Carnine, 1976; McKenzie & Henry, 1979; Miller et al., 1995; Sainato, Strain, & Lyon, 1987; Sutherland, Alder, & Gunter, 2003).

Providing students frequent opportunities to respond is important because researchers suggest that increased stu- dent responding is linked to on-task behavior and engage- ment during instruction (Carnine, 1976; Sainato et al., 1987; Sutherland et al., 2003). When students are engaged and actively responding to questions, teachers can focus on academic content rather than being concerned with inap- propriate student behaviors. Increasing the focus on academic content is particularly important for teachers who instruct students with or at risk for emotional or behavioral disorders (EBD), because students with or at risk for EBD are more likely to engage in inappropriate behaviors than their typically achieving peers (Hastings & Oakford, 2003; Nelson & Roberts, 2002).

In a study designed to verify various effective instruc- tional techniques, Anderson, Evertson, and Brophy (1979) found that during small group first-grade reading groups, choral responding was negatively related to achievement and individual responding (ordered turns) was positively related to achievement. However, McKenzie and Henry (1979) compared an individually addressed question condition with a unison hand-raising condition in two third-grade classrooms and found that students in the unison hand-raising condition had significantly fewer intervals of off-task behavior than students in the indi- vidual hand-raising condition. Results from several more recent studies indicate that increased rates of opportuni- ties to respond by using choral responding produced a higher percentage of intervals of on-task behavior. Sutherland et al. (2003) demonstrated that when a teacher increased his rate of opportunities to respond and used choral responding, nine students identified as EBD in a self-contained classroom had more correct responses, fewer disruptions, and increased on-task behavior during math lessons. Haydon, Mancil, and Van Loan (in press) systematically replicated the Sutherland et al. (2003) study. Similar to the Sutherland et al. study, when the teacher increased his rate of opportunities to respond and used choral responding in a general education classroom setting, a fifth-grade student identified as at risk for EBD had higher percentages of on-task behavior and correct

responses and lower rates of disruptive behavior than during the individual responding condition.

Two more studies lend support for the advantage of using choral responding over individual responding. Sainato et al. (1987) compared choral responding with a baseline individual responding condition with three pre- school children identified as having significant behavioral and developmental delays during morning circle time. Sainato and colleagues compared the use of two rates (three/min; five/min) of choral responding with a baseline individual responding condition and results indicated that on-task behavior and correct responding improved during the higher rate of choral responding. In a similar investiga- tion, Sindelar, Bursuck, and Halle (1986) compared two modes of responding: ordered and choral. Findings sug- gested a slight but significant difference between sight words mastered across all three groups of students during the choral responding condition in comparison with the ordered response condition. On a post-instruction test, the students in the choral responding condition had a higher percentage of words read correctly than the students in the ordered responding condition. There was not a substantial difference in the percentage of on-task behavior between conditions (83% for the choral responding condition and 79% for the ordered responding condition).

However, Wolery, Ault, Doyle, Gast, and Griffin (1992) compared choral versus individual responding in small group arrangements and had contrasting findings to earlier findings. Results on the effectiveness of the two types of responding differed depending on the amount of opportu- nities to respond provided and student exposure to ques- tions during each condition. The authors concluded that the two types of responding produced relatively equal learning and only a slight difference in effectiveness and efficiency (choral over individual responding) were found. However, these results could be due to a small group set- ting and may not replicate in a large group classroom set- ting where students may typically receive fewer individual opportunities to respond and be required to passively lis- ten for longer periods of time.

Whereas researchers have compared choral and indi- vidual responding, Stevens and Rosenshine (1981) sug- gested that using mixed responding (a ratio of 70:30 choral to individual responding) might be a more effec- tive and efficient instructional strategy. They hypothe- sized that students could benefit from frequent practice of choral responding, whereas teachers could test specific children and gain information on individual performance by using individual responding. In classroom-based research, it is important to determine the most efficient method of instruction to increase the likelihood that

Haydon et al. / Types of Opportunities to Respond 29

teachers will use that strategy in the future. Comparing types of instructional strategies is one way to determine which instructional strategies produce the best results (Skinner, Johnson, Larkin, Lessley, & Glowacki, 1995).

This study addresses this issue and extends the learning trial literature in several ways. First, the effectiveness of decreasing students’ disruptive and off-task behavior as well as increasing students’ active student responding was exam- ined by comparing three types of opportunities to respond (individual, choral, and a mixture of 70% choral responding and 30% individual responding) in a second-grade general education classroom setting. Second, the three types of opportunities to respond represented the use of an antecedent instructional strategy in the beginning of a learning trial as opposed to an error correction strategy at the end of a learning trial. Third, the three types of opportunities to respond were used with students identified as at risk for EBD.

The purpose of this study was to investigate the follow- ing research question: What effects do choral responding, individual responding, and a mixture of choral and indi- vidual responding procedures have on the disruptive, off- task behavior, and active student responding of students identified as high risk for EBD during group instruction in a general education classroom?

Method

Participants

To recruit participants, the first author contacted assis- tant principals at elementary schools in a southern school district to determine if the school was interested in partici- pating in the study and if they had any second-grade teach- ers who would be interested in being participants. Two schools in the school district and three second-grade teachers from each school volunteered to participate.

Teachers. Six teachers were recruited and served as participants in this study. Teacher participants (a) had a minimum of 2 years of teaching experience, (b) used less than two opportunities to respond per minute during a pre-assessment condition, and (c) consented to partici- pate in the study. All six teachers were Caucasian, and five of the six teachers were female. The average years of teaching experience was 3.0 (range = 2–6 years), and all six teachers had taken a behavior management class as undergraduate students.

Students. Six students identified as having chronic disruptive behaviors that placed them at risk for EBD participated in this study. Table 1 reports information

on each participant’s gender, ethnicity, age, and risk score. To determine if the student participants were at risk for EBD, the Systematic Screening for Behavior Disorders (SSBD; Walker & Severson, 1993) was con- ducted. The SSBD is an empirically validated, multiple gating procedure used to identify students with pat- terns of internalizing and externalizing behaviors (Walker & Severson, 1993). Identification occurs in three stages: (a) teacher nomination of students with externalizing or internalizing behavior problems, (b) ranking and evaluation of the top three students in either category, and (c) observation of these students by another profes- sional in the classroom and playground setting.

Stage 1 of the SSBD was one of the regular practices used by the school to identify students at elevated risk for EBD. Thus, a separate consent was not needed for ranking students in the Stage 1 process. Following Stage 1, the first author obtained consent from the parents of the students who were ranked highest in the Stage 1 process in each class. Informed consent granted permission to implement the second and third stages of the SSBD (Walker & Severson, 1993) and participate in the study if scores indicated that the student was at high risk. Once informed consent was obtained, Stage 2 of the SSBD was conducted. All participants targeted for Stage 2 met criteria for inclusion in the study.

The following eligibility criteria were used to identify participants: They (a) were rated by the teachers as hav- ing high rates of disruptive behavior for more than 1 month according to the critical events index and com- bined frequency index on the SSBD, (b) were enrolled in a second-grade general education class, (c) were between the ages of 7 and 8, and (d) had parental consent to par- ticipate in the study.

Setting and Materials

Setting. The setting for the study was six second-grade general education classrooms in the south. Two schools,

Table 1 Participant Characteristics

SSBD Name Gender Ethnicity Age Score

Frank male African American 7 years 6 months 25/35 D’Andy male African American 8 years 2 months 31/35 Monty male African American 7 years 5 months 29/35 Teo male African American 8 years 2 months 32/35 Amber female African American 8 years 2 months 30/35 Mats male Caucasian 7 years 6 months 27/35

Note: SSBD = Systematic Screening for Behavior Disorders.

30 Journal of Emotional and Behavioral Disorders

one urban and the other suburban, were selected. Class size ranged from 18 to 22 students. Participants and Teachers 1 through 3 attended the urban school, whereas Participants and Teachers 4 through 6 attended the sub- urban school. The racial/ethnic make-up of the classrooms in the urban school was approximately 70% African American and 30% Caucasian, whereas in the suburban school, the percentage was roughly 50% African American and 50% Caucasian. This study took place during a large group instruction, teacher-directed academic activity that had the potential for high rates of opportunities to respond (Skinner et al., 1996). All instructional activities took place in the morning.

Materials. During the targeted activity, materials that are commonly used for language arts instruction were used in the study (5 in. × 7 in. flash cards). The primary experimenter developed, along with teachers, consistent lesson plans and instructional materials to teach content vocabulary and syllable practice. All six teachers used sight words that were at an equivalent level of difficulty and indicated that the content of the cards covered the same stories and review of previous spelling tests. The set of flash cards was the same for Teachers 1 and 3 and Teachers 4, 5, and 6. Teacher 2 opted to use her own sight word cards.

Teacher Training

Teacher training consisted of two phases: (a) informa- tion sharing and (b) practice until mastery occurred. Training was implemented during two 30-minute prac- tice sessions on two separate days based on procedures employed by Sutherland et al. (2003). The first author trained Teachers 1 through 3 and Teachers 4 through 6 on separate occasions.

Phase 1: Information sharing. The first author pre- sented a review of the operational definition of opportu- nities to respond and showed a 5-minute viewing of video clips of teachers using individual and choral responding. He then explained the expectations, proce- dures, and rules for the three responding conditions.

Phase 2: Practice until mastery. Practice of the three types of opportunities to respond consisted of (a) show- ing a sight word card to the class; (b) cuing the students verbally “5-4-3-2-1” to allow adequate wait time, provid- ing a verbal prompt to respond; (c) providing feedback on whether the answer was correct or incorrect (e.g., “That is correct,” or “That is not correct. The correct answer is

______.”); and (d) selecting another sight word card and beginning the next learning trial (Heward, Courson, & Narayan, 1989).

Comparison of the Three Interventions

Based on a randomized schedule, the teacher was instructed to implement either (a) individual responding, (b) choral responding, or (c) mixed mode responding— all at a rate of approximately five per minute. This rate was selected based on findings from Sainato et al. (1987) that suggested only slight differences between rates of three and five opportunities to respond per minute. The faster rate of five opportunities to respond per minute was selected because this rate produces a pacing that is appropriate for review of sight words and syllable prac- tice (Heward, 1994).

During each of the three conditions, teachers imple- mented the four-step procedure for each learning trial as indicated in Phase 2 of the teacher training phase: The four-step procedure was identical for the three conditions (except for the second step, cueing procedures differed slightly for each condition; see below).

Individual responding. In this condition, the teacher followed the four-step procedure; reviewed the proce- dures, expectations, and rules for individual responding; and called on each student randomly to pronounce the word on the sight word card or indicate how many syl- lables were in the word. During this condition, the total exposure to questions (i.e., the number of times a student saw a sight word card presented either to oneself or to a peer) was approximately 40 over the entire session, however, the teacher ensured that the number of indi- vidual opportunities to respond (the number of times the teacher asked the targeted student to respond per word) was three.

Choral responding. In this condition, the teacher fol- lowed the four-step procedure; explained the expecta- tions, procedures, and rules for the choral responding condition (specifically cueing procedures); cued all the students by saying “group”; showed a sight word card to the class; and cued the entire class to respond. During this condition, the total exposure to questions and opportuni- ties to respond was equal and was approximately 40 ques- tions over the entire session for the targeted student.

Mixed responding. In this condition, the teacher explained the expectations, procedures, and rules for the mixed responding condition (specifically cueing

Haydon et al. / Types of Opportunities to Respond 31

procedures) and read from a list developed by the researcher indicating the type of opportunity to respond, either a choral or an individual opportunity to respond. For each individual response, the teacher said, “This is individual,” showed a sight word card, read the defini- tion, counted down from five, called on a student, and asked, “What word?” For each choral response, the teacher said, “This is for everyone,” showed a sight word card, read the definition, counted down from five, and asked the entire class, “What word?” Using a ratio of 70% choral to 30% individual at a rate of approximately five opportunities to respond per minute yielded 28 cho- ral responses to 12 individual responses for the 8-minute session. During choral responding, all students includ- ing the targeted student responded 28 times. During individual responding, the targeted student responded 3 times and 9 peers were randomly called to respond once. During this condition, the total exposure to ques- tions for the targeted student was approximately 40 over the entire session, whereas the number of opportunities to respond was 31.

Dependent Measures

The dependent measures for this study included (a) disruption, (b) off-task behavior, and (c) active stu- dent responding. Disruptive behavior was defined as any behavior demonstrated by the target student that inter- rupted the flow of instruction or was disruptive to the on-task behavior of other students. The following behav- iors are examples of disruptive behaviors: getting up from seat, touching others, talking to another student, speaking out loud without raising hand, taking things from others, throwing objects, making noise (tapping, banging), moving head up and down or from side to side, talking to others, rocking in chair, and so forth (Armendariz & Umbreit, 1999).

Because the activity required student eye contact with the flash card, off-task behavior was defined as occurring when the target student was not actively directed (looking) toward the teacher (e.g., looking around the room, looking at or drawing on the desk, playing with materials in the desk, hair, or clothes, etc.; Miller et al., 1995).

Active student response was defined as engaging in the behavior that was expected during the specific opportuni- ties to respond condition and included (a) independent hand raising for the individual responding, (b) responding in unison with the group for choral responding, or (c) both, in the mixed responding condition (Godfrey, Grisham- Brown, Schuster, & Hemmeter, 2003).

Measurement

All observations lasted a total of 8 minutes; the first 4 minutes consisted of review of sight words followed by 4 minutes of syllable practice using the same sight words. During this time, the primary researcher served as the primary observer and collected real-time data using direct sequential recording of the teachers’ use of oppor- tunities to respond followed by student active responding during the activity period. Student disruptive and off-task behaviors were also recorded during the activity period using direct recording. Data were collected using a paper-and-pencil data collection system.

Prior to data collection, the first author spent time in each classroom for 30 minutes on five separate occasions to familiarize the students with his presence and decrease potential reactivity. To accurately capture the occurrence of both discrete and continuous behaviors, different types of measurement strategies were used. Student disruption was measured using a frequency count and translated into rate per minute using the following formula: frequency of disruption/total number of minutes (i.e., 8 min). Active student responses were measured using a percentage for- mula derived from counting the number of active student responses following a teacher’s use of a specific opportu- nity to respond strategy (i.e., individual, choral, or mixed responding) and dividing each of those numbers by the total number of questions the student was exposed to.

Student off-task behavior was measured using momen- tary time sampling. During the 8-minute observation period, the primary observer continuously observed the teacher and target student. The observer was cued every 20 seconds (by a taped tone) to look at the targeted student and code if the student was off-task at that moment (Gunter et al., 2003). Because the length of each session was 8 minutes, there was a total of 24 observations for off-task behavior.

Interobserver Agreement

To provide evidence that the measures of the dependent variables were accurate, secondary observer(s) collected interobserver agreement (IOA) data within each condition of the study (Kennedy, 2005). Data collectors were aware of which condition was being observed (i.e., individual, choral, or mixed), but they were unaware of the relative effectiveness of any condition on the dependent variables.

IOA checks for the dependent variable of disruption were scored by exact event occurrence only formula (i.e., an agreement was scored when two observers scored the same number of events of disruption during each interval of observation) and calculated by using an

32 Journal of Emotional and Behavioral Disorders

interval agreement formula, dividing the total number of agreements by the total number of agreements and dis- agreements and multiplying by 100%. Off-task behavior was calculated using an interval agreement formula. Active student responding interobserver agreement was calculated by using a total agreement method. Both observers maintained a frequency count of active student responding and agreement was computed by dividing the smaller total of occurrence of responding by the larger total occurrence of responding and multiplying by 100%. Prior to beginning data collection and IOA data, the primary and secondary observer(s) were trained to a reli- ability of at least 85% for three consecutive sessions on each dependent measure. To control for observer drift, the primary observer met with the secondary observer(s) on a weekly basis and/or repeated the training exercises once every five sessions (Cooper, Heron, & Heward, 1987). Interobserver agreement was calculated during 33.8% of observations. Average interobserver agreement for disruption was 93.02% (range = 75–100%), off-task 91.5% (range = 80.0–100%), and active student respond- ing 98.63% (range = 90.47–100%).

Treatment Integrity

Direct measurement of the independent variable (i.e., teacher’s implementation of the opportunities to respond procedure, i.e., individual, choral, or mixed mode at a rate of five/min) was conducted as a measure of treatment integrity on approximately 15% of the sessions by two secondary observers. Having two observers allowed the researchers to calculate IOA on integrity. Although this is not typically done (Yarbrough, Skinner, Lee, & Lemmons, 2004), clear support for the claim that the treatment was implemented as intended can be made when there is agreement between two independent observers (Noell & Witt, 1998). Interobserver agreement for treatment integ- rity was 100% for the rate of opportunities to respond (between 4.5 and 5.0 per min), the start of syllable prac- tice, sequence of steps, and steps in the sequence (cue, wait time, questions, and feedback given).

A checklist sheet was used to record the occurrence or nonoccurrence of each step of the opportunities to respond instructional sequence in the individual, choral, and mixed conditions as described above. The accuracy of the teach- ers’ implementation of the individual, choral, and mixed procedures—the four components: cueing students, allow- ing adequate wait time (counting down by 5), asking ques- tions, and providing feedback on student responses, as well as the number of opportunities to respond per 8-minute session—was calculated using the total agreement approach.

In addition, the accuracy of the teachers’ start of the imple- mentation of syllable practice after 4 minutes (within 10 s) was also calculated. During mixed responding, two observ- ers followed the teachers’ verbal prompt (i.e., “This is individual.” “This is group.”) and recorded on the treat- ment integrity checklist the accuracy with which the teacher implemented the 70:30 ratio. They also recorded the number of questions asked to the targeted student.

Social Validity

After the completion of the study, the teachers were asked to complete three social validity surveys to obtain information about their perception of the acceptability and usefulness of each type of opportunities to respond. Teachers rated nine questions using a 4-point Likert- type scale, where 1 represents not at all and 4 represents very much. The rating scale consisted of three catego- ries: (a) teacher’s perceived ease of implementing each type of intervention, (b) teacher’s perceived effectiveness of each type of intervention, and (c) teacher’s likelihood of using each intervention in the future. Mean scores for each question were calculated by totaling each teacher’s response and dividing by 6.

Experimental Design and Procedures

An alternating treatments design (Barlow & Hayes, 1979) was used to compare the three types of opportuni- ties to respond (i.e., individual responding vs. choral responding vs. mixed responding [70% choral respond- ing and 30% individual responding]). All sessions were 8 minutes in length; the first 4 minutes consisted of review of sight words followed by 4 minutes of syllable practice using the same sight words. Differences between conditions were determined mainly by noting distinct separation of data points using visual inspection as well as by examining mean difference between conditions. Trend lines were determined by using a split-middle trend estimation line (Kazdin, 1982).

Results

Treatment Integrity

Treatment integrity data were collected for each teacher to assess the implementation of each type of responding condition. Data indicated that the six teachers implemented the rate of opportunities to respond (between 4.5 and 5.0 per min) and the start of syllable practice 100% of the time. For Teachers 4 through 6, integrity on sequence of steps was 100%, and integrity on steps in the

Haydon et al. / Types of Opportunities to Respond 33

sequence was 100% for cue, wait time, questions, and feedback given. For Teacher 1, integrity on sequence of steps averaged 98.5% (range = 94.03–100%), and integ- rity on steps in the sequence averaged 100% for cue, wait time, and questions, and 94.03% for feedback. For Teacher 2, integrity on sequence of steps averaged 99.8% (range = 99.2–100%), and integrity on steps in the sequence was 100% for cue, wait time, and questions, and 99.2% for feedback (range = 97.6–100%). For Teacher 3, integrity on sequence of steps averaged 99.7% (range = 98.68–100%), and integrity on steps in the sequence averaged 100% for cue, wait time, and ques- tions, and 98.68% for feedback (range = 97.36–100%).

Disruptive Behavior

Table 2 summarizes the means and ranges of disruptive behavior per minute, percentage of intervals of off-task behavior, and percentage of active student responses for the six students across the three types of opportunities to respond. For five out of six students, the mean rate of disrup- tive behavior per minute was less during the mixed responding condition than during individual responding or choral responding conditions. Because of the large vari- ability and extreme scores in Teo’s data, median scores rather than mean scores are reported (Borg & Gall, 1989).

Figure 1 depicts the rate of disruptive behavior per ses- sion for the six students. The level of disruptive behavior was typically lowest during mixed responding and high- est during individual responding for all students except for Teo. With the exception of Teo, all the participants’ data indicated stable trend lines with little variability.

Off-Task Behavior

Table 3 summarizes the mean scores and ranges of percentage of intervals of off-task behavior per type of opportunities to respond across students. Five out of six students demonstrated a lower mean percentage of off-task behavior in the mixed responding condition in comparison

with the individual and choral responding conditions. One student (Amber) demonstrated a slightly lower mean per- centage of off-task behavior in the choral responding con- dition than in the mixed responding condition.

As indicated in Figure 2, the level of off-task behavior was typically lowest during the mixed responding condi- tion for Frank, D’Andy, Monty, and Mats, whereas the level of off-task behavior was typically highest during the individual responding condition for Frank, Monty, Amber, and Mats. For Amber, the level of off-task behavior was lowest during the last two data points of the choral responding condition. For D’Andy, the level of off-task behavior was highest during the last two data points of the choral responding condition. Similar to dis- ruptive behavior, Teo’s data indicated a great deal of overlap between the conditions with no clear differences in magnitude, and with moderate to large variability.

Active Student Responding

The means and ranges for active student responses across the six participants are presented in Table 4. All six students demonstrated a higher mean percentage of active student responding in the mixed responding con- dition in comparison with the individual responding condition and a higher percentage of active student responding in the choral responding condition in com- parison with individual responding. Results for active student responding between the mixed and choral responding were less clear. Three students (Frank, D’Andy, and Monty) demonstrated a higher mean per- centage of active student responding in the mixed responding condition in comparison with the choral responding condition, whereas three students (Teo, Amber, and Mats) demonstrated a higher mean percent- age of active student responding in the choral responding condition than in the mixed responding condition.

As indicated in Figure 3, the level of percentage of active student responding was typically highest during

Table 2 Means, Standard Deviations, and Ranges for Disruptive Behavior in Each Condition

Individual Choral Mixed

Student M (SD) Range M (SD) Range M (SD) Range

Frank 1.54 (0.32) 1.25–1.88 0.71 (0.27) 0.25–1.00 0.16 (0.12) 0.00–0.38 D’Andy 0.89 (0.33) 0.50–1.50 0.43 (0.20) 0.25–0.75 0.19 (0.11) 0.00–0.38 Monty 1.21 (0.19) 1.00–1.50 0.81 (0.31) 0.50–1.38 0.49 (0.19) 0.25–0.75 Teo 1.52 (0.87) 0.63–2.75 1.65 (1.28) 0.38–4.13 1.61 (0.48) 0.88–2.75 Amber 1.33 (0.14) 1.13–1.50 0.90 (0.13) 0.75–1.13 0.44 (0.22) 0.16–0.75 Mats 1.25 (0.16) 1.00–1.36 0.35 (0.09) 0.25–0.50 0.08 (0.07) 0.00–0.13

34 Journal of Emotional and Behavioral Disorders

the mixed responding condition for Frank, D’Andy, and Monty, whereas the level of percentage of active student responding was typically highest during the choral responding condition for Amber, Teo, and Mats. For Frank, Monty, Amber, and Mats, the level of percentage of active student responding was typically lowest and had the greatest amount of variability during individual

responding. The least amount of variability for the six participants occurred during mixed responding.

Social Validity

At the end of the study, all six teachers completed the social validity questionnaire, which consists of nine

0.5

1.5

2.5

3.5

4.5

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 0

0.5

1.5

2.5

3.5

4.5

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

0.5

1.5

2.5

3.5

4.5

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 0

0.5

1.5

2.5

3.5

4.5

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

0.5

1.5

2.5

3.5

4.5

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0

0.5

1.5

2.5

3.5

4.5

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Sessions

Frank D’Andy

Monty Teo

Amber Mats

Sessions

Figure 1 Rate of Disruptive Behavior per Minute

Haydon et al. / Types of Opportunities to Respond 35

questions with 4-point Likert-type scale responses ranging from 1 (not at all) to 4 (very). In response to which inter- vention was the most difficult to implement, four of six teachers thought the mixed responding was the most dif- ficult to implement (M = 2.33; range = 1–3), one teacher believed choral responding was the most difficult, and one teacher replied that individual responding was most difficult to implement. Low mean scores (M = 1.0) on teachers’ perceived difficulty with the study’s procedures suggested that the teachers implemented individual and choral responding with ease. High mean scores (M = 4.0) suggested that teachers found the training sessions to be very helpful. Although teachers already implemented individual responding, the midrange scores for choral (M = 2.83; range = 1–4) and mixed (M = 2.5; range = 1–4) responding suggested that teachers might be likely to implement choral responding in the future.

Discussion

This study identifies several key findings. First, in terms of disruptive behavior, mixed responding appears to be a more effective instructional strategy than either choral or individual responding. Five of six students had lower mean rates of disruptive behavior during mixed

responding than during choral or individual responding. This finding supports Stevens and Rosenshine’s (1981) strong recommendation for the use of mixed responding (70% choral, 30% individual). Second, results indicate that choral responding is a more effective instructional strategy than individual responding in terms of decreas- ing disruptive and off-task behavior. Five out of six participants had lower mean rates of disruptive behavior and lower mean percentages of intervals of off-task behavior during choral responding than during individ- ual responding. This finding is consistent with earlier research (McKenzie & Henry, 1979; Sainato et al., 1987; Sindelar et al., 1986; Sutherland et al., 2003).

Differences between choral and mixed responding are less consistent for off-task behavior. Four students had fewer intervals of off-task behavior during mixed responding, and one student had fewer intervals of off- task behavior during choral responding. However, the group mean for intervals of off-task behavior during mixed responding was 18.8%, whereas the group means for off-task behavior during choral and individual responding were 26.9% and 42.0%, respectively. Given the criterion of 90% for student on-task behavior by the Council for Exceptional Children (1987), only the mixed responding condition (81.2%) somewhat approached

Table 3 Means, Standard Deviations, and Ranges for Off-Task Behavior in Each Condition

Individual Choral Mixed

Student M (SD) Range M (SD) Range M (SD) Range

Frank 56.25 (6.85) 45.83–62.50 32.73 (5.60) 25.00–41.66 16.55 (7.91) 8.32–25.00 D’Andy 25.60 (6.56) 16.66–33.33 19.05 (5.82) 12.50–25.00 9.89 (4.42) 4.17–16.66 Monty 40.27 (10.21) 29.16–62.50 26.56 (7.69) 12.50–37.50 16.67 (9.32) 8.33–37.50 Teo 28.47 (20.65) 4.16–62.50 31.24 (20.72) 8.30–62.50 22.02 (9.54) 8.33–33.33 Amber 47.50 (15.21) 25.00–66.70 23.33 (6.97) 16.66–33.33 24.30 (7.17) 16.66–33.33 Mats 54.17 (9.86) 45.83–66.67 28.47 (9.29) 20.83–45.83 23.33 (5.59) 20.83–33.33

Table 4 Means, Standard Deviations, and Ranges for Active

Student Responding in Each Condition

Individual Choral Mixed

Student M (SD) Range M (SD) Range M (SD) Range

Frank 22.31 (13.31) 12.5–46.15 69.34 (16.78) 44.73–89.18 84.35 (5.57) 76.31–94.73 D’Andy 89.32 (7.23) 74.28–94.73 93.25 (6.26) 81.57–100.00 97.28 (4.16) 88.88–100.00 Monty 60.19 (22.68) 33.33–91.89 84.2 (6.34) 71.42–91.66 90.50 (6.47) 77.77–100.00 Teo 82.20 (15.72) 52.63–94.44 93.79 (6.30) 82.60–100.00 84.28 (7.60) 72.72–94.17 Amber 58.10 (22.54) 34.14–92.10 96.38 (2.28) 94.87–100.00 87.65 (7.74) 80.55–97.14 Mats 42.20 (27.88) 13.04–67.50 75.27 (19.06) 40.00–94.44 62.80 (4.27) 56.41–67.50

36 Journal of Emotional and Behavioral Disorders

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 0

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Sessions

Frank D’Andy

Monty Teo

Amber Mats

Sessions

Figure 2 Percentage of Intervals Off-Task

Haydon et al. / Types of Opportunities to Respond 37

CEC standards. However, in comparison with individual responding, there appears to be a measurable benefit (Horner et al., 2005).

For active student responding, three of six students had their highest mean percentages during mixed responding (M = 90.7%), whereas three students had their highest mean percentages during choral responding (M = 82.3%). In light of recommendations by the CEC (1987), these percentages approach or exceed the 85% criterion for student correct responses during review. However, the group mean for active student responding

during individual responding was lowest among the three conditions (M = 59.1%), and this percentage was well below the criterion set by the CEC. In addition, mean percentages were highest for off-task behavior for all six participants during individual responding, again being consistent with previous research findings (McKenzie & Henry, 1979; Sainato et al., 1987; Sindelar et al., 1986; Sutherland et al., 2003).

The lack of differential effects across the three types of opportunities to respond on disruptive behavior and off-task behavior for one student, Teo, deservers further

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 0

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 0

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Frank D’Andy

Monty

Mats

Teo

Amber

Figure 3 Percentage of Active Student Responding

38 Journal of Emotional and Behavioral Disorders

attention. The mean rate of disruptive behavior was approximately equal across the three types of opportuni- ties to respond. Teo’s data for disruptive and off-task behavior among the three conditions show substantial variability and overlap. The high rates of Teo’s disrup- tive behavior and off-task behavior may indicate that the instructional intervention of mixed or choral responding was not powerful enough to decrease his disruptive behavior and off-task behavior. For example, incidental observations indicate that during the teacher feedback procedure, Teo talked with a peer sitting next to him and the peer responded. It is possible that teacher prompts could have decreased the rate of Teo’s disruptive behav- ior and frequency of off-task behavior. However, his teacher informally reported that she did not feel comfort- able implementing and following up on negative conse- quences because she was not his homeroom teacher. Teo’s disruptive and off-task behavior may also have been altered by the presence of setting factors (Davis & Fox, 1999). For example, Teo suffered from migraine headaches and this was not discovered until halfway through the study. Furthermore, the teacher indicated that she was aware of serious problem behavior during transi- tion time before language arts.

Social validity data reveal that the six teachers felt that the study did not disrupt their classroom environ- ment and that the training session was very helpful. All six teachers stated that they currently used individual responding and indicated that choral responding was easy to implement, supporting earlier research wherein teachers provided similar feedback (Sainato et al., 1987). Four of six teachers commented that mixed responding was the most difficult type of opportunities to respond to implement because they had to read a randomized list. Instead, these teachers endorsed approximating the 70% choral to 30% individual ratio from memory, indicating the acceptability of mixed responding as a teaching strat- egy (Schwartz & Baer, 1991). However, Teachers 1 and 6 reported that they would be very likely to use mixed responding in the future. Teacher 6 commented that the mixed responding had an “element of surprise” because students did not know if they were called on individually until the “very last second.” After a visual inspection of the data, Teacher 1 stated that she would be very likely to use mixed responding in the future. Implementing increased rates of opportunities to respond that fit within the details of day-to-day classroom instruction and that do not radically alter teachers’ curriculum are a few ways researchers can get teachers to maintain evidence-based practices in their classrooms (Gersten, Vaughn, Deshler, & Schiller, 1997).

It is interesting that most teachers’ perceptions of the effects of the three types of opportunities to respond on the dependent variables were not confirmed by the data. For example, among the five teachers where mixed responding produced the lowest rate of disruptive behav- ior, only Teacher 3 had noticed decreases in disruptive behavior after implementing the mixed responding pro- cedure; the other four teachers believed choral respond- ing produced the largest effect. The fact that the teachers did not reliably discern the differential effects of the three different teaching strategies makes a strong case for using data collection and using objective criteria to make decisions about student classroom behavior (Witt, VanDerHyeden, & Gilbertson, 2004).

Although mixed responding appeared to be more effective in reducing disruptive behavior than choral and individual responding for five out of six students, a few limitations may temper the power of the statements that can be made as a result of this study. First, as is inherent in all single subject research designs, the small sample size limits the generalizability of the findings. Thus, gen- eralization to other academic activities and other settings, or to students by age, grade, gender, or learning histories, requires systematic replication (Kazdin, 1982). However, obtaining similar responses across individuals and two different types of schools suggests that the effect of mixed responding might be generalizable (Trolinder, Choi, & Proctor, 2004).

Second, there are several overlapping data points among the participants’ dependent variables with active student responding during choral and mixed responding. Thus, it is difficult to determine which instructional strategy is most effective in increasing active student responding. Third, although two observers were used to assess treatment integrity data and IOA was 100%, only 15% of the treatment sessions were observed.

Fourth, teacher implementation of contingent conse- quences outside of the learning trial was not recorded. Therefore, the extent of teacher use of individual attention, punishment, or extinction on the outcomes of the depen- dent variables is not known. For example, teacher attention may have affected the percentage of intervals of off-task behavior. Skinner and colleagues (1994) noted a similar limitation and reported in their study that individual atten- tion might have been functionally related to high rates of attention to tasks.

As a logical next step, further research could compare choral responding with mixed responding: with students of different ages and across various subject areas such as math and science (Carnine, 1976), across sessions of more than 8 minutes (Sainato et al., 1987), and with children

Haydon et al. / Types of Opportunities to Respond 39

identified with various learning disabilities or with autism (Koegel, Dunlap, & Dyer, 1980). These extensions would help establish and verify the conditions under which vary- ing types of responding are more effective and efficient.

In addition, further research would do well to include summative assessments at the end of the study to measure the effect of the three types of opportunities to respond on individual student learning. For example, researchers could examine the influence of the three types of oppor- tunities to respond on sight word acquisition and then measure increases in reading comprehension or sight word vocabulary (Skinner & Shapiro, 1989). Because the effects of the three types of opportunities to respond on one student were inconclusive, researchers could use functional assessments to gather information on the ante- cedent and consequent events that are associated with the occurrence of challenging behaviors in combination with instructional strategies (Scott & Kamps, 2007). In addi- tion, social validity could be obtained from the students’ perspective as part of future directions. Finally, research- ers should continue to investigate an optimal rate of opportunities to respond on the percentage of correct responses and error rates (West & Sloane, 1986).

Implications for Practice

Before implementing the mixed and choral responding procedures, teachers could consider that for a few stu- dents who lack impulse control, the implementation of precorrection strategies (i.e., reminding students to remain quiet after each response and to use inside voices) may be needed. The long-term benefits of using a systematic questioning strategy may outweigh the initial time involved to acquire a new instructional technique. These benefits include the following: students can respond up to three or four times more (depending on group size) dur- ing choral responding than during individual responding (Sindelar et al., 1986), and teachers could use mixed and choral responding to reduce disruptive and off-task behavior and reduce the amount of time students pas- sively attend during instruction (Sterling, Barbetta, Heward, & Heron, 1997).

References

Anderson, L. M., Evertson, C. M., & Brophy, J. E. (1979). An exper- imental study of effective teaching in first grade reading groups. The Elementary School Journal, 79, 193–223.

Armendariz, F., & Umbreit, J. (1999). Using active responding to reduce disruptive behavior in a general education classroom. Journal of Positive Behavior Interventions, 1, 152–158.

Barbetta, P. M., & Heward, W. L. (1993). Effects of active student response during error correction on the acquisition and mainte- nance of geography facts by elementary students with learning disabilities. Journal of Behavioral Education, 3, 217–233.

Barlow, D. H., & Hayes, S. C. (1979). Alternating treatments design: One strategy for comparing the effects of two treatments in a single subject. Journal of Applied Behavior Analysis, 12, 199–210.

Borg, W. R., & Gall, M. D. (1989). Educational research: An intro- duction. White Plains, NY: Longman.

Carnine, D. W. (1976). Effects of two teacher-presentation rates on off-task behavior, answering correctly, and participation. Journal of Applied Behavior Analysis, 9, 199–206.

Cooper, J. O., Heron, T. E., & Heward, W. L. (1987). Applied behavior analysis. Columbus, OH: Merrill.

Council for Exceptional Children. (1987). Academy for effective instruction: Working with mildly handicapped students. Reston, VA: Author.

Davis, C. A., & Fox, J. (1999). Evaluating environmental arrange- ment as setting events: Review and implications for measurement. Journal of Behavioral Education, 9, 77–96.

Gersten, R., Vaughn, S., Deshler, D., & Schiller, E. (1997). What we know about using research findings: Implications for improving special education practice. Journal of Learning Disabilities, 30, 466–476.

Godfrey, S. A., Grisham-Brown, J., Schuster, J. W., & Hemmeter, M. L. (2003). The effects of three techniques on student participation with preschool children with attending problems. Education and Treatment of Children, 26, 255–272.

Good, T. L. (1970). Which pupils do teachers call on? The Elementary School Journal, 70, 190–198.

Greenwood, C. R. (2001). Science and students with learning and behavioral problems. Behavioral Disorders, 27, 37–52.

Greenwood, C. R., Delquadri, J., & Hall, R. V. (1984). Opportunity to respond and student academic achievement. In W. L. Heward, T. E. Heron, D. S. Hill, & J. Trap-Porter (Eds.), Focus on behavior analysis in education (pp. 58–88). Columbus, OH: Merrill.

Gunter, P. L., Reffel, J. M., Barnett, C. A., Lee, J. M., & Patrick, J. (2004). Academic response rates in elementary-school class- rooms. Education and Treatment of Children, 27, 105–113.

Gunter, P. L., Shores, R. E., Jack, S. L., Denny, R. K., & DePaepe, P. A. (1994). A case study of the effects of altering instructional interac- tions on the disruptive behavior of a child identified with severe behavior disorders. Education and Treatment of Children, 17, 435–444.

Gunter, P. L., Venn, M. L., Patrick, J., Miller, K. A., & Kelly. L. (2003). Efficacy of using momentary time samples to determine on-task behavior of students with emotional/behavioral disorders. Education and Treatment of Children, 26, 400-412.

Hastings, R. P., & Oakford, S. (2003). Student teachers’ attitudes towards the inclusion of children with special needs. Educational Psychology, 23, 87–94.

Haydon, T., Mancil, G. R., & VanLoan, C. (in press). The effects of opportunities to respond on the on-task behavior for a student emitting disruptive behaviors in a general education classroom: A case study. Education and Treatment of Children.

Heward, W. L. (1994). Three “low tech” strategies for increasing the frequency of active student response during group instruction. In R. Gardner, III, D. M. Sainato, J. O. Cooper, & T. E. Heron (Eds.), Behavior analysis in education: Focus on measurably superior instruction (pp. 283–320). Monterey, CA: Brooks/Cole.

Heward, W. L., Courson, F. H., & Narayan, J. S. (1989). Using choral responding to increase active student response. Teaching Exceptional Children, 21, 72–75.

Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165-180.

40 Journal of Emotional and Behavioral Disorders

Kazdin, A. E. (1982). Single case research designs. New York: Oxford University Press.

Kennedy, C. H. (2005). Single-case designs for educational research. Boston: Allyn & Bacon.

Koegel, R. L., Dunlap, G., & Dyer, K. (1980). Intertrial interval duration and learning in autistic children. Journal of Applied Behavior Analysis, 13, 91–99.

McKenzie, G. R., & Henry, M. (1979). Effects of testlike events on on-task behavior, test anxiety, and achievement in a classroom rule-learning task. Journal of Educational Psychology, 71, 370–374.

Miller, A. D., Hall, M. A., & Heward, W. L. (1995). Effects of sequential 1-minute time trials with and without inter-trial feedback and self-correction on general and special education students’ fluency with math facts. Journal of Behavioral Education, 5, 319–345.

Nelson, J. R., & Roberts, M. L. (2002). Ongoing reciprocal teacher- student interactions involving disruptive behaviors in general education classrooms. Journal of Emotional & Behavioral Disorders, 8, 27–39.

Noell, G. H., & Witt, J. C. (1998). Toward a behavior analytic approach to consultation. In T. S. Watson & F. M. Gresham (Eds.), Handbook of child behavior therapy (pp. 41–57). New York: Plenum.

Sainato, D. M., Strain, P. S., & Lyon, S. R. (1987). Increasing academic responding of handicapped preschool children during group instruc- tion. Journal of the Division for Early Childhood, 12, 23–30.

Schwartz, I. S., & Baer, D. M. (1991). Social-validity assessments: Is current practice state of the art? Journal of Applied Behavior Analysis, 24, 189–204.

Scott, T. M., & Kamps, D. M. (2007). The future of functional behavior assessment in school settings. Behavioral Disorders, 32, 146–157.

Sindelar, P. T., Bursuck, W. D., & Halle, J. W. (1986). The effects of two variations of teacher questioning on student performance. Education and Treatment of Children, 9, 56–66.

Skinner, C. H., Belfiore, P. J., Mace, H. W., William-Wilson, S., & Johns, G. A. (1997). Altering response topography to increase response efficiency and learning rates. School Psychology Quarterly, 12, 54–64.

Skinner, C. H., Fletcher, P. A., & Henington, C. (1996). Increasing learning rates by increasing student responses rates: A sum- mary of research. School Psychology Quarterly, 11, 313-325.

Skinner, C. H., Johnson, C. W., Larkin, J., Lessley, D. J., & Glowacki, M. L. (1995). The influence of rate of presentation during taped-words interventions on reading performance. Journal of Emotional & Behavioral Disorders, 3, 214–224.

Skinner, C. H., & Shapiro, E. S. (1989). A comparison of taped-words and drill interventions on reading fluency in adolescents with behav- ior disorders. Education and Treatment of Children, 12, 123–133.

Skinner, C. H., Smith, E. S., & McLean, J. E. (1994). The effects of intertrial interval duration on sight-word learning rates in children with behavioral disorders. Behavioral Disorders, 19, 98–107.

Sterling, R. M., Barbetta, P. M., Heward, W. L., & Heron, T. E. (1997). A comparison of active student response and on-task instruction on the acquisition and maintenance of health facts by fourth grade special education students. Journal of Behavioral Education, 7, 151–165.

Stevens, R., & Rosenshine, B. (1981). Advances in research on teaching. Exceptional Education Quarterly, 2, 1–9.

Sutherland, K. S., Alder, N., & Gunter, P. L. (2003). The effect of increased rates of opportunities to respond on the classroom behavior of students with emotional/behavioral disorders. Journal of Emotional and Behavioral Disorders, 11, 239–248.

Trolinder, D. M., Choi, H., & Proctor, T. B. (2004). Use of delayed praise as a directive and its effectiveness on on-task behavior. Journal of Applied School Psychology, 20, 61–83.

Walker, H. M., & Severson, H. H. (1993). Systematic Screening for Behavior Disorders. Longmont, CO: Sopris West.

West, R. P., & Sloane, H. N. (1986). Teacher presentation rate and point delivery rate: Effects on classroom disruption, perfor- mance accuracy, and response rate. Behavior Modification, 10, 267–286.

Witt, J. C., VanDerHyeden, A. M., & Gilbertson, D. (2004). Troubleshooting behavioral interventions: A systematic process for finding and eliminating problems. School Psychology Review, 33, 363–383.

Wolery, M., Ault, M. J., Doyle, P. M., Gast, D. L., & Griffin, A. M. (1992). Choral and individual responding: Identification of interactional effects. Education and Treatment of Children, 15, 289–309.

Yarbrough, J. L., Skinner, C. H., Lee, Y. J., & Lemmons, C. (2004). Decreasing transition times in a second grade classroom: Scientific support for the timely transitions game. Journal of Applied School Psychology, 20, 85–107.

Todd Haydon, PhD, is an assistant professor of special educa- tion at the University of Cincinnati. His research interests include effective teaching practices, functional behavior assessments, and positive behavior and supports.

Maureen A. Conroy, PhD, is a professor of special education at Virginia Commonwealth University. Her research interests include students with learning and behavior disorders.

Terrance M. Scott, PhD, is a professor and director of Special Education Programs at the University of Louisville and con- ducts research on interventions related to student academic and behavioral success.

Paul T. Sindelar, PhD, is a professor of special education at the University of Florida. His research interests include teacher preparation, induction, and mentoring.

Brian R. Barber, MEd, is a doctoral student at the University of Florida. His research interests include the neuropsychologi- cal bases of emotional and behavioral disorders and the inter- face of these characteristics with social-emotional learning.

Ann-Marie Orlando, MS, is a doctoral student at the University of Florida. Her current interests include communi- cation, assistive technology, and inclusion.