final

Learning Disabilities Research & Practice, 28(2), 81–88 C© 2013 The Division for Learning Disabilities of the Council for Exceptional Children

Training for Generalization and Maintenance in RtI Implementation: Front-Loading for Sustainability

Matthew K. Burns, Andrea M. Egan, Amy K. Kunkel, Jennifer McComas, Meredith M. Peterson, Naomi L. Rahn, and Jennifer Wilson

University of Minnesota

Response to Intervention (RtI) is being implemented as a new initiative in PK-12 schools with increasing frequency. However, the model must be sustained at the school level, which is potentially difficult due to a number of challenges brought about by systems change. This article applied the Stokes and Baer (1977) framework for programming for generalization and maintenance of behavior change to suggest specific activities in which schools could engage to better ensure RtI sustainability. We specifically discussed ways to (1) introduce to natural maintaining contingencies, (2) train with sufficient exemplars, (3) train loosely, (4) program common stimuli, (5) mediate generalization, and (6) train to generalize. Directions for future research are included.

Response to Intervention (RtI) and other multitiered inter- vention systems are being adopted nationwide with increas- ing frequency (Berkeley, Bender, Peaster & Saunders, 2009) to increase student achievement for all students, reduce re- ferrals to special education, and close existing achievement gaps (Fuchs, Fuchs & Stecker, 2010). RtI has the potential to positively affect both systemic and student outcomes (Burns, Appleton & Stehouwer, 2005), but, some question whether the RtI movement will sustain over time (Burns, 2007; Ys- seldyke, 2005). RtI initiatives must ultimately be sustained at the school level, and organizations adopting a system of RtI are faced with a multitude of challenges brought about by systems change (Grimes, Kurns & Tilly, 2006).

Previous research has found that implementation integrity could be a serious threat to the validity of RtI models (Gansle & Noell, 2007). For example, school personnel consistently assessed fidelity of implementation for interventions that oc- curred at tier 2, but did not assess fidelity at tier 1, and the alignment between tiers was not explicit (Hill, King, Lemons & Partanen, 2012). Moreover, implementation integrity of problem-solving teams (PSTs) was low to the point of po- tentially affecting student outcomes (Burns & Symington, 2002). Some of the challenges regarding implementation in- tegrity can be avoided by building on the existing knowledge of the school personnel, streamlining processes, and using a clear system of communication between interventionist and teacher (Johnson, Pool & Carter, 2012). However, implemen- tation integrity can still wane as the implementation moves further from the initial supports (Burns & Symington, 2002; Kovaleski, Gickling, Morrow & Swank, 1999), which further highlights the need to focus on sustainability.

Requests for reprints should be sent to Matthew K. Burns, University of Minnesota. Electronic inquiries should be sent to [email protected].

Sustainability is best obtained by changing the system in which the initiative is implemented (Hargreaves & Fink, 2000). Systems change is an “intentional process designed to alter the status quo by shifting and realigning the form and function of a targeted system” (Foster-Fishman, Nowell & Yang, 2007, p. 197), and is multifaceted with theoretical, ethical, and pragmatic implications (Noell & Gansle, 2009). Prior to implementation, theoretical and ethical dimensions of systems change are considered, including issues of what to change, why to change it, and how that change will take place. Promoting adoption and implementation of RtI in schools re- quires that stakeholders see the value of its implementation in their schools (McIntosh, Filter, Bennett, Ryan & Sugai, 2010) and that teacher “buy-in” is high. Moreover, educators are more likely to implement interventions or practices in which they have experience, support, and belief in overall effectiveness (Fixsen, Blase, Naoom & Wallace, 2009). Al- though RtI implementation research has found that collabo- ration is important for teacher acceptance, teacher buy-in can be difficult to gain (Mahdavi & Beebe-Franenberger, 2009).

Alberto and Troutman (2009) suggest the school and teacher environment should be examined to uncover what teachers value and invoke strategies or interventions that may have existing support. After schools have resolved the theoretical and ethical dimensions of systems change, the pragmatic aspects of implementation, including generaliza- tion and sustainability, can be addressed (Noell & Gansle, 2009). Behavior change among all organizational partici- pants (i.e., teachers and administrators) is crucial (Sarason, 1996), and must be accompanied by sustained environmen- tal supports that are responsive and adapted to inevitable challenges (Grimes et al., 2006). Moreover, in order for the long-term goal of sustained change to be realized, the ex- pected behaviors exhibited by key stakeholders must gen- eralize across situations and maintain over time (Sarason, 1990; 1996).

82 BURNS ET AL.: SUSTAINABILITY OF RTI IMPLEMENTATION

There is an inexorable link between generalization and sustainability within educational reform (Hargreaves & Fink, 2000). Generalization occurs when a learned behavior con- tinues to occur across time, setting, and target in the absence of the conditions that promoted its acquisition (Stokes & Baer, 1977). Thus, generalization is at least a prerequisite for sustainability, but it could be argued that promoting gen- eralization over time could provide a framework to address sustainability because sustainability is the continued behav- ior over time after the conditions in which it was required are removed or changed.

Stokes and Baer (1977) introduced a framework for as- sessing and programming for generalization and mainte- nance of behavior change. Prior to that publication, the most frequent method of considering generalization in behavior change programs was to “train and hope” (p. 351). In other words, new behaviors were trained and any generalization across settings, time, or responses were not actively planned; rather, it was hoped that generalization would occur. A re- cent survey of special education directors found that the most common support for RtI implementation provided by state departments of education was short-term trainings and pro- fessional development (Werts, Lambert & Carpenter, 2009). Accordingly, RtI implementation could attempt to be gen- eralized through train and hope, but it will likely not be successful without sustained environmental support. How- ever, deliberate programming for generalization and mainte- nance of expected behaviors in a system of RtI could result in successful outcomes and sustained actions. The process of programming for generalization of RtI implementation includes the following techniques discussed by Stokes and Baer in 1977: (1) introduce to natural maintaining contin- gencies, (2) train sufficient exemplars, (3) train loosely, (4) program common stimuli, (5) mediate generalization, and (6) train to generalize.

The purpose of this article is to discuss each of these strate- gies within the context of implementation and sustainability of RtI and implications for practice in schools. The goal will be to describe specific actions that schools can take to pro- mote generalization and maintenance of practices in order for RtI implementation to be sustained over time. In other words, we will discuss ways that school personnel can frontload im- plementation efforts to better assure sustainability. Table 1 provides a succinct summary of the generalization strategies and related practices for RtI sustainability. We will also pro- vide suggestions for future research, which will likely be the primary outcome associated with these suggested strategies and practices.

INTRODUCE TO NATURAL MAINTAINING CONTINGENCIES

Stokes and Baer (1977) stated that introducing naturally maintaining contingencies is the most dependable way to ob- tain generalization, even though this strategy may not always be feasible. To generalize in this manner is to transfer the behavioral control to the natural contingencies that operate in the environment where the practice will occur. Apply- ing naturally maintaining contingencies in training involves

teaching behaviors or practices and bringing them into con- tact with naturally existing contingencies for reinforcement.

Using naturally occurring contingencies to promote gen- eralization can take many varied forms. For example, imagine that during a professional development session, teachers are taught to examine student data and make instructional pro- gramming decisions based on the data. Now imagine that a month later when they examine their students’ progress, the data indicate that the most struggling students have made sub- stantial gains. Seeing those substantial gains may naturally reinforce the practice of making instructional programming decisions based on student data. In this example, the instruc- tional leader might select specific examples of the data-based instructional programming that resulted in the substantial academic gains for a sample of the students and could discuss the types of instructional programming decisions that would be more and less likely to produce future academic gains.

The first step in planning for a sustainable RtI model at the individual school level might be to have school person- nel implement the model within their daily practice (Fixsen et al., 2009), rather than having district or university person- nel handle the initial implementation. Despite the temptation to provide significant support during the initial implementa- tion of RtI, if school personnel are the ones who implement the RtI-related practices, then they are likely to directly ex- perience the natural successes that result. Similarly, teachers should be included in all aspects of planning and implement- ing RtI, including making intervention decisions, and doing so resulted in improved student outcomes (Lembke, Garman, Deno & Stecker, 2010). For example, teachers could create the list of instructional practices and interventions for spec- ified skill deficits available for use at each tier of service in their building. The list could take the form of a menu of evidence-based options and include evidence-based in- structional practices and interventions that teachers in the building have used and found successful. A teacher may be more likely to implement an instructional practice or inter- vention that s/he has found successful in the past because s/he has witnessed the effect it had on producing academic growth and therefore has contacted the natural consequence, student success, which was produced by implementing the practice or intervention.

Implementation of RtI should be considered within the context of what components are already in place and what components need to be established. If numerous components need to be added, a format for establishing the model that al- lows individual teachers to make a relatively small number of changes to their practice at a time is advisable (Grimes et al., 2006). This approach allows teachers to come into contact with naturally maintaining contingencies, whereas if they are forced to change numerous aspects of their practice at once, they are less likely to contact the reinforcing consequences of any one of the practices (McIntosh et al., 2010). When sys- tems change is time-consuming and requires implementation of numerous novel practices, competition with existing, less effective practices presents a considerable challenge (Noell & Gansle, 2009). Alternately, schools might consider invit- ing teachers to be involved in or responsible for particular components of RtI (e.g., screening, interventions) according to their interest (Johnson et al., 2012). Preference is related

LEARNING DISABILITIES RESEARCH 83

TABLE 1 Strategies for Generalization from Stokes and Baer (1977) and Accompanying Activities to Build Sustainability in Response to Intervention (RtI)

Implementation

Strategy Description Activities

Natural maintaining Teach the skill to be reinforced by naturally • Involve school personnel in implementation decisions contingencies existing contingencies • School personnel implement interventions and assessments

• Use efficient data collection procedures Train sufficient exemplars Use numerous examples during training • Provide ongoing professional development in the core components/skill

sets of RtI • Use a broad range of examples of forms that RtI core components can

take (e.g., collecting progress monitoring data for a variety of academic skills)

• Train personnel to implement multiple aspects of the grade-level and problem-solving team processes

Train loosely Expose learners to a diverse array of the contexts or situations in which skill set is

• Train using a variety of contexts and situations in which the same set of skills are required (e.g., monitor progress in multiple areas)

to be used • Use a broad range of examples (e.g., what teams are called, which data collection tools are selected)

Program common stimuli Incorporate into training stimuli that are common across contexts or situations

• Use grade-level teams as professional learning communities to make decisions at various tiers

• Configure teams (e.g., grade-level teams) of consistent members who will address a variety of contexts and situations together

Mediate generalization Incorporate tools or strategies that the • Use implementation fidelity protocols and checklists learner can readily use across contexts or situations

• Provide continuous feedback to school personnel (e.g., team processes, intervention fidelity, assessment procedures)

Train to generalize Raise awareness of need for generalization during training and suggest use of trained skill sets across contexts and situations

• Discuss how existing RtI practices contextualize into other areas of practice

to quality of reinforcement and if teachers are encouraged to participate in aspects of RtI that fit their preferences, they may experience relatively higher-quality reinforcement for their participation.

Finally, response effort impacts the effects of naturally maintaining contingencies of reinforcement. For example, the amount of data collected in an RtI model should not be exorbitant, but rather focused on useful information that can be collected efficiently (Horner, Sugai & Todd, 2001). Assessment procedures should be quick and easy, and yet result in sufficiently reliable data and valid decisions (e.g., curriculum-based measurement). When teachers are involved in collecting their own data, it allows them to see the effects of their practice through a direct link to student outcomes (McIntosh et al., 2010). Moreover, previous RtI im- plementation efforts emphasized the importance of stream- lining data collection and giving the teachers responsibility for collecting the data (Johnson et al., 2012). However, teach- ers must view data collection and analysis as an investment (Horner et al., 2001), and the payoff of positive outcome data presents natural reinforcement for teachers. If data reveal an absence of positive outcomes for certain students, teachers are provided an efficient and effective means by which to in- form further instruction, and will see the benefits of program modification for students as interventions are intensified and data collection continues.

TRAIN SUFFICIENT EXEMPLARS

Training sufficient exemplars is described as one of the most valuable techniques for programming generalization (Stokes

& Baer, 1977). Teaching only a single exemplar limits the effectiveness of the lesson to the teaching situation, whereas providing additional exemplars across a variety of situations is crucial for generalization of the skill set to occur across a variety of situations. To illustrate, in a special education program, an instructor might teach how to use a vending machine. Such teaching necessitates some careful planning because there are a wide variety of vending machines, many of which require different approaches. Some vending ma- chines require pushing the button that depicts the product, others require finding the code for the product and enter- ing into a keypad. Depositing money can take the form of coins, bills, or a combination, and the coin slot is sometimes vertical and sometimes horizontal. By exposing students to these variations during training, they are more likely to ex- perience success when they use a vending machine when they are not with their teacher. Within the context of pro- fessional development, which is a crucial aspect of effective RtI implementation (Kratochwill et al., 2007), it is wise to provide educators with several examples of potential imple- mentation models including structures for delivering quality core instruction for all students, a variety of screening and progress monitoring tools, evidence-based interventions for tiered intervention delivery, and teaming strategies for data- based decision making. How these individual components are implemented within a school can vary depending on the school’s model. Providing educators with a variety of ex- amples of these core components, as well as examples of successful RtI models in other schools or districts, can allow educators to adapt and adopt an ideal model for the situations their setting presents, leading to a much greater probability of sustaining RtI within a given school.

84 BURNS ET AL.: SUSTAINABILITY OF RTI IMPLEMENTATION

Although schools are limited by the standardized assess- ments they are required to use with students with and without disabilities, the screening and progress monitoring tools are typically open to teacher discretion. Providing teachers with training and materials to monitor progress across content ar- eas (i.e., reading, writing, and math) and with various tools (e.g., oral reading fluency, timed mathematics probes) im- parts additional examples of efficient assessments of student outcomes, thus creating multiple ways from which teachers can choose to monitor progress. Furthermore, varied exam- ples of monitoring frequency (e.g., weekly or bimonthly) and data collection personnel (e.g., paraprofessionals, volunteers, or classroom teachers) provides teachers with additional op- tions. For example, screening or monitoring data might be completed by a small cadre of individuals across grades and days to limit disruptions to the classroom and instruction. Conversely, such data might also be collected in a unified approach involving many individuals completing all class- rooms within a shorter time span.

Ensuring that teachers are knowledgeable of multiple evidence-based interventions will allow them to make in- formed intervention decisions for struggling students to re- ceive targeted interventions in appropriate groups. However, providing training in too many intervention approaches can be counterproductive to maintaining naturally occurring con- tingencies and may overwhelm the teachers. Thus, it might be beneficial to train teachers and school personnel in in- tervention implementation within content areas, age groups, and academic needs, which may help promote a foundation for a successful model and its generalization.

Training educators on various teaming strategies may be one of the most important considerations for RtI sustain- ability, particularly with frequent changes in staffing models, movement of administrators within a district, and teacher turnover. Many schools implementing RtI form grade-level teams, which are strongly related to the quality of later prepa- rations for sustainability (Perkins et al., 2011). Although the function of grade-level teams can differ among schools, de- pending on the RtI model in place, they are often involved in examining screening data for all students, analyzing progress monitoring data, making informed intervention decisions re- garding struggling students, and discussing adaptations and modifications to the model at each tier of instruction. Pro- viding schools with examples of successful grade-level team models and professional development of effective teaming strategies will allow them to choose and adapt the best model fitting their resources, increasing the sustainability of RtI over time. Unfortunately, inconsistent implementation of school- based teams is well documented and a potential threat to RtI implementation (Burns, Vanderwood & Ruby, 2005), which reinforces the need for schools to train personnel with posi- tive examples before implementation begins.

TRAIN LOOSELY

Whereas training sufficient exemplars involves teaching in such a way that individuals make appropriate adaptations and adjustments in their behavior (e.g., how to indicate a selection with any vending machine) given the specific requirements

of the context or situation, training loosely (Stokes & Baer, 1977) refers to teaching a behavior or skill set such that it oc- curs in the presence of a variety of contexts and situations. To train loosely, an approach must be taken that exposes learners to a diverse array of situations in which the same response might be expected. For example, in a classroom, a teacher might say, “Have a seat,” “Take a seat,” “Find your place,” or gesture toward a circle of chairs; in all cases, the expected behavior is for the student to sit down. In the previous sec- tion, we mentioned grade-level teams, which often function to provide a forum and structure, as well as accountability for analyzing progress monitoring data, making informed in- tervention decisions, and discussing necessary adjustments to the model at each tier of instruction. However, PSTs can also play an important role in RtI models, especially within tier 3. PSTs go by a wide variety of names across the coun- try, including but not limited to Instructional Support Team, Instructional Leadership Team, Academic Leadership Team, Child Study Team, and Teacher Support Team. By inter- changeably using a variety of names for teams but pointing out their unifying function, the notion of problem-solving instruction and interventions within a team is trained loosely. The purpose of training loosely is to allow for responding in a singularly appropriate manner in a variety of situations that differ superficially but are functionally equivalent. Thus, transfer of the targeted behavior to new situations is facil- itated by exposure to the many contextual dimensions that may vary.

The concept of training loosely can inform multiple as- pects important to the sustainability of RtI, including data collection, intervention delivery, and teaming strategies for effective decision making. School personnel must pay at- tention to the fit between the conceptual framework of a school-wide program and the local, contextual variables of a given school (McIntosh et al., 2010). While adherence to the conceptual framework of RtI is necessary to increase the efficacy of the practice, acknowledgement of contextual fit is important to its sustainability within a given school en- vironment (Goldenberg, 2003). For example, a school may strongly embrace a strengths-based approach to instructional planning. In this case, the term “PST” would likely be less acceptable than the term “Instructional Leadership Team.” Increased flexibility of RtI implementation combined with an emphasis on local control may create the potential for RtI to sustain in a manner that is both building-based and consistent with the general concept.

The function and makeup of PSTs might also allow for flexibility regarding how often the team meets, who is re- sponsible for leading the meetings, and the relationship be- tween team discussions and professional development. For example, some schools may use a designated leader who organizes and leads meetings, whereas flexibility and sus- tainability may be enhanced by having several individuals within a school able to lead meetings at different times. Sim- ilarly, the data analysis completed by PSTs may serve as a springboard for related professional development, or could support already-implemented school-wide initiatives. This relationship is often reciprocal and can buttress the sustain- ability of similar data-driven practices such as School-wide Positive Behavior Interventions and Supports.

LEARNING DISABILITIES RESEARCH 85

PROGRAM COMMON STIMULI

Programming common stimuli, another technique used to train for generalization, involves incorporating in training stimuli that are essential features and therefore will likely be present in a variety of generalized situations (Stokes & Baer, 1977). One example in school settings with students has been the use of peers as common stimuli to promote generalization of desired social interactions across settings (Stokes & Baer, 1976). Incorporating peers as common stimuli to train for generalization can be applied to professional development related to RtI.

The literature on professional development, particularly the use of professional learning communities (PLCs), pro- vides an opportunity to utilize peers as common stimuli to build sustainability of RtI. It is increasingly clear that high- quality professional development in schools represents an essential link between teacher performance and student out- comes (Kratochwill et al., 2007). In a PLC, teachers work together in small groups on a particular topic to analyze and improve school practices to enhance student learning. PLCs are composed of three “Big Ideas” (DuFour, 2004): (1) ensur- ing that students learn, (2) building a culture of collaboration, and (3) a focus on results. The mission of professional devel- opment in a PLC framework is not simply that students are taught, but rather that they learn. When learning does not oc- cur for all students, a PLC will focus on improving teaching practices to enhance student learning. To build a culture of collaboration, PLCs provide an ideal occasion to use peers as common stimuli to build sustainability. With a focus on re- sults, improving student achievement through collaboration between teachers becomes routine work for everyone in the school. PLCs allow schools to “create a multi-tiered, coordi- nated, and collective response to support students” (DuFour, 2011, p. 61).

Within RtI, teachers in grade-level teams comprise the PLCs. Teachers on a grade-level teamwork together to adopt specific aspects of RtI. As new aspects of RtI are adopted, the likelihood that a teacher will successfully implement new RtI components is increased if it is done in the context and with the support of the other teachers on the grade-level team. The presence of peer teachers can facilitate generalization by sim- ulating the environment—the grade-level team—in which successful adoption of the initial RtI components occurred. Research regarding professional development related to RtI found that isolated training was not sufficient (Kratochwill et al., 2007). This seems particularly relevant to the sus- tainability of RtI, and the use of peers to program common stimuli in professional development practices in schools can ensure generalization and maintenance of the model.

MEDIATE GENERALIZATION AND TRAIN TO GENERALIZE

Two final ways to increase sustainability and generalization of RtI are to: (1) build procedures into the RtI process that will increase the likelihood of generalization of desired be- haviors, and (2) directly discuss and ask for generalization. In mediated generalization, a response that is likely to be

used in new situations is established to promote generaliza- tion (Stokes & Baer, 1977). For example, to multiply poly- nomials, we are taught “FOIL”—first, outside, inside, last, which is the order in which the products are to be computed (Crawford, 1980). Thus, whenever one is confronted with a polynomial, use of FOIL will facilitate successful multiplica- tion of the polynomial in any situation. Within an RtI context, tools for individual teacher and program self-evaluation may play a mediating role in generalization of RtI components. These tools include checklists for fidelity of implementation of specific evidence-based practices (e.g., a reading interven- tion), and for implementation of various aspects of the RtI process more generally. Fidelity of implementation at both the teacher and school levels should be evaluated to ensure the effectiveness of RtI (Riley-Tillman & Burns, 2009).

Teacher-level implementation across RtI components and settings can be measured through observations of imple- mentation fidelity of specific evidence-based practices in the classroom using checklists developed by researchers or school districts. For example, the St. Croix River Education District in Minnesota and Heartland Area Education Agency in Iowa have developed checklists for assessing fidelity of implementation of specific instructional or intervention pro- grams (see Table 2). School-level evaluation tools are also necessary for measuring generalization and maintenance of RtI over time. For example, the School-wide Evaluation Tool (SET; Sugai, Lewis-Palmer, Todd & Horner, 2001), was de- signed to evaluate implementation fidelity of School-wide Positive Behavior Support. Data from the SET are reviewed by school teams and state-level teams to guide sustainabil- ity efforts at both levels (McIntosh, Filter, Bennett, Ryan & Sugai, 2010).

In addition to considering mediators, Stokes and Baer (1977) advise directly discussing generalization and suggest- ing that individuals generalize the desired behaviors or skills sets to other contexts or situations. Training to generalize involves explicitly suggesting or reminding the implementer (e.g., teacher) to implement the RtI components in novel sit- uations. Within an RtI framework, professional development efforts should include discussions with staff of how exist- ing RtI skill sets, such as universal screening and data-based decision making, could be generalized to other areas of prac- tice. For example, in a school already implementing RtI in reading, school leaders might initiate discussions of how the RtI model could be expanded to include math or behavior. As generalization occurs, staff efforts should be reinforced. Reinforcing generalization when it happens results in quick wins for teachers and other RtI team members.

DIRECTIONS FOR FUTURE RESEARCH

Although the recommendations made by Stokes and Baer (1977) are well grounded in research, the application to sus- taining RtI requires additional research. Schools are com- plex systems with several considerations when implement- ing change initiatives (Fixsen et al., 2009). Thus, researchers could examine a method to best identify potential applica- tion (e.g., quality core instruction, screening and progress monitoring tools, evidence-based interventions for tiered

86 BURNS ET AL.: SUSTAINABILITY OF RTI IMPLEMENTATION

TABLE 2 Response to Intervention (RtI) Implementation Checklists

Resource Tool(s)

Evidence-Based Intervention Network http://ebi.missouri.edu/ Intervention protocols for reading, math, writing, and behavior

Heartland Area Education Agency http://www.aea11.k12.ia.us/idm/ Observation and permanent product checkists.html treatment integrity checklists for academic interventions

National Center on Response to Intervention http://www. RtI Integrity Rubric and Worksheet rti4success.org/categorycontents/continuously_improving/page

Pennsylvania Department of Education http://www.pattan.net/category/ • Response to Instruction & Intervention Educational%20Initiatives/Response%20to%20Instruction%20and% (RtII) Readiness and Implementation 20Intervention%20%28RtII%29 (Elementary): Self-Assessment Tool

• Secondary RtII Framework: Self-Assessment Tool Path to Reading Excellence in School Sites http://www. • Reading intervention protocols for all three tiers

cehd.umn.edu/reading/PRESS/default.html • Intervention implementation checklists • Professional development materials

RtI Action Network http://www.rtinetwork.org/ Self-Assessment of Problem Solving Implementation getstarted/checklists-and-forms

Scientifically based research http://gosbr.net/ • Reading and math intervention protocols • Assessment tools

St. Croix River Education District http://www.scred. • Integrity checklists for reading interventions k12.mn.us/School/Index.cfm/go:site.Page/Page:3/index.html

Technical Assistance Center on Positive Behavioral Interventions and PBIS evaluation checklists including: Supports (PBIS) http://www.pbis.org/evaluation/evaluation_ • School-Wide Evaluation Tool (SET)

tools.aspx • Early Childhood System-wide Evaluation Tool: Program Wide (EC SET-PW)

• Benchmarks for Advanced Tiers (BAT)

intervention delivery, and data-based decision-making mod- els) that matches the needs and circumstances of each unique system. Second, research has found that school-based problem-solving teams were effective, but implementation integrity of the process may have substantially reduced team effectiveness (Burns & Symington, 2002), and implementa- tion integrity was rarely assessed in tier 1 (Hill et al., 2012). Moreover, unanticipated staffing changes can occur within schools, which may result in a change in problem-solving and leadership teams. When team members leave, responsibilities have the potential to shift or be forgotten. Ensuring neces- sary components of the RtI process are in place throughout change is crucial to sustainability. Thus, additional research is needed to examine issues such as the essential attributes of an effective team and how to best measure integrity of core instruction.

Implementing multiple changes, such as training suffi- cient exemplars, training loosely, and programming common stimuli, comes with additional difficulties that could provide targets for additional research. Moreover, future researchers could examine the recommendations made here to determine both effectiveness and a potential heuristic to prioritize the strategies given characteristics of the schools.

CONCLUSION

Education has a long history of fads in which, as Ellis (2005) elegantly stated, “today’s flagship is often tomorrow’s aban- doned shipwreck” (p. 200). RtI has the potential to be the next in a long line of innovations about which school person-

nel are initially enthusiastic and result in immediate gains in student learning, but then implementation wanes as the initial enthusiasm fades. Educational change is a slow and difficult process, but it can result in lasting reform if school personnel consider long-term implications during the initial phases. Applying the framework for generalization during initial RtI implementation could potentially frontload sus- tainability efforts and provide a roadmap to sustainability. The goal of this article was to suggest potential methods to apply generalization strategies to RtI implementation efforts, primarily to provide directions for future research. Some of the strategies mentioned above would be easily implemented and some would require extensive research. However, given the increased frequency of RtI implementation, the research seems warranted.

Acknowledgments

This publication was made possible in part by Grant Number H325D090012 from the United States Department of Edu- cation Office of Special Education Programs. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the USDE OSEP.

REFERENCES

Alberto, P. A., & Troutman, A. C. (2009). Applied behavior analysis for teachers (8th ed). Upper Saddle River, NJ: Pearson.

Berkeley, S., Bender, W. N., Peaster, L. G., & Saunders, L. (2009). Imple- mentation of response to intervention: A snapshot of progress. Journal of Learning Disabilities, 42, 85–95.

LEARNING DISABILITIES RESEARCH 87

Burns, M. K., & Symington, T. (2002). A meta-analysis of prereferral in- tervention teams: Systemic and student outcomes. Journal of School Psychology, 40, 437–447.

Burns, M. K. (2007). RTI WILL fail, unless . . . .Communiqué, 35(5), 38–40. Burns, M. K., Appleton, J. J., & Stehouwer, J. D. (2005). Meta-analytic

review of responsiveness-to-intervention research: Examining field- based and research-implemented models. Journal of Psychoeduca- tional Assessment, 23, 381–394.

Burns, M. K., Vanderwood, M., & Ruby, S. (2005). Evaluating the readiness of prereferral intervention teams for use in a problem-solving model: Review of three levels of research. School Psychology Quarterly, 20, 89–105.

Crawford, C. G. (1980). Math without fear. New York: New View- points/Vision Books.

DuFour, R. (2004). What is a “professional learning community?” Educa- tional Leadership, 61(8), 6–11.

DuFour, R. (2011). Work together, but only if you want to. Phi Delta Kappan, 92(5), 57–61.

Ellis, A. K. (2005). Research on educational innovations (4th ed.). Larch- mont, NY: Eye on Education.

Fixsen, D. L., Blase, K. A., Naoom, S. F., & Wallace, F. (2009). Core implementation components. Research on Social Work Practice, 19, 531–540.

Foster-Fishman, P. G., Nowell, B., & Yang, H. (2007). Putting the sys- tem back into systems change: A framework for understanding and changing organizational and community systems. American Journal of Community Psychology, 39, 197–215.

Fuchs, D., Fuchs, L. S., & Stecker, P. M. (2010). The “blurring” of special education in a new continuum of general education placements and services. Exceptional Children, 76, 301–323.

Gansle, K. A., & Noell, G. H. (2007). The fundamental role of intervention implementation in assessing resistance to intervention. In S. Jimerson, M. K. Burns, & A. M. VanDerHeyden (Eds.), Handbook of response to intervention: The science and practice of assessment and intervention (pp. 244–254). New York: Springer.

Goldenberg, C. (2003). Settings for school improvement. International Jour- nal of Disability, Development and Education, 50, 7–16.

Grimes, J., Kurns, S., & Tilly, W. D. III. (2006). Sustainability: An enduring commitment to success. School Psychology Review, 35, 224–244.

Hargreaves, A. & Fink, D. (2000). The three dimensions of education reform. Educational Leadership, 57(7), 30–34.

Hill, D. R., King, S. A., Lemons, C. J., & Partanen, J. N. (2012). Fidelity of implementation and instructional alignment in Response to Interven- tion research. Learning Disabilities Research & Practice, 27, 116–124.

Horner, R. H., Sugai, G., & Todd, A. W. (2001). “Data” need not be a four-letter word: Using data to improve schoolwide discipline. Beyond Behavior, 11, 20–22.

Johnson, E. S., Pool, J. L., & Carter, D. R. (2012). Lessons learned from a tiered service delivery implementation project. Intervention In School & Clinic, 47, 139–143.

Kovaleski, J. F., Gickling, E. E., Morrow, H., & Swank, P. (1999). High versus low implementation of instructional support teams: A case for maintaining program fidelity. Remedial & Special Education, 20, 170– 183.

Kratochwill, T. R., Volpiansky, P., Clements, M., & Ball, C. (2007). Profes- sional development in implementing and sustaining multitier preven- tion models: Implications for response to intervention. School Psychol- ogy Review, 36, 618–631.

Lembke, E. S., Garman, C., Deno, S. L., & Stecker, P. M. (2010). One ele- mentary school’s implementation of Response to Intervention. Reading & Writing Quarterly, 26, 361–373.

Mahdavi, J. N., & Beebe-Frankenberger, M. E. (2009). Pioneering RTI systems that work. Teaching Exceptional Children, 42(2), 64–72.

McIntosh, K., Filter, K. J., Bennett, J. L., Ryan, C., & Sugai, G. (2010). Principles of sustainable prevention: Designing scale-up of school-wide positive behavior support to promote durable systems. Psychology in the Schools, 47, 5–21.

Noell, G. H., & Gansle, K. A. (2009). Moving from good ideas in educa- tional systems change to sustainable program implementation: Com- ing to terms with some of the realities. Psychology in the Schools, 46, 78–88.

Perkins, D. F., Feinberg, M. E., Greenberg, M. T., Johnson, L. E., Chilenski, S. M., Mincemoyer, C. C., et al. (2011). Team factors that predict to sustainability indicators for community-based prevention teams. Eval- uation and Program Planning, 34, 283–291.

Riley-Tillman, T. C. & Burns, M. K. (2009). Single case design for measur- ing response toeducational intervention. New York: Guilford.

Sarason, S. (1990). The predictable failure of educational reform. San Fran- cisco, CA: Jossey-Bass.

Sarason, S. (1996). Revisiting the culture of school and the problem of change. New York: Teachers College Press.

Stokes, T. F., & Baer, D. M. (1976). Preschool peers as mutual generalization- facilitating agents. Behavior Therapy, 7, 549–556.

Stokes, T. F., & Baer, D. M. (1977). An implicit technology of generalization. Journal of Applied Behavior Analysis, 10, 349–367.

Sugai, G., Lewis-Palmer, T., Todd, A. W., & Horner, R. H. (2001). School- Wide Evaluation Tool (SET). Eugene, OR: Educational and Community Supports.

Werts, M. G., Lambert, M., & Carpenter, E. (2009). What special ed- ucation directors say about RTI. Learning Disability Quarterly, 32, 245–254.

Ysseldyke, J. (2005). Assessment and decision making for students with learning disabilities: What if this is as good as it gets. Learning Dis- ability Quarterly, 28, 125–128.

About the Authors

Matthew K. Burns is a Professor of Educational Psychology, Coordinator of the School Psychology program, and Co-Director of the Minnesota Center for Reading Research at the University of Minnesota. His research interests include response to intervention, using curriculum-based assessment for instructional design to determine academic interventions, and facilitating problem-solving teams.

Andrea M. Egan is a doctoral student in special education at the University of Minnesota. Her research interests include assessment and intervention strategies for students with co-occurring academic and behavioral problems and methods to address these within a response to intervention framework.

Amy K. Kunkel is a graduate research assistant in Educational Psychology at the University of Minnesota. Her research interests include computer-assisted instruction and response to intervention.

Jennifer McComas, Ph.D., is a Professor with the Special Education Program in the Department of Educational Psychology at the University of Minnesota. Her current research interests include functional analysis and treatment for problem behavior and academic skill deficits, the influence of the principles of behavior on learning, and the influence of social context on severe problem behavior.

88 BURNS ET AL.: SUSTAINABILITY OF RTI IMPLEMENTATION

Meredith M. Peterson is a doctoral student in Educational Psychology at the University of Minnesota. Her research inter- ests include assessment and intervention strategies for students with behavioral problems within a response to intervention framework.

Naomi L. Rahn is a doctoral candidate in Educational Psychology, Special Education at the University of Minnesota. She has over 15 years of experience in early childhood special education. Her research interests include naturalistic language interventions, response to intervention, and teacher preparation.

Jennifer Wilson is a doctoral candidate in Educational Psychology, Special Education at the University of Minnesota. She holds a Director of Special Education license and has over 10 years of experience in the field. Her research interests include response to intervention and teacher preparation.

Copyright of Learning Disabilities Research & Practice (Blackwell Publishing Limited) is the property of

Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without

the copyright holder's express written permission. However, users may print, download, or email articles for

individual use.