discussion 5

rsfeir

articlediscussion5.pdf

Home >Education homework help >discussion 5

Exceptional Children

The Sustainability of Schoolwide Positive Behavior Interventions and Supports

JENNIFER H. COFFEY

ROBERT H. HORNER University of Oregon

ABSTRACT:r: A summary of the available literature on sustainability is provided and recommended

sustainability features are applied to implementation of schoolwide positive behavior interventions

and supports (SWPBIS). One hundred and seventeen schools from 6 states completed the sustain-

ability survey, which determined the presence of 8 sustainability features as they related to sustain-

ing schoolwide positive behavior supports. Results from a logistic regression analysis demonstrate

that together the sustainability features of administrative support combined with communication

and data-based decision making create the best-fitting model of sustainability for

SWPBIS. The results suggest that an educational innovation is more likely to be implemented and

sustained with fidelity if it (a) has support from an administrator who encourages communication

about the core features of the innovation and (b) uses data to plan and make changes. Implications

for large-scale, sustained iniplementation of evidence-based practices are provided.

E ducation research has made important advances in defining practices that are effective, or evidence-based, in improving students' academic and social

outcomes (Slavin, Holmes, Madden, Chamber- lain, & Cheung, 2010). Using evidence-based practices with fidelity is more important than ever as schools, districts, and state departments of education strive to close the gaps between the achievement of students with disabilities and their peers. Practitioners cannot afford to "experi-

ment" on students with practices that have not been proven effective. Instead, students need to be given the best possible chance for succeeding by receiving instruction and supports that have an evidence base. Although the content of the ev- idence-based practice, or innovation, is critical, it is insufficient to ensure academic or behavioral success (Datnow, 2005). How the innovation is executed (i.e., its implementation) is an underem- phasized component necessary for transforming the "promise" of an effective innovation into the outcome of improved student achievement

Exceptional Children 4 0 7

(Buzhardt, Greenwood, Abbott, & Tapia, 2006; Fixsen, Naoom, Blase, Friedman, & Wallace, 2005).

Unfortunately, sustained use of an innova- tion is not guaranteed even when full and effec- tive implementation occurs. The history of the field of education is littered with the detritus of successful programs that fell out of favor or were just forgotten over time, as evidenced by dusty kits, books, and teachers' guides safely tucked away in school closets all over the United States. In education, reinvention of the wheel occurs on a regular basis. Education systems do not have the funding or manpower to continually replace prac- tices, nor can students "wait" for the education system to get it right; fully implemented evi- dence-based practices are needed now.

F E A T U R E S OF

S U S T A I N A B I L I T Y

A review of the literature on sustainability of edu- cational practices produces a large number of con- ceptual models and recommendations, but few empirical analyses. Existing conceptual models consistendy emphasize the following variables as critical features that affect sustainability of an im- plemented practice:

• A Contextually Appropriate Innovation. A strong model of implementation for sustain- ability begins with an innovation that is aligned to state education agency (SEA) and local education agency (LEA) standards and requirements (Mihalic, Irwin, Fagan, Ballard, & Elliott, 2004). Furney and colleagues' lon- gitudinal policy analysis of four schools found that state and federal policy initiatives can in- fluence outcomes for all students but will not be wholly effective unless contextually appro- priate implementation is considered (Furney, Hasazi, Glark, & Keefe-Hartnett, 2003; Mi- halic et al., 2004). Datnow (2005) found in her study of the sustainability of comprehen- sive school reform models that changing dis- trict and state contexts affected the sustainability of these models, but that the effects were tempered by the schools' strate- gies for dealing with change, demonstrating that each level of the school system should be

considered when determining the contextual appropriateness of an innovation.

Staff Buy-In. It has been recommended that 80% of staff "buy in" before a decision is made by the school to implement an innova- tion (DeStefano, Dailey, Berman, & Mclner- ney, 2001). Buy-in is defined as verbal statements supporting change and the overt nonverbal behaviors necessary for change to take place (Boyce & Roman, 2002).

A Shared Vision. A shared vision is an agree- ment between school personnel about the core components of the innovation and what implementation of those core components will look like, as well as the teachers' desired outcomes for the innovation. A shared vision becomes tangible in a plan detailing how im- plementation and sustainability will be pro- grammed; vague or tentative plans typically end in unsuccessful implementation (Elias, Zins, Graczyk, oí Weissberg, 2003; Fullan, 2005).

Administrative Support. Administrative sup- port is the feature of sustainability most strongly emphasized in the literature (Elliott & Mihalic, 2004). Although different types of administrators play a role in sustaining in- novations in the school (e.g., district superin- tendents, SEA personnel), the principal is seen as the most critical player (Benz, Lind- strom, Unruh, & Waintrup, 2004; Heller & Firestone, 1995; Huberman, 1983) and has been described as the gatekeeper of change (Berman &C McLaughlin, 1976). The admin- istrator can increase the likelihood of the in- novation's sustainability by (a) acquiring resources for the implementation effort, (b) orienting staff to new ways of doing business, (c) providing clear expectations to the staff, and (d) prompting frequent feedback from staff regarding the progress of implementa- tion and the types of support they need (Adelman & Taylor, 1998; Blase & Fixsen, 2004; Fullan, 2002; Heller & Firestone, 1995; Kam, Greenberg, &c Walls, 2003; Mi- halic et al., 2004).

Leadership at Various Levels. Though the importance of administrative support in sus- taining an innovation cannot be overstated,

Summer 2012

leadership from other personnel in the school is also critical for success. Administrative turnover occurs frequently and administra- tors have many duties, so it is often necessary for leadership to come from within the ranks. In addition, there are typically one or two administrators in a school, whereas there are many practitioners. In various studies, practi- tioner leadership, especially when the practi- tioner is well respected by other school personnel, has led to greater commitment and use of the innovation (Berman & McLaughlin, 1976; Gottfredson & Gottfred- son, 2002; Mihalic & Irwin, 2003; Mihalic et al., 2004).

Ongoing Technical Assistance. Technical assistance is a "means of using knowledge to improve the adoption and implementation of some type of educational practice or proce- dure" (Yin & White, 1984). The quality of technical assistance activities (e.g., training and coaching) is of critical importance for the success of implementation (Adelman & Taylor, 1998; Berman & McLaughlin, 1976; Mihalic & Irwin, 2003; Mihalic et al., 2004; Ringeisen, Henderson, & Hoagwood, 2003). Focusing on practical classroom issues and skill building rather than on theoretical in- formation helps teachers to build che compe- tence that leads to sustained innovations (American Education Research Association, 2005; Berman & McLaughlin, 1976). In ad- dition, focusing activities on the core princi- ples of the innovation increases the likelihood that teachers will sustain the inno- vation (Elias et al., 2003; Sindelar, Shearer, Yendol-Hoppey, & Liebert, 2005).

Data-Based Decision Making and Sharing. One task of coaches and school leaders is to assist practitioners in collecting and analyzing data related to the innovation (Joyce & Showers, 2002). Having explicit systems to collect and share the data with the entire school staff, whether in celebration of im- provements or to provide corrective feedback, can increase short- and long-term commit- ment to an innovation (FuUan, 2005). Imple- mentation fidelity and outcome data can be used to improve implementation quality and

should be accessible to practitioners (Adel- man & Taylor, 2003; Greenwood, Delquadri, & Bulgren, 1993; Martinez & Harvey, 2004), as monitoring of implementation data allows for the innovation to be improved and refined over time (Berman & McLaughlin, 1976; DeStefano et al., 2001; Huberman, 1983; Weissberg & Utne-O'Brien, 2004).

• Continuous Regeneration. Regeneration is the set of procedures that allow a system to continually compare valued outcomes against current practice and modify practices to con- tinue to achieve these outcomes as the con- text changes over time (Mclntosh, Horner, & Sugai, 2009). Regeneration is necessary to prevent or to remedy an implementation dip (FuUan, 2002), which is a decrease in imple- mentation fidelity that occurs after a period of implementation and is the result of de- creasing levels of interest in the program. During this time, resources will be needed to ensure teachers can receive training that will review previously learned skills and teach new skills so that they can reach a more ad- vanced level with the innovation (Hatch, 2000). Cherniss (2006) further recommends that teachers (a) crepte a culture of experi- mentation, (b) set aside time for planning, and (c) create an open and fiexible decision- making structure.

The sustainability features provided here are not exhaustive; rather, they are the features most con- sistently recommended to lead to sustainability of innovations.

P O S I T I V E B E H A V I O R

I N T E R V E N T I O N S

A N D S U P P O R T S

A technology of tiered behavior supports at the universal, targeted group, and individual levels has been created over the years through the expansion of applied behavior analysis (Sugai & Horner, 2002). Termed positive behavior inter- ventions and supports (PBIS), this approach uses systems change methodology to minimize indi- viduals' problem behavior, ificrease their quality of life, and also increase their likelihood of success

Exceptional Children 4 O 9

academically and beyond (Carr et al., 2002). Schools implementing PBIS focus on building students' academic skills along with their social competencies. Through a behaviorally based sys- tems approach, PBIS enhances the capacity of the school to use research-validated ptactices and in- sttuction (Sugai et al., 2000).

The core components of schoolwide PBIS include (a) a statement of purpose, (b) schoolwide expectations, (c) procedures for teaching school- wide expectations, (d) a continuum of procedures for encouraging schoolwide expectations, (e) a continuum of procedures for discouraging prob- lem behaviors, and (0 procedures for using data to monitor the impact of schoolwide PBIS imple- mentation (Lewis & Sugai, 1999). Students at a PBIS school know which behaviors are appropriate, can expect to receive both social and tangible rewards when using those appropriate behaviors, and also know what to expect when they act inappropriately. Students at PBIS schools do not "fall through the cracks" because educa- tots, through the use of office discipline referrals and systemwide communication, monitor all stu- dents who exhibit problem behaviors. A PBIS school is unified in its approach to supporting students both academically and behaviorally. Ad- ditional suppott is provided along a continuum all the way to functional behavioral assessment (Mclntosh, Chard, Boland, & Homer, 2006). Re- sults have demonstrated that the use of PBIS pto- cedures results in positive changes throughout the school: A body of tesearch provides evidence of PBIS's effectiveness in decreasing problem behav- iors for the whole school (Hotnet et al., 2009; Nelson, Hurley, Synhorst, & Epstein, 2008; Nel- son, Martella, & Marchand-Martella, 2002; Safran &C Oswald, 2003).

A PBIS school is unified in its

approach to supporting students both academically and behaviorally.

In addition to its schoolwide benefits, PBIS offers important and meaningful benefits to stu- dents with disabilities. First is the idea previously intimated that in a PBIS school each teacher feels responsibility for each student in the school. The

model of a sepatate general education and special education has led to students with disabilities being under the purview of special educators and experiencing a lack of connection with genetal ed- ucatots. PBIS encoutages and enables educatots to share a commitment fot all students, whether or not they are on record as the ptimaty provider of services (Btadshaw, Koth, Bevans, Ialongo, & Leaf, 2008). Second, full implementation of a preventive model of behavior suppott may decrease the number of students who are inappro- priately determined to need mote intensive sup- potts. When the number of students moving into mote targeted or intensive services decreases, the students who apptoptiately teceive those services may receive more focused attention and assistance (Mclntosh et al., 2006). Third, the teaming struc- tures that ate a ctitical component of PBIS imple- mentation enable the collaborative work necessary to support students with intensive behavior needs (Ebet, Sugai, Smith, & Scott, 2002). Accordingly, a school that attempts to use evidence-based in- terventions for students with intensive behavior needs is likely to have more success if they alteady have universal and tatgeted preventive supports and interventions in place (Skiba, 2002). Addi- tionally, the use of PBIS at the univetsal and tat- geted levels can provide a tecord of interventions, observations, and assessments that can be used as part of a comptehensive evaluation for special ed- ucation. In all, a preventive system using positive behavior supports makes the provision of a free and appropriate public education (FAPE) more likely (Ebet et al., 2002; Skiba, 2002), and a schoolwide system of PBIS may lead to a compe- tent school culture capable of sustaining the use of evidence-based practices (Hotnet & Sugai, 1999) that benefit students with disabilities.

This study was conducted to identify and validate the components of sustainability that increase the ability of schools to sustain school- wide PBIS (SWPBIS). This was accomplished by detetmining the significant differences of sustain- ability features in schools that have sustained SW- PBIS and schools that have not. As a result, a model of sustainability will be presented for SW- PBIS with the intention that the model may be generalized to othet innovative ptactices being implemented in schools.

Summer 2012

METHOD

SAMPLE AND PARTICIPANT SELECTION

All schools included in this study had existing PBIS implementation data from the Schoolwide Evaluation Tool (SET; Horner et al., 2004; Sugai, Lewis-Palmer, Todd, & Horner, 2001) or the Team Implementation Ghecklist (TIG; Sugai, Horner, & Lewis-Palmer, 2001). The PBIS Tech- nical Assistance Genter database houses results for schools using these tools. When data were ex- tracted for this study, results were available from 1998 to 2006, with 429 schools in the SET database and 932 schools in the TIG database.

The operational definition of sustaining was established at a minimum of 3 years of imple- mentation with the last 2 years demonstrating cri- terion levels of implementation fidelity. Three to 5 years of implementation has been widely ac- cepted as a marker for sustained use of a program (Mihalic & Irwin, 2003; Rog et al., 2004; Schräg, 1996). To be included in the sample, schools needed to have implemented PBIS for at least 3 years. The sample consisted of two groups: sus- tainers and nonsustainers. The first sample, sus- tainers, was made up of schools that sustained their PBIS system with fidelity, as demonstrated by a minimum SET total score of 80% for the last 2 years on record or 2 years above 80% and a consecutive year with a score of 75% or above. The second sample, nonsustainers, was made up of schools that had observed for at least 3 years but did not meet the criteria for sustainer. School characteristics gathered for all schools included (a) socioeconomic status (SES) of the students (the percentage of students receiving a free or reduced lunch), (b) size of school (student enrollment), (c) school academic level (e.g., middle school), (d) geographic location (e.g., rural), (e) number of years PBIS has been implemented, (f) percentage of minority students, and (g) Title I status.

The sample schools were asked to take part in a survey containing 40 questions about the sus- tainability components in place in each school related to SWPBIS implementation. The survey is available upon request and is explained in more depth in the next section. Recruitment was accomplished by sending schools the sustainabil- ity survey with a letter explaining the purpose and importance of the study, the role the participating

schools would play, and the results that would be reported, including how these results could en- hance the field's knowledge about PBIS sustain- ability. Through the original screening process, 146 schools were categorized as sustainers. Non- sustainers were fewer in number, with 111 schools. Surveys were sent to these 257 schools, and unresponsive schools were sent two subse- quent mailings.

MEASURES

Extant data were collected for two methods of assessment: the SET, which measures the imple- mentation of the core features of SWPBIS, as de- scribed in the previous pages, and the TIG, which tracks SWPBIS implementation activities in the school. For both measures, scores are percentages of essential features of SWPBIS in place—a score of 100 equates to having 100% of the features in place.

Schoolwide Evaluation Tool (SET). The SET is designed to assess and evaluate the critical fea- tures of SWPBIS for each school year (Sugai, Lewis-Palmer, et al., 2001). An external reviewer administers the 28-item research tool on site by reviewing materials that relate to SWPBIS, per- forming observations, and conducting staff and student interviews. The SET was found to be valid and reliable in measuring the implementa- tion of schnolwide PBIS, can be measured with high interobserver agreement, demonstrates excel- lent test-retest reliability (97.3%), produces a valid index of schoolwide behavior support as defined by Lewis and Sugai (1999), and is sensi- tive enough to be useful in documenting change in levels of implementation of SWPBIS programs in schools (Horner et al., 2004). In a study con- ducted by Horner and colleagues, the average in- terobserver agreement on SET item scores was 99% across 17 schools (2004). The SET has seven subscales that measure the essential features of SWPBIS:

• Behavior expectations defined.

• Behavior expectations taught.

• Ongoing behavior reward system

• System for responding to behavior violations.

• Monitoring and decision making.

Exceptional Children

• Management.

• District-level support.

The subscale alphas were found to range from .63 to .92 with a full-scale alpha of .90 (Horner et al., 2004).

Team Implementation Checklist (TIC). The TIC monitors implementation and maintenance of SWPBIS systems. When beginning implemen- tation, the school's SWPBIS team completes the checklist and uses the results to create an action plan that describes the most needed resources (e.g., training, coaching, and financial resources). The T I C is then used at regular intervals (monthly or quarterly) to monitor progress. Im- plementation activities are divided into two sec- tions: Startup Activities and Ongoing Activities. The Startup Activities section is primarily used to monitor the work of initial implementation. The Ongoing Activities section assists the team in evaluating the activities required to sustain a PBIS system. Six main areas of implementation activi- ties are contained in the Startup Activities section:

• Establish commitment.

• Establish and maintain team.

• Conduct self-assessment.

• Establish schoolwide expectations.

• Establish information system.

• Build capacity for function-based support.

The implementation goals in the Ongoing Activi- ties section are

• SWPBIS team has met at least monthly.

• SWPBIS team has given status report to faculty at least monthly.

• Activities for SWPBIS action plan imple- mented.

• Accuracy of implementation of SWPBIS action plan assessed.

• Effectiveness of SWPBIS action plan imple- mentation assessed.

• SWPBIS data analyzed.

Barrett, Bradshaw, and Lewis-Palmer (2008) reported that a slightly modified version of the TIC was found to have high internal consistency

(Cronbach's alpha = .93, n = 1,633 forms com- pleted).

Sustainability Survey. The sustainability sur- vey contains questions about the organizational features of the school that have facilitated or inhibited implementation and sustainability of SWPBIS. By mapping the literature-based sus- tainability model onto the SWPBIS sustainability model and determining shared features, survey questions were created that have a foundation in research and are appropriate for examining the SWPBIS system. The survey questions are catego- rized by the sustainability model components, with five questions for each of the following sub- scales:

• Shared vision and resources in a contextually appropriate setting.

• Buy-in and agreement.

• Ongoing and active technical assistance.

• Use of data to make decisions.

• Lateral and vertical communication.

• Leadership from various levels.

• Administrative support.

• Regeneration.

The formatting of the survey includes closed questions on the Likert scale, frequency ratings, and two open-ended questions about aspects of sustainability specific to the school.

The survey's construct validity and clarity were analyzed by a panel of experts, all of whom have a background working with SWPBIS sys- tems and are well-versed in issues of sustainability. Feedback from the panel was used to make modi- fications before the survey was sent to pilot schools. Feedback from these pilot schools regard- ing the format of the survey, comprehensiveness of the content, and question clarity was then used to revise the survey. Internal consistency reliability (coefficient alpha) was calctilated from completed questionnaires using item total correlations and subscale correlations, resulting in adjustments to the survey's items and subscales.

PROCEDURE

When comparing the surveys of sustaining schools and nonsustaining schools, is the level of

4 1 2 Summer 2012

TABLE 1

Item Total Correlations and Suhscale Alphas

Administrative Support

Item

5 Overall

Subscale

Consistency

.78

.80

.79

.69

.82

Communication

Item

3 4

.73

.76

.74

.70

.77

.78

Data-Based Decision Making

Item a

1 .81 2

3 .78

4 .63

5 .66

.78

Regeneration

Item

3 4

.61

.73

.60

.66

.72

Technical Assistance and Dissemination

Item

3 4

.63

.55

.60

.65

sustainability associated with ratings on specific sustainability components as measured by the sus- tainabilit)' survey? Logistic regression was used to test the associations between the predictor vari- ables—the sustainability components in place in the school as measured by the sustainability sur- vey as continuous variables—and the criterion variable, which is the presence or absence of sus- tainability as determined by the results of the im- plementation measure as a dichotomous variable. The school is the unit of analysis.

One question inherent to this study is whether the latent constructs included in the sus- tainability survey validly discriminate between sustaining and nonsustaining schools. Statistical analyses are necessary to determine if there is a specific subset of variables that most parsimo- niously describes a model of sustainability. To an- alyze survey results witb logistic regression, the following activities were completed:

• Analyze each predictor and its relation with the outcome variable.

• Model the probability of the outcome of sus- tainability by a series of univariate logistic regression analyses.

• Fit a preliminary multivariate logistic model using all predictors of importance.

• Fit alternative models.

• Add interactions.

• Remove statistically insignificant predictors.

• Gompare performance of alternative models on the significance of each predictor, good- ness of fit statistics, accuracy of prediction, and diagnostic results.

The null hypothesis posits that the sustain- ability model is not an improvement over the in- tercept-only model {a = .05). The alternative hypothesis is that the sustainability model does provide a better fit to the data by demonstrating a significant improvement over the intercept-only model. The logistic model was applied to the sus- tainability survey data using the LOGISTIG and GENMOD procedures implemented in SAS® Release 9.1 (SAS Institute, 2004).

PRELIMINARY ANALYSES

Of the 257 surveys sent to PBIS team leaders, 117 were returned (42%). Seventy-nine (79) of the respondents were sustainers and 38 were non- sustainers. Goefficient alphas were computed at the subscale level and items that diminished the subscales' consistency were dropped. Generally accepted conventions for coefficient alphas are: S.90 = excellent, .80 to .89 = good, .70 to .79 = acceptable, .60 to .69 = questionable, .50 to .59 = poor, and <.5O = unacceptable (George & Mallery, 2003). Three subscales were released from the sustainability survey because their scale alphas were less than .60: (a) buy-in and commit- ment, (b) leadership from various levels, and (c) shared vision and resources. The five remaining subscales had coefficient alphas ranging from .65

Exceptional Children 4 1 3

TABLE 2

Descriptive Statistics for tbe Sustainability Survey

Survey Category

Administrative support (20 pts.)

Communication (25 pts.)

Data-based decisions making (20 pts.)

Regeneration (20 pts.)

Technical assistance (20 pts.)

16.54

21.19

15.32

16.05

18.05

Sustainers

4.07

3.44

2.61

2.58

4.05

Subscales by Sample

Nonsustainers

13.75

18.18

13.69

15.67

16.83

3.71

3.28

2.53

2.38

3.88

Group

All Respondents

15.65 20.21 14.80 15.93 17.97

3.70 3.66 2.68 2.51 4.06

117 116 116 117 115

Note. The maximum number of points available is provided for each subscale.

to .82. Table 1 demonstrates item total correla- tions and subscale alphas. Descriptive statistics for the remaining sustainability survey categories are presented in Table 2. Each of the survey cate- gories or subscales is divided into sustaining and nonsustaining groups and the overall means are given for each of the subscales.

A logistic regression model was fit to the sur- vey data to explain the predicted odds of sustain- ability (i.e., sus = 1). There were five possible main effects: (a) administrative support, (b) com- munication, (c) data-based decision making, (d) regeneration, and (g) technical assistance. With the exception of technical assistance, which was not included in the final logistic regression model, these are the categories of the sustainability survey that demonstrated strong internal consistency. The Maximum Likelihood Estimator (MLE) is the method used with logistic regression analysis to estimate coefficients for the fitted model. Each of the independent variables was regressed with the outcome variable of sustainability. Four of the five variables—administrative support, communi- cation, data-based decision making, and technical assistance—showed a significant relationship {p ^ .05) with the dependent variable of sustainability. The Pearson correlations between independent variables, the Variance Infiation Factor (VIF), the Tolerance, and the absolute correlation between the coefficient estimates all indicated the absence of multicollinearity.

R ES U LTS

LOGISTIC REGRESSION ANALYSIS

In performing backward logistic regression, we first tested a model that included all variables de- termined significant in the univariate regression. This model did not have a significantly better fit than the intercept-only model. The second multi- variate logistic regression included the three vari- ables found to be most significant in the univariate logistic regression models: administra- tive support, communication, and data-based decision making. The correlation between admin- istrative support and communication (.60), as seen in Table 3, seemed to create substantial inter- ference in the model by causing significant changes in the coefficients; therefore, it was neces- sary to test the two variables together as an inter- action effect. Statistics that test for predictiveness and effectiveness of the fitted model, the Akaike Information Criterion (AIC), the Schwarz Crite- rion (SC), and negative twice the log likelihood (-2 Log L) were tested for the model with and without the independent variables. When the model that includes data-based decision making and an interaction between administrative sup- port and communication is compared to the in- tercept-only model, the AIC, SC, and —2 Log L decrease by 26.05, 15.07, and 34.05, respectively

When comparing the intercept-only model and the model that includes data-based decision making and an interaction between administra- tive support and communication, the model that

Summer 2012

TABLE 3

Pairwise Correlations Between Independent Variables

Independent Variable

Administrative support

Communication

Data-based decision making

Regeneration

Technical assistance

Communication

.60

Data-Based Decision Making

.42

.43

Regeneration

.13

.31

.24

Technical

Assistance

.28

.49 •

.26

.40

includes the two effects predicts the outcome of sustainability significantly better than the model without any effects. The Wald chi-square of 18.50 is significant [p = .001), and thus we can reject the null hypothesis that states all coefficients are zero. The maximum likelihood ratio results demonstrate significance.

PREDICTION OE THE MODEL

The fitted model has a strong prediction for sus- tainability measuted by a concordance of about 82%. The Somet's D index of the degree that pre- dicted probabilities match actual outcomes is 64%, interpreted as 64% fewer etrors made in predicting which of two schools demonstrated sustainability by utilizing the estimated probabili- ties rather than by chance alone. The Hosmer and Lemeshow goodness of fit statistic (Hosmer & Lemeshow, 1989) is 3.61 with/) = .89, indicating a good fit. When the probability is not significant (p > .05), a good fit is indicated. Thus, the logis- tic fitted model appears to be the right model for detecting sustaining schools.

Univariate logistic regression analyses were conducted with the five subscales (administtative support, communication, data-based decision making, regeneration, and technical assistance), and all but tegenetation were significant {p < .05), as shown in Table 4. Accordingly, a multi- variate logistic regression model was created that included all individually significant vatiables; however, this fout-variable model did not have a significantly better fit than the intercept-only model. Technical assistance, the vatiable that had the smallest effect within the model, was thus removed, and only administrative suppott, com- munication, and data-based decision making remained. Results of the new model with these three main effects demonstrated an intetaction occurring between administrative support and communication, further supported by the medium-sized correlation between the two vari- ables. This led to the creation of a model that included two main effects: data-based decision making and an interaction between admini- sttative support and communication. The null hypothesis could be tejected because a parsi- nionious model containing the two effects of

TABLE 4

Univariate Logistic Regression Analyses

Independent Variable

Administrative support

Communication

Data-based decisions making

Regeneration

Technical assistance

P of Likelihood Ratio of Overall Model

.0002

<.OOO1

.0024

.4330

.0378

Wald Chi-Square

12.39

10.77

8.48

0.61

4.00

P Value

.0004

.0010

.0036

.4331

.0414

Exceptional Children

data-based decision making and an interaction between administrative support and communica- tion had a significantly better fit than the inter- cept-only model. Accordingly, a school that has data-based decision making along with a combi- nation of administrative support and communica- tion has better odds of sustaining PBIS than schools that do not have this combination of organizational features in place.

D I S C U S S I O N

RESPONDENT DEMOGRAPHICS

Many more elementary schools returned surveys than middle or high schools. About 12% of the surveys went out to middle schools and 16% of the responses were from middle schools, whereas 14% of respondents were K-8 schools. High schools made up 9% of the original sample and returned 4% of the surveys. PBIS is more preva- lent in elementary schools, so the ratio of surveys returned (elementary schools returned about 65% of the surveys) is appropriate for the national makeup of PBIS schools. Tukey-Kramer one-way analysis of variance (Hsu, 1996; ANOVA) was used to compare the means of the sustainability survey for the various demographic categories and pairwise comparisons were formed. To ensure familywise error was taken into consideration, comparisons were not considered significantly dif- ferent from each other unless the adjusted/) value was jess than .01. Tukey-Kramer pairwise com- parisons resulted in no significant differences be- tween the different levels of schools, including elementary, K-8, and K-12 schools.

A few significant effects were found for the different demographic categories. Schools in small- or medium-sized cities were significantly more likely to have a higher rating for the sustain- ability component of regeneration than schools in rural areas. Ghallenges to sustaining innovations in rural areas, such as the economic outlook of the community, staffing, and opportunities to collaborate, have been reported by multiple au- thors (Kannapel & DeYoung, 1999; Seal & Har- mon, 1995; Theobald & Nachtigal, 1995). Also, schools with 100 to 299 students (as compared to schools with 300-749 students) had significantly

greater scores for three organizational features: (a) communication, (b) data-based decision making, and (c) technical assistance. Further differences were seen when comparing schools that have im- plemented for 5 years and schools that have im- plemented for 3 years. Schools that have implemented for 5 years or more have a greater level of administrative support, data-based deci- sion making, and technical assistance. It is likely that a school that has continued to implement PBIS would have assistance from their adminis- trator, would be using data to inform their prac- tices, and would be receiving ongoing training to assist in their PBIS activities. In addition, regular use of data may result in administrators and staff having more accurate and timely feedback, which more closely connects the desired outcomes to the implementation activities.

OPEN-ENDED QUESTIONS

Two open-ended questions were included in the survey:

a. Is there anything else that helped your school to sustain PBIS?

b. Were there any major obstacles for your school that made sustaining PBIS difficult?

Of the 117 respondents who returned surveys, 63 respondents explained what had helped them sus- tain PBIS and 84 respondents described obstacles to their school's efforts to sustain. The areas reported as conducive to sustainability were lead- ership (principals, districts, SEAs, teachers, coaches, PBIS coordinators, consultants, and counselors) with 20 responses, teacher buy-in with eight responses, funding with seven re- sponses, time to meet/regularly held meetings with six responses, decision-making procedures and technical assistance opportunities, both with five responses. If fewer than five respondents re- marked on the same facilitators they are not listed here.

When asked what had helped their SWPBIS implementation to sustain, the largest number of respondents named leadership. Implementation and sustainability research have demonstrated that leadership is the base upon which a sustain- able program is built (Benz et al., 2004; Berman & McLaughlin, 1976; Heller & Firestone, 1995;

4 1 6 Sur? r2012

Huberman, 1983) and that it has a direct rela- tionship to quality implementation (Kam et al., 2003). Leaders provide direction and motivation for the innovation by sheltering teachers from other pressures, demonstrating that the innova- tion is part of the central mission of the school, and communicating positively about the innova- tion with staff (Kam et al., 2003; Rohrbach, Cra- ham, & Hansen, 1993; Sindelar et al, 2005; Solomon, Battistich, Watson, Schaps, & Lewis, 2000). In terms of district support, one respon- dent explained that "centtal office support and the message that PBIS is important and an expec- tation for schools" led to sustainability. District support can increase the capacity of schools to carry out innovations (Martinez & Harvey, 2004). The importance of teacher leadership in one respondent's school was demonstrated by "a very strong universal team, led by two highly effi- cient veteran teachers who were respected by staff," which together resulted in sustainability. This response about teaming is reinforced by Sin- delar and colleagues' finding that teachers on long-standing teams were less likely to face serious teaching challenges. Martinez and Harvey agree that having a cohesive team that has time to col- laborate will lead to a more successful effort.

The areas reported as conducive to sustainability were leadership, teacher buy-in, funding, time to meet/regularly

held meetings, decision-making procedures, and technical assistance opportunities.

Teacher buy-in and commitment was the next most frequently reported factor leading to sustainability; two of these respondents stated that better than expected outcomes in the first year led to increased teacher commitment. Short- term gains can be critical (Elias et al., 2003; Han & Weiss, 2005), especially in this time of in- creased accountability when innovations need to show quick results to be maintained in the schools. Implementation and sustainability re- search have demonstrated that teacher buy-in is needed before the program can be implemented, but perhaps there are multiple paths to buy-in and, along with that, sustainability.

Regarding the second open-ended question ("Were there any major obstacles for your school that made sustaining PBIS difficult?"), 22 respon- dents stated that funding was an obstacle, whereas 17 stated they were unable to meet consistently as a team and did not have enough time to complete all of the activities that were necessary for a strong PBIS effort. Eight respondents believed that they needed more staff to implement PBIS with fi- delity. Also making the human resources issues more difficult were turnover in administration and turnover in staff, both reported by five respondents. Further, seven respondents believed lack of staff buy-in hurt the PBIS effort in schools.

Appropriate funding was reported by seven individuals as a beneficial factor in sustaining PBIS; conversely, 22 respondents stated that inad- equate funding was an obstacle in sustaining PBIS. An organizational structure that ensures re- sources are dedicated to the innovation will assist the effort (Adelman & Taylor, 1998; Cherniss, 2006; Mihalic & Irwin, 2003; Mihalic et al., 2004). Resource allocation also demonstrates pri- ority. When innovations are not assigned a high priority, they tend to be marginalized in favor of activities that have high priority (Center for Men- tal Health in Schools, 2010).

Four respondents spoke of philosophical issues. Philosophical beliefs are cultural and orga- nizational traditions of the schools and exert im- plicit force on the change effort (McLaughlin & Mitra, 2001). For some teachers, the use of rewards, whether they are tangible or intangible, is seen as inappropriate, and three respondents reported this was the case in their schools. One respondent reported the obstacle of "philosophi- cal issues of some staff members who give out few positives and no negatives." There is a delicate balance between adoption and adaptation of an innovation when ensuring the innovation fits the context of the schools, including the philosophi- cal beliefs pf the teachers (Gager & Elias, 1997; Martinez & Harvey, 2004; Weissberg Sí Creen- berg, 1998). Although it is important that the features of the innovation fit well with the philo- sophical views of the school staff, modifying the innovation by eliminating essential components likely decreases the probability of achieving the desired outcomes (Battistich, Schaps, Watson,

Exceptional Children 4 1 7

Solomon, &C Lewis, 2000; Kam et al., 2003; Karachi, Abbott, Gatalano, Haggerty, & Fleming, 1999).

LOGISTIC REGRESSION MODEL

The literature describes the organizational feature of communication mainly in tetms of collabora- tion, specifically through regular communication and staff meetings, which can create a sense of collective efficacy (Berman & McLaughlin, 1976; Rog et al., 2004). In one study, the more time teachers spent communicating about the prac- tice, the more skilled their implementation of the innovation was (Bauchner, Eiseman, Gox, & Schmidt, 1982). In the model of sustainability tested here, lateral and verhal communication occurs not just between teachers but also between teachers and administrators. Without administrative support, vertical communication is unlikely to occur consistently or strategically. In addition, administrative suppott is necessary to allow time for teachers to collaborate laterally.- Accordingly, this study's model of sustainability logically leads to an interaction between the inde- pendent variables of administrative suppott and communication. Gommunication between peers, between teachers and administrators, between schools and centtal district offices, and between LEAs and SEAs is necessary if an innovation is to be seen as important enougb to teceive ongoing support from all levels. Most of this communica- tion is not possible without support from admin- istrators. Also, dedicated time for teachers to collaborate on an innovation is unlikely to occur without an administratot's support or assistance. For all of these reasons, administrative support and communication seem to logically comple- ment each other, and one can reasonably be seen as a prerequisite of the other. Of all the organiza- tional features hypothesized to lead to sustainabil- ity, administrative support has the largest research base and is the most well known by those work- ing in the field of education who are concerned with sustainability of innovations.

Data-based decision making did not meet the .05 cutoff but did have Ap value less than .10, and because it is supported by responses to the open-ended questions, it has research validation and is part of an overall model that fits signifi-

cantly better than an intercept-only model. It re- mained in the final logistic regression model with the intetaction between administrative support and communication. Part of what makes commu- nication in the system of PBIS so successful is that PBIS team membets and other educatots are able to use data to discuss the status and goals of theit school. Data helps administrators to make decisions about programming and modification of instructional practices and aspects of the learn- ing and social environment.

L I M I T A T I O N S

The analyses in this study wete exploratory and caution should be used in drawing conclusions from the results. The main limitations of this study were the instrumentation used and the sam- ple sizes. We used a previously untested survey in- strument to make associations between organizational features and sustainability. This survey instrument is unlikely to have been com- prehensive in measuring the presence of the eight sustainability components that resulted from the literature review. In addition, the subscale of administtative support had fairly high correlations with most of the other subscales. When using logistic regression, it was difficult to determine effects of other subscales wbenever administrative support was included in the model. Perhaps the construct of administrative support needs to be conceptualized differently, parsing out specific administrative activities to the various categories (e.g., data-based decision making, and technical assistance). Without question, administrative sup- port touches upon each of the organizational fea- tures, so determining if there is a way to create a stand-alone organizational feature of administra- tive support could be a next step in this line of re- search. In addition, when the construct validity of a survey, its internal consistency, and the norms associated with the instrument are being estab- lisbed, multiple iterations surveying different samples of the population are necessary. This study is merely the first of many steps necessary for the survey to be accepted as valid and reliable.

The second main limitation is that the sam- ple groups for research questions one and two are unbalanced, with the sustainers making up the

4 1 8 Sun r2012

preponderance of the sample. This mirrors the imbalance in the databases from which the sam- ple was drawn. Although the sample size for the nonsustainers was large enough to meet the crite- ria for minimum cell sizes for the statistical analy- ses, a follow-up study with more balanced samples would increase the probability that the conclusions drawn in this study are based on real differences in the populations.

This study analyzed survey and existing im- plementation data to determine the organiza- tional and programmatic features that would predict or be associated with sustainability. Results demonstrated that having a combination of the organizational features of administrative support and communication along with data- based decision making is associated with schools sustaining PBIS over a number of years. Training and practice in special education can be enhanced by the findings described in this article; however, much work is left to be completed. A focus on evidence-based practices is only part of the equa- tion in reaching the desired student outcomes (Odom, 2009). Unless we support the initial and continued implementation of those practices, the field of special education cannot hope for signifi- cant and lasting improvements for children with disabilities.

R E F E R E N C E S

Adelman, H., & Taylor, L. (1998). Reframing mental health in schools and expanding school reform. Educa- tional Psychologist, 33(4), 135-152. http://dx.doi.org /10.1207%2Fsl5326985ep3304_l

Adelman, H., & Taylor, L. (2003). On sustainability of project innovation as systemic change. Journal of Edu- cational and Psychological Consultation, 14(1), 1-25. http://dx.doi.org/10.1207%2FSl532768XJEPCl401_ 01

American Education Research Association. (2005). Teaching teachers: Professional development to improve student achievement. Research Points: Essential Information for Education Policy, 3(1), 1—4.

Barrett, S. B., Bradshaw, C. P., & Lewis-Palmer, T. (2008). Maryland statewide PBIS Initiative: Systems, evaluation, and next steps. Journal of Positive Behavior Interventions, 10(1), 105-114. http://dx.doi.org/10 .1177%2F109830070731254l

Battistich, V, Schaps, E., Watson, M., Solomon, D., & Lewis, C. (2000). Effects of the child development pro- ject on students' drug use and other problem behaviors. Journal of Primary Prevention, 27(1), 75—99. http:// dx.doi.org/10.1023%2FA%3A10070574l4994 Bauchner, J. E., Eiseman, J. W, Cox, P L., & Schmidt, W. H. (1982). People, policies, and practices: Examining the chain of school improvement. Volume III: Models of change. Andover, MA: The NETWORK.

Benz, M. R., Lindstrom, L., Unruh, D., & Waintrup, M. (2004). Sustaining secondary transition programs in local schools. Remedial and Special Education, 25(\), 39-50. http://dx.doi.org/10.1177%2F0741932504025 0010501

Berman, P, & McLaughlin, M. W (1976). Implemen- tation of educational innovation. The Educational Eorum, 11(3), 343-370. http://dx.doi.org/10.1080%2 FOO131727609336469

Blase, K., & Eixsen, D. (2004). Infrastructure for imple- menting and sustaining evidence-based programs with fidelity. Tampa, FL: National Implementation Research Network.

Boyce, T E., & Roman, H. R. (2002). Institutionaliz- ing behavior based theory: Theories, concepts and prac- tical suggestions. The Behavior Analyst, 3(1), 76—82.

Bradshaw, C , Koth, C , Bevans, K., Ialongo, N., & Leaf, P. (2008). The impact of school-wide positive be- havioral interventions and supports (PBIS) on the orga- nizational health of elementary schools. School Psychology Quarterly 23(4), 462-473. http://dx.doi.org /10.1037%2Fa0012883

Buzhardt, J., Greenwood, C. R., Abbott, M., & Tapia, Y. (2006). Research on scaling up evidence-based instructional practice: Developing a sensitive measure of the rate of implementation. Educational Technology Research and Development, 54(5), 467-492. http://dx.doi.org/10.1007%2Fsll423-006-0129-5 Carr, E., Dunlap, G., Horner, R., Koegel, R., Turnbull, A., Sailor, W , . . . Fox, L. (2002). Positive behavior support: Evolution of an applied science. Journal of Pos- itive Behavior Interventions, 4, 4-16. http://dx. doi.org/10.1177%2F 109830070200400102

Center for Mental Health in Schools. (2010). Turning around, transforming, and continuously improving schools: Eederal proposals are still based on a two- rather than a three-component blueprint. Retrieved from http://smhp.psych.ucla.edu/pdfdocs/tutning.pdf

Cherniss, C. (2006). School change and the microsociety program. Thousand Oaks, CA: Corwin.

Datnow, A. (2005). The sustainability of comprehen- sive school reform models in changing district and state

Exceptional Children

contexts. Educational Administration Quarterly, 4l{\),

121-151.

DeStefano, L., Dailey, D., Berman, K., & Mclnetney, M. (2001). Synthesis of discussions about scaling up effec- tive practices (OSEP Publication Number HS970 17002). Washington, DC: U.S. Department of Edu- caiton. Office of Special Education Programs.

Eber, L., Sugai, G., Smith, C. R., & Scott, T. M. (2002). Wraparound and positive behavioral interven- tions and supports in the schools. Journal of Emotional and Behavioral Disorders, 10{5), 171-180. http:// dx.doi.org/10.1177%2F10634266020100030501

Elias, M. J., Zins, J. E., Gtaczyk, P A , & Weissberg, R.

P. (2003). Implementation, sustainability, and scaling

up of social-emotional and academic innovations in

public schools. School Psychology Review, 32{3),

303-319.

Elliott, D. S., & Mihalic, S. (2004). Issues in dissemi- nating and replicating effective prevention programs. Prevention Science, 5(1), 47—52. http://dx.doi.org /10.1023%2FB%3APREV.0000013981.28071.52

Fixsen, D. L., Naoom, S. F., Blase, K. B., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: National Imple- mentation Research Network, Louis de la Parte Florida Mental Health Institute, University of Soutb Florida. Available online at http://nirn.fmhi.usfedu/resources /publications/Monogtaph/index.cfm

Fullan, M. (2002). The change leader. Educational Leadership, 59(8), 16-21.

Fullan, M. (2005). Leadership and sustainability. Thou- sand Oaks, CA: Corwin.

Futney, K. S., Hasazi, S. B., Clark, K , & Keefe-Hart- nett, J. (2003). A longitudinal analysis of shift:ing pol- icy landscapes in special and general education reform. Exceptional Children, 70, 81-94.

Gager, P. J., & Elias, M. J. (1997). Implementing pre- vention programs in high-risk environments: Applica- tion of tbe resiliency paradigm. American Journal of Orthopsychiatry 67{3), 3 6 3 - 3 7 3 . http://dx.doi.org /10.1037%2Fh0080239

George, D., & Mallery, P (2003). SPSS for Windows Step by Step: A Simple guide and reference. 11.0 update (4tb ed.). Boston, MA: Allyn & Bacon.

Gottfredson, D. C , & Gottfredson, G. D. (2002). Quality of school-based prevention programs: Results from a national survey. Journal of Research in Crime and Delinquency, 39{l), 3 - 3 5 . h t t p : / / d x . d o i . o r g / 1 0 .1177%2F002242780203900101

Greenwood, C. R., Delquadri, J., &C Bulgren, J. (1993). Current challenges to behavioral technology in the re-

form of schooling: Large-scale, high-quality implemen- tation and sustained use of effective educational prac- tices. Education and Treatment of Children, J6{4), 401-404.

Han, S. S., & Weiss, B. (2005). Sustainability of teacher implementation of school-based mental health programs. Journal of Abnormal Child Psychology, 33(6), 665-679. http://dx.doi.org/10.1007%2Fsl0802-005- 7646-2

Hatch, T (2000). What does it take to break the mold?

Rhetoric and reality in new American schools. Teachers

College Record, 102, 561-589. http://dx.doi.org/10

.llll%2F0l6l-4681.00068

Heller, M. F., & Firestone, W A. (1995). Who's in

charge here? Sources of leadership for change in eight

schools. Elementary School Journal, 96(\), 6 5 - 8 6 .

http://dx.doi.org/10.1086%2F461815

Horner, R., Sugai, G., Smolkowski, K., Todd, A.,

Nakasato, J., & Esperanza, J. (2009). A randomized

control trial of school-wide positive behavior support in

elementary schools. Journal of Positive Behavior Inter-

ventions, 11{3), 1 1 3 - 1 4 4 . h t t p : / / d x . d o i . o r g / 1 0 .

1177%2F1098300709332067

Horner, R. H., & Sugai, G. (1999). Developing posi- tive behavioral support systems. In G. Sugai & T. J. Lewis (Eds.), Developing positive behavioral support for students with challenging behaviors [Monograph]. Gouncil for Children with Behavioral Disorders. Reston, VA.

Horner, R. H., Todd, A. W , Lewis-Palmer, T , Irvin, L. K., Sugai, G., & Boland, J. B. (2004). The school-wide evaluation tool (SET): A research instrument for assess- ing school-wide positive behavior support. Journal of Positive Behavior Interventions, 6{\), 3—12. http:// dx.doi.org/10.1177%2F 10983007040060010201

Hosmer, D. J. & Lemeshow, S. (1989). Applied Logistic Regression. New York, NY: John Wiley.

Hsu, J. (1996). Multiple comparisons procedures. Lon-

don, England: Cbapman and Hall.

Huberman, A. M. (1983). School improvement strate-

gies that work: Some scenarios. Educational Leadership,

41{3), 23-27.

Joyce, B., &C Sbowers, B. (2002). Student achievement through staff development (3rd ed.). Alexandria, VA: Association for Supervision and Curriculum Develop- ment.

Kam, C , Gteenberg, M. T , & Walls, C. T (2003). Ex- amining the role of implementation quality in school- based prevention using the PATHS curriculum. Prevention Science, 4(\), 55-63. http://dx.doi.org/10 .1023%2FA%3A1021786811186

Summer 2012

Kannapel, R J., & DeYoung, A. J. (1999). The rural school problem in 1999: A review and critique of the literature, fournal of Research in Rural Education, 15(2), 67-79.

Karachi, T W, Abbott, R. D., Catalano, R. E, Hag- gerty, K. P, & Fleming, C. B. (1999). Opening the black box: Using process evaluation measures to assess implementation and theory building. American fournal of Community Psychology, 27(5), 711-731. http:// dx.doi.org/10.1023%2FA%3A1022194005511

Lewis, T. J., & Sugai, C. (1999). Effective behavior support: A systems approach to proactive school-wide management. Eocus on Exceptional Children, 31(6), 1-24.

Martinez, M., & Harvey, J. (2004). Erom whole school reform to whole system reform. Washington, DC: National Clearing House for Comprehensive School Reform.

Mclntosh, K, Chard, D. J., Boland, J. B., & Horner, R. H. (2006). Demonstration of combined efforts in school-wide academic and behavioral systems and inci- dence of reading and behavior challenges in early ele- mentary grades, fournal of Positive Behavior Interventions, 8(3), 146-155.

Mclntosh, K , Horner, R., & Sugai, C. (2009). Sus- tainabiliry of systems-level evidence-based practices in schools: Current knowledge and future directions. In W. Sailor, G. Dunlap, G. Sugai, & R. Horner (Eds.) Handbook of Positive Behavior Support (pp. 327-352). New York, NY: Springer, http://dx.doi.org/10.1007 %2F978-0-387-09632-2_l 4

McLaughlin, M. W , & Mitra, D. (2001). Theory- based change and change-based theory: Going deeper, going broader, fournal of Educational Change, 2(4), 301-323. http://dx.doi.org/10.1023%2FA%3A1014 616908334

Mihalic, S., & Irwin, K. (2003). Blueprints for violence prevention: From research to real-world settings—Fac- tors influencing the successful replication of model pro- grams. Youth Violence and fuvenile fustice, 1(4), 307-329. http://dx.doi.org/10.1177%2F154l20400 3255841

Mihalic, S., Irwin, K, Fagan, A., Ballard, D., & Elliott, D. (July, 2004). Successful program implementation: Lessons from blueprints, fuvenile fustice Bulletin, 1-11.

Nelson, J., Hurley, K , Synhorst, L., & Epstein, M. (2008). The Nebraska three-tiered behavioral, preven- tion model case study. In C. Greenwood, T. Kra- tochwill and M. Clements (Eds.) Schoolwide prevention models (pp. 61-86). New York, NY: Guilford. Nelson, J. R., Martella, R., & Marchand-Martella, N. (2002). Maximizing student learning: The effects of a

comprehensive school-based program for preventing problem behaviors, fournal of Emotional and Behavioral Disorders, 10(3), 136-148.

Odom, S. L. (2009). The tie that binds: Evidence- based practice, implementation science, and outcornes for children. Topics in Early Childhood Special Educa- tion, 29(1), 53-61.

Ringeisen, H., Henderson, K., & Hoagwood, K. (2003). Context matters: Schools and the "research to practice gap" in children's mental health. School Psychol- ogy Review, 32(2), 153-168.

Rog, D., Boback, N., Barton-Villagrana, H., Marrone- Bennett, P, Cardwell, J., Hawdon, J., . . . Reischl, T. (2004). Sustaining collaboratives: A cross-site analysis of The National Funding Collaborative on Violence Prevention. Evaluation and Program Planning, 27, 249-261.

Rohrbach, L. A., Graham, J. W , & Hansen, W B. (1993). Diffusion of a school-based substance abuse prevention program: Predictors of program implemen- tation. Prevention Medicine, 22(2), 237-260. http:// dx.doi.org/10.1006%2Fpmed. 1993.1020

Safran, S. P, & Oswald, K. (2003). Positive behavior supports: Can schools reshape disciplinary practices? Exceptional Children, 69, 361-373.

SAS Institute. (2004). SAS/STAT User's Cuide, Version 9.1. SAS Institute, Cary, North Carolina, USA.

Schräg, J. A. (1996). Systems change leading to better integration of services for students with special needs. School Psychology Review, 25(4), 489-496.

Seal, K. R., & Harmon, H. L. (1995). Realities of rural school reform. Phi Delta Kappan, 77(2), 119-123.

Sindelar, P T, Shearer, D. K., Yendol-Hoppey, D., & Liebert, T. W. (2005). The sustainability of inclusive school reform. Exceptional Children, 72, 317-331.

Skiba, R. (2002). Special education and school disci- pline: A precarious balance. Behavioral Disorders, 27(2), 81-97.

Slavin, R. E., Holmes, G., Madden, N. A., Chamber- lain, A., & Cheung, A. (2010). Ejfects of a data-driven district-level reform model. Retrieved from http://vvww . b e s t e v i d e n c e . o r g / w o r d / d a t a _ d r i v e n _ r e f o r m _Mar_09_2010.pdf

Solomon, D., Battistich, V., Watson, M., Schaps, E., & Lewis, C. (2000). A six-district study of educational change: Direct and mediated effects of the child devel- opment project. Social Psychology of Education, 4(\), 3-51. http://dx.doi.org/10.1023%2FA%3A 1009609 606692

Sugai, G., & Horner, R. H. (2002). The evolution of discipline practices: School-wide positive behavior

Exceptional Children

Supports. Child Ó' Eamily Behavior Therapy, 24(112), 23-50. http://dx.doi.org/10.1300%2FJ019v24nOl_03

Sugai, G., Horner, R. H., Dunlap, G., Heineman, M., Lewis, T. J., Nelson, C , & Wilcox, B. (2000). Apply- ing positive behavioral support and functional behav- ioral assessment in schools. Journal of Positive Behavior Interventions, 2(3), 131-143.

Sugai, G., Horner, R. H., & Lewis-Palmer, T. (2001). Team implementation checklist. Eugene, OR: Educa- tional and Community Supports, University of Oregon.

Sugai, G., Lewis-Palmer, T., Todd, A., & Horner, R. H. (2001). School-wide Evaluation Tool (version 2.0). Eu- gene, OR: Educational and Community Supports, University of Oregon.

Theobald, P, & Nachtigal, P (1995). Culture, commu- nity and the promise of rural education. Phi Delta Kap- pan, 77(2), 132-135.

Weissberg, R. P, & Greenberg, M. T. (1998). School and community competence-enhancement and preven- tion programs. In 1. E. Siegel & K. A. Renninger (Eds.), Handbook of child psychology: Vol. 4. Child psy-

chology in practice (5th ed., pp. 877-954). New Yotk, NY: Wiley.

Weissberg, R. P, & Utne-O'Brien, M. (2004). Positive development: Realizing the potential of youth: What works in school-based social and emotional learning programs for positive youth development. The Ameri- can Academy of Political and Social Science, 86, 591-691. http://dx.doi.org/10.1177%2F000271620 3260093

Yin, R. K., & White, J. L. (1984). Federal technical assistance efforts: Lessons and improvement in education

for 1984 and beyond. Washington, DC: Cosmos.

A B O U T T H E A U T H O R S

JENNIFER H. COFFEY (Washington, DC CEC), Doctoral Student; and R O B E R T H .

HORNER (Oregon CEC), Professor, College of

Education, University of Oregon at Eugene.

Jennifer Coffey is now in the Office of Special

Education Programs at the U.S. Department of

Education, Washington, DC.

Address correspondence concerning this article

to Robert H. Horner, College of Education, 1235

University of Oregon, Eugene, OR 97402

([email protected]).

This manuscript was supported in part by a grant

from the Office of Special Education Programs,

U.S. Department of Education (H326S980003).

Opinions expressed herein are those of the au-

thors and do not necessarily reflect the position of

the U.S. Department of Education, and such en-

dorsements should not be inferred.

Manuscript received November 2010; manuscript

accepted June 2011.

4 2 2 Summer 2012

be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written

permission. However, users may print, download, or email articles for individual use.