Applied Sciences Week #2 Assignments

Readinson540W2alaetal.2021.Asystematicreviewoftier1PBISimplementationinhighschools.pdf

Home >Applied Sciences homework help >Applied Sciences Week #2 Assignments

https://doi.org/10.1177/1098300720929684

Journal of Positive Behavior Interventions 2021, Vol. 23(4) 288 –302 © Hammill Institute on Disabilities 2020 Article reuse guidelines: sagepub.com/journals-permissions DOI: 10.1177/1098300720929684 jpbi.sagepub.com

Literature Review

Positive Behavioral Interventions and Supports (PBIS) is a multitiered framework for delivering evidence-based behavioral supports aligned to students’ needs (Horner et al., 2010). Increasingly, high schools (HSs) are adopting PBIS to improve school climate, promote positive behav- iors, and decrease problem behaviors. In 2017, more than 300 HSs adopted PBIS, a three-fold increase from 2010 (Flannery et al., 2018). Although numerous literature reviews on PBIS have been conducted (e.g., Gage et al., 2015), HSs are consistently underrepresented across these reviews, thus preventing generalization of findings. For example, in a systematic review of PBIS and academic achievement, only three of 19 studies included HS partici- pants (Gage et al., 2015). Authors found limited to no improvements in academic achievement for HSs imple- menting PBIS. With so few HSs represented, implications for HS practices are limited.

Recently, Gage et al. (2018) conducted a meta-analysis to investigate the impact of PBIS across tiers on behavioral measures, including office discipline referrals (ODRs) and suspensions, while also evaluating studies on their adher- ence to What Works Clearinghouse (WWC, 2015) Group Design Standards. Authors reported significant reductions in suspension rates (g = −.86), but not ODRs. Of the four studies in the analysis, two implemented PBIS across all three tiers, two implemented only Tier 1, and only one

included HSs implementing Tier 1. Two of the four studies, including the HS study, did not meet WWC standards (e.g., Flannery et al., 2014). Although evidence indicated PBIS was an effective practice to reduce suspensions, how school characteristics (e.g., school size and grade level) impacted suspensions or ODRs was not addressed due to a small sample size.

In another review of 55 studies on Tier 1 PBIS, only seven studies included HS participants (Noltemeyer et al., 2019). Authors reported positive effects on ODRs (e.g., 91% of studies) and suspensions (e.g., 75% of studies), but less promising results on academic achievement (e.g., 56.4% of studies showed no significant effects). Although this review covered multiple variables related to PBIS and included HS participants, the authors described a need to study PBIS in a variety of educational settings (e.g., HSs). They stated, “This is particularly important when consider- ing the developmental changes occurring in later levels of compulsory education . . . changes that produce motivations

929684 PBIXXX10.1177/1098300720929684Journal of Positive Behavior InterventionsEstrapala et al. research-article2020

1The University of Iowa, USA

Corresponding Author: Sara Estrapala, Department of Teaching and Learning, The University of Iowa, N259 Lindquist Center, Iowa City, IA 52242, USA. Email: [email protected]

Action Editor: Jennifer Freeman

A Systematic Review of Tier 1 PBIS Implementation in High Schools

Sara Estrapala, PhD1, Ashley Rila, PhD1, and Allison Leigh Bruhn, PhD1

Abstract An emerging body of research shows Tier 1 Positive Behavioral Interventions and Supports (PBIS) can be successfully implemented in high schools to improve school climate and graduation rates and reduce problem behaviors. However, high schools are often hesitant to adopt PBIS because of contextual barriers such as school size, organizational culture, and student developmental level. Resistance to high school implementation is also related to teachers perceiving PBIS as less socially valid for high school students. Although previous systematic reviews of Tier 1 have examined implementation and effects, none have exclusively focused on the unique contextual needs related to high school implementation. In this review, we synthesized 16 published research studies conducted at the high school level, described how authors addressed the unique challenges of implementing PBIS in high schools, reported findings related to academic and behavioral outcomes, and made recommendations for future research and practice based on our findings.

Keywords PBIS, high school, behavior, Tier 1, schoolwide

Estrapala et al. 289

and values that link performance with future career aspira- tions and attainment” (p. 6).

Evidence indicates PBIS is effective for improving behavior (Bradshaw et al., 2009; Horner et al., 2010) with emerging evidence for improving academics (Gage et al., 2015). However, most research on Tier 1 has been at ele- mentary and middle school levels (Noltemeyer et al., 2019). Thus, there is a need to examine HS implementation litera- ture, particularly given the differences associated with age and setting (Flannery et al., 2018). For example, in a study of PBIS in HSs, Flannery and colleagues note contextual fea- tures such as (a) school size, (b) organization and culture, and (c) developmental level of students can affect imple- mentation efforts at the HS level (Flannery et al., 2013).

Challenges to HS Implementation

Compared with elementary and middle schools, HSs are more complex in terms of size, administrative and faculty organization, and student development; there also are increasing academic and social demands placed on students (Flannery et al., 2013; Flannery & McGrath Kato, 2017). Further exacerbating these complexities is the perception that PBIS is less socially valid at the HS level compared with elementary and middle schools (Vancel et al., 2016).

School size. Generally, HSs have more students, teachers, staff, and administrators, and the building is larger than elementary and middle schools. This increase in size corre- sponds to increases in the number of classrooms, courses offered, and scope and variety of extracurricular activities. Moreover, many school districts have multiple neighbor- hood-based elementary schools that feed into a larger, cen- tralized HS, which further increases diversity among students in terms of socioeconomic status, race, ethnicity, and experiences. This increase in size and complexity results in diverse social, behavioral, and academic needs across students and staff, thus highlighting the need for Tier 1, or schoolwide, prevention that affects all students (Flannery et al., 2013; Flannery & McGrath Kato, 2017).

Organization and culture. A primary goal of Tier 1 is to cre- ate a unified culture and positive school climate across all settings within a school (Horner et al., 2010). This requires consistent, systematic communication and collaboration among all administration, faculty, and staff in schools. Given the departmentalization of content areas in HSs and expectations of high academic achievement, creating a unified school culture may be more difficult in HSs for sev- eral reasons (Flannery et al., 2009). First, administrative teams are larger with responsibilities split among different individuals (e.g., discipline, curriculum), thus increasing the need for effective communication and clearly defined roles within Tier 1. Furthermore, federal calls to improve

academic outcomes and decrease dropout rates places sub- stantial pressure on faculty and administrators (Swain- Bradway et al., 2015). Second, HS teachers are organized into separate departments (e.g., science, math, and English) that typically function independently, each creating their own micro-culture with varying social and behavioral expectations. On one hand, this allows departments auton- omy; however, variance across departments could be chal- lenging because some teachers may emphasize social and behavioral skills, whereas others might feel unprepared to teach these types of skills (Bohanon et al., 2006). Or they feel it is unnecessary (Flannery et al., 2009) and interferes with teaching academic content (Flannery & McGrath Kato, 2017). To overcome the size and independence of academic departments, Tier 1 leadership teams are tasked with facilitating communication and collaboration so that expectations are universal, while meeting varying needs across departments.

Developmental level. Student-level variables such as age and development also create a challenge for PBIS imple- mentation in HSs. Adolescents find greater social, emo- tional, and behavioral motivation from their peers than adults as they enter HS (Romer et al., 2017), and schools typically shift from adult-provided supports in elementary and middle school to independent and peer-driven supports (Bohanon et al., 2006; Flannery & McGrath Kato, 2017). With increased independence and peer influence, this shift can be leveraged by involving students throughout the PBIS planning and implementation processes (Flannery & McGrath Kato, 2017). Furthermore, as HS students near the end of their education, they face increased pressures to demonstrate college and career readiness (CCR) skills (e.g., academic engagement, academic mindsets, learning pro- cesses, critical thinking, social skills, and transition knowl- edge; Morningstar et al., 2017).

Social validity and treatment fidelity. A final challenge for HSs is related to social validity and treatment fidelity of PBIS. Selecting and implementing socially valid inter- ventions is essential for successful outcomes (Marchant et al., 2012; Vancel et al., 2016) and sustainability (Coffee & Horner, 2012). Social validity refers to acceptability, significance, and appropriateness of intervention goals, procedures, and effects (Gresham & Lopez, 1996; March- ant et al., 2012). Experts suggest collecting and analyzing social fidelity data before, during, and after implementation from a variety of direct (e.g., students) and indirect (e.g., teachers and administrators) stakeholders, others involved in the school community (e.g., bus drivers, school volun- teers), and extended community members (e.g., taxpaying community members, school board members; Marchant et al., 2012). This approach provides essential information for (a) targeting professional development (PD) efforts; (b)

290 Journal of Positive Behavior Interventions 23(4)

identifying staff, parent, student, and community needs; and (c) developing program goals for sustained implementation.

Furthermore, the relation between social validity and treatment fidelity of PBIS has been documented as positive perceptions are correlated with better fidelity (Lane et al., 2009). In a study comparing elementary, middle, and HS teachers’ perceptions of PBIS, researchers found HS teach- ers rated PBIS as significantly less socially valid than ele- mentary and middle school teachers (p ≤ .0001; Vancel et al., 2016). Unfortunately, when buy-in is low, factors such as dissenting staff members, unproductive meetings, and inconsistent enforcement of policies and procedures negatively affect fidelity and outcomes (Flannery et al., 2009). Thus, determining the extent to which Tier 1 PBIS in HSs is socially acceptable may help inform future research and practice, particularly as it relates to buy-in and sustain- ability (Lane et al., 2009).

Due to the increase in HSs adopting PBIS, the call for empirical studies documenting the effects and implementa- tion efforts of PBIS in HSs and their unique contextual vari- ables, we conducted a systematic literature review of Tier 1 PBIS in HSs. The purpose of this literature review was to determine the nature of PBIS research conducted in HSs, describe how authors addressed implementation, and report findings related to student outcomes. Specifically, our research questions were as follows: (a) What were method- ological characteristics of the study (e.g., study design, purpose, and participant demographics)?; (b) What Tier 1 components (e.g., schoolwide expectations, procedures for teaching schoolwide expectations, acknowledgment sys- tem, behavior response system, stakeholder involvement) were reported?; (c) What strategies, methods, or deliberate use of practices were used to fit the HS context?; (d) To what extent did researchers collect and report treatment fidelity and social validity data?; and (e) What were the reported student behavioral and academic outcomes?

Method

Article Selection Procedures

Electronic search. We conducted a Boolean search, with no minimum year, through December 2018 in electronic data- bases ERIC (Education Resources Information Center) and PsycInfo. We used combinations and derivations of the following search terms: (a) HS or secondary; (b) positive behavioral interventions and supports, positive behavioral support, positive behavioral intervention, or multitiered systems of support; and (c) Tier 1, universal, or schoolwide. To be considered for inclusion, articles had to be peer- reviewed and published in English; unpublished works or dissertations were not included. Two doctoral students (i.e., authors one and two [S.E. and A.R.]) independently com- pleted the search, which yielded 172 articles.

Next, we read titles and abstracts of each article for inclu- sion. We looked for information indicating (a) implementa- tion of Tier 1 PBIS, (b) data reported at the HS level, and (c) student outcome data were reported. In addition, if the title and abstract indicated exclusive focus on a specific Tier 2 strategy (e.g., check-in/check-out) used within a PBIS system (e.g., McDaniel & Bruhn, 2016), the article was excluded. Of the 172 articles generated in the electronic search, and after removing duplicates, we retained 41 for further screening.

Finally, we searched the National TA Center on PBSI website (OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports, 2017) in the fall of 2018 for articles. We reviewed titles and abstracts of arti- cles under the research, Tier 1 supports, and HS PBIS pub- lications tab. This yielded 11 articles and one was retained for further screening. Finally, we hand searched three jour- nals that yielded two or more articles from the electronic search (e.g., Preventing School Failure, Journal of Positive Behavior Interventions, and Journal of Emotional and Behavioral Disorders). This search yielded 34 additional articles for consideration, and four were retained. Initial reliability for this phase was 91%, and all discrepancies were discussed until we were in 100% agreement for all articles retained for further screening. In sum, 46 articles moved to the next phase of screening.

Inclusion and exclusion criteria. We read each retained article to determine if it met inclusion criteria. First, authors had to describe Tier 1 components (e.g., schoolwide expectations, acknowledgment and behavior response systems, and PBIS team composition; Bohanon et al., 2006), or include fidelity data related to Tier 1 implementation (e.g., Barrett et al., 2008). All practitioner-oriented articles were excluded (e.g., Swain-Bradway et al., 2015). Second, articles needed to include behavioral or academic student outcome data (e.g., ODR, suspensions, state assessments, and grade point average [GPA]). Finally, if articles included elementary or middle schools (in addition to HSs), data had to be disag- gregated at the HS level (e.g., Grades 9–12). For example, some articles described secondary schools (e.g., Grades 7–12); however, they were excluded as HS and middle school data were grouped together (e.g., Tobin, 2008).

In sum, we retrieved 217 articles across all searches (e.g., 172 articles from ERIC and PsycInfo, 11 from PBIS. org, and 34 from hand searches), and 46 were read in their entirety for inclusion and exclusion criteria. Sixteen articles met inclusion criteria and moved to coding, and all authors were in 100% agreement on the included articles.

Coding Procedures

Two doctoral students (i.e., the first and second authors) independently coded 100% of the 16 articles. They coded each article across seven categories (e.g., methodologi- cal characteristics, Tier 1 components, HS adaptations,

Estrapala et al. 291

treatment fidelity, social validity, student behavior outcomes, and student academic outcomes), which are detailed in Table 1. Definitions of categories are available upon request. Prior to independent coding, the authorship team operationally defined coding categories, with each category followed by guiding questions for further detail. In total, all articles were guided by 31 questions (see Table 1). Coders trained on three articles using the coding system until they were 100% reliable. However, after coding additional articles indepen- dently, all authors met to discuss coding and realized the vast differences in how articles reported components. Thus, authors elected to refine definitions related to school size, culture, and student developmental level. Once new defini- tions were created, the first and second authors recoded all 16 articles. This coding agreement was 98% (e.g., 645 agree- ments out of 656 total codes), and we discussed all discrep- ancies until 100% agreement was met.

Results

Methodological Characteristics

Study design. Most studies reported nonexperimental designs (see Table 2). Descriptive designs were used in seven studies (Barrett et al., 2008; Bohanon et al., 2012; Lane et al., 2007; Malloy et al., 2018; McCrary et al., 2012; Muscott et al., 2008; Vincent & Tobin, 2011). One study used a mixed methods design (Bohanon et al., 2006). Three studies used a quasi-experiemental design (Bohanon & Wu, 2014; Freeman et al., 2015, 2016) One study was a random- ized control trial (Bradshaw et al., 2015) though only data from intervention schools were used in analysis. Two used longitudinal methods (Childs et al., 2010, 2016). One study used a pre–post comparison cluster trial (Flannery et al., 2014). Finally, one study did not report research design (Smolkowski et al., 2016).

Table 1. Article Coding Categories and Coding Questions.

Category Coding questions

Methodological characteristics • Was the study experimental? • What was the research design of the study? • What was the purpose of the study? • What level was the sample (e.g., national, state, district, school, and grade)? • How many high schools were reported in the sample? • Total student population and grade level breakdown

Tier 1 components • Did the article state schoolwide expectations were defined? • List schoolwide expectations • Did the article state schoolwide expectations were taught? • Did the article state an on-going acknowledgment system? • Describe the on-going acknowledgment system. • Did the article state a behavior response system? • Describe the behavior response system. • Did the article state a system for monitoring data? • Were students involved in planning or implementation? • Were parents involved in planning or implementation? • Were community members involved in planning or implementation?

High school adaptations • Did the article specifically state strategies, methods, or practices to fit the high school context? • Describe any high school adaptations. • Was CCR language incorporated in the Tier 1 plan?

Treatment fidelity • Were treatment fidelity data collected? • What treatment fidelity measurement tool was used? • What were treatment fidelity outcomes?

Social validity • Were social validity data reported? • What were social validity outcomes?

Student behavior outcomes • Were student behavior outcome data reported? • What type of behavior data were reported? • What were the behavioral outcomes?

Student academic outcomes • Were student academic outcome data reported? • What type of academic data were reported? • What were the academic outcomes?

Note. CCR = college and career readiness.

292

T ab

le 2

. St

ud y

D es

ig n,

S am

pl e,

F id

el ity

, S oc

ia l V

al id

ity , B

eh av

io r,

a nd

A ca

de m

ic O

ut co

m es

A rt

ic le

D es

ig n

N o.

o f H

S En

ro llm

en t

T re

at m

en t

fid el

ity So

ci al

v al

id ity

Be ha

vi or

o ut

co m

es A

ca de

m ic

o ut

co m

Ba rr

et t

et a

l. (2

00 8)

D es

cr ip

tiv e

52 N

R N

R b

y H

S le

ve l

N R

37 %

fe w

er O

D R

s fo

r tr

ea tm

en t

sc ho

ol s;

w er

O D

R s

co m

pa re

d w

ith n

at io

na l

da ta

ba se

N R

Bo ha

no n

et a

l. (2

00 6)

M ix

ed -

m et

ho ds

c as

e st

ud y

1 1,

80 0

SE T

: m et

c ri

te ri

on fo

r fiv

e of

ve n

ca te

go ri

es N

R A

ve ra

ge 2

0% r

ed uc

tio n

in O

D R

s fr

om Y

ea r

2 to

3 ; s

ig ni

fic an

t* r

ed uc

tio ns

in s

tu de

nt s

re ce

iv in

g O

D R

N R

Bo ha

no n

et a

l. (2

01 2)

D es

cr ip

tiv e

1 1,

73 8

EB S:

s ta

ff pe

rc ei

ve d

im pl

em en

ta tio

n in

cr ea

se d

SE T

: f ul

l c ri

te ri

on b

y Y

ea r

N R

53 %

r ed

uc tio

n in

O D

R s

fr om

Y ea

r 1

to 3

; si

gn ifi

ca nt

* O

D R

r ed

uc tio

ns w

ith b

oo st

se ss

io ns

a nd

P D

N R

Bo ha

no n

W u

(2 01

4) Q

ua si

- ex

pe ri

m en

ta l

2 T

2 C

T 1:

1 ,7

22 T

2: 2

,3 42

C 1:

3 ,3

77 C

2: 1

,5 87

SE T

Y ea

r 1:

T 1:

9 2%

; T 2:

6 1%

C 1:

5 3%

; C 2:

7 8%

SE T

Y ea

r 2:

T 1:

9 9%

; T 2:

7 4%

C 1:

3 6%

; C 2:

5 6%

N R

T 1

an d

T 2:

3 9%

O D

R r

ed uc

tio n;

C 1

an d

C 2:

% O

D R

in cr

ea se

; r ed

uc tio

ns in

a dj

us te

d m

on th

ly O

D R

m ea

N R

Br ad

sh aw

e t

al . (

20 15

) R

C T

a 31

T M

= 1

,3 31

(S D

48 8.

SE T

: Ba

se lin

e =

6 0.

59 Y

ea r

1 =

7 0.

63 Y

ea r

2 =

8 2.

5 (2

2 sc

ho ol

s m

et c

ri te

ri on

)

N R

Bu lly

in g

si gn

ifi ca

nt ly

* as

so ci

at ed

w ith

S ET

or es

; h ig

he r

ra te

s of

v er

ba l a

nd p

hy si

ca l

vi ct

im iz

at io

n as

so ci

at ed

w ith

h ig

he r

SE T

or es

; h ig

he r

ba se

lin e

pr ev

al en

ce o

f ph

ys ic

al v

ic tim

iz at

io n

as so

ci at

ed *

w ith

gh er

S ET

s co

re b

y Y

ea r

2; h

ig he

r ba

se lin

e ve

rb al

v ic

tim iz

at io

n as

so ci

at ed

w ith

gn ifi

ca nt

ly *

hi gh

er S

ET s

co re

s ov

er t

im e

N R

C hi

ld s

et a

l. (2

01 6)

Lo ng

itu di

na l

15 0

N R

BO Q

= 6

9. 77

N R

R ed

uc ed

O D

R , I

SS , a

nd O

SS ; i

m m

ed ia

te a

su st

ai ne

d dr

op in

d is

ci pl

in e

in ci

de nt

s w

ith

hi gh

er fi

de lit

N R

C hi

ld s

et a

l. (2

01 0)

Lo ng

itu di

na l

3, 5

, o r

9b N

R BO

Q : M

= 6

6% (

SD =

7– 19

.7 )

ac ro

ss 3

y ea

rs N

R b

y H

S le

ve l

O D

R : 3

3% d

ec re

as e

(n in

e sc

ho ol

s) ; I

SS :

4% d

ec re

as e

(t hr

ee s

ch oo

ls );

O SS

: 2 8%

cr ea

se (

fiv e

sc ho

ol s)

1% in

cr ea

se in

r ea

di ng

or es

o n

st at

e as

se ss

m en

t Fl

an ne

ry e

t al

. (2

01 4)

Pr e-

po st

C C

T 8

T 4

C M

= 1

,8 86

(S D

77 1.

69 )

SE T

: N R

N R

Si gn

ifi ca

nt *

re du

ct io

n in

O D

R s

w he

n co

nt ro

lli ng

fo r

en ro

llm en

t an

d %

F R

as fi

de lit

y in

cr ea

se d,

O D

R s

si gn

ifi ca

nt ly

* de

cr ea

se d

N R

Fr ee

m an

e t

al .

(2 01

5) Q

ua si

- ex

pe ri

m en

ta l

88 3

M =

1 ,0

80 BO

Q : N

R SE

T : N

R N

R T

ie r

1 as

so ci

at ed

w ith

a tt

en da

nc e

in cr

ea se

; m

ar gi

na lly

s ig

ni fic

an t*

d ro

po ut

r ed

uc tio

w ith

in cr

ea se

d fid

el ity

N o

si gn

ifi ca

nt o

ut co

m es

s ta

te a

ss es

sm en

(c on

tin ue

293

A rt

ic le

D es

ig n

N o.

o f H

S En

ro llm

en t

T re

at m

en t

fid el

ity So

ci al

v al

id ity

Be ha

vi or

o ut

co m

es A

ca de

m ic

o ut

co m

Fr ee

m an

e t

al .

(2 01

6) Q

ua si

- ex

pe ri

m en

ta l

88 3

M =

1 ,0

80 BO

Q a

nd S

ET c :

nu m

be rs

s ch

oo l i

m pl

em en

tin g

in cr

ea se

d ov

er t

im e:

20 05

8 02

N I,

43 P

I, an

d 29

t o

43 2

N I,

21 6

PI , a

21 7

FI in

2 01

N R

Sc ho

ol s

pa rt

ia lly

o r

fu lly

im pl

em en

tin g

w ith

fi de

lit y

ha d

si gn

ifi ca

nt ly

* hi

gh er

te nd

an ce

r at

es ; s

ch oo

ls p

ar tia

lly o

r fu

lly

im pl

em en

tin g

w ith

fi de

lit y

ha d

si gn

ifi ca

nt ly

w er

O D

R r

at es

; s ch

oo ls

w ith

h ig

he r

st ar

tin g

O D

R r

at es

h ad

s ig

ni fic

an tly

* gr

ea te

r de

cl in

e in

O D

R r

at es

o ve

r tim

N o

si gn

ifi ca

nt o

ut co

m es

s ta

te a

ss es

sm en

ts ;

sc ho

ol s

w ith

fu ll

fid el

ity h

ad in

cr ea

si ng

ad em

ic a

ch ie

ve m

en t

sc or

La ne

e t

al .

(2 00

7) D

es cr

ip tiv

e 2

17 8

Po si

tiv e

Be ha

vi or

S up

po rt

an : P

ri m

ar y

Le ve

l— D

is ci

pl in

e, P

os iti

Be ha

vi or

S up

po rt

P la

Pr im

ar y

Le ve

l— So

ci al

S ki

lls

= M

od er

at e

to h

ig h

fid el

ity

PI R

S U

ne xc

us ed

t ar

di es

: d ec

re as

es fo

r m

os t

st ud

en ts

; s us

pe ns

io ns

: d ec

re as

es fo

r hi

gh in

ci de

nc e

di sa

bi lit

ie s,

in te

rn al

iz in

co m

or bi

d an

d ty

pi ca

l, sl

ig ht

in cr

ea se

r ex

te rn

al iz

in g;

d is

ci pl

in ar

y co

nt ac

t: ty

pi ca

l s tu

de nt

s sh

ow ed

s lig

ht d

ec re

as e;

te rn

al iz

in g

sh ow

ed m

od er

at e

in cr

ea se

H ig

he r

G PA

s fo

r ty

pi ca

l be

ha vi

or ; h

ig he

r G

PA s

fo r

in te

rn al

iz in

g be

ha vi

or t

ha n

st ud

en ts

ith h

ig h-

in ci

de nc

e di

sa bi

lit ie

M al

lo y

et a

l. (2

01 8)

D es

cr ip

tiv e

1 57

0 to

6 10

SE T

: In

cr ea

se in

im pl

em en

ta tio

n fr

om 3

6% t

o 93

N R

D ec

re as

e in

r at

es o

f d ro

po ut

s, O

D R

s, a

O SS

; i nc

re as

e in

IS S

N R

M cC

ra ry

e t

al .

(2 01

2) D

es cr

ip tiv

e 1

16 6

SE T

: N R

N R

Fa ilu

re r

at es

d ec

re as

e 71

% M

us co

tt e

t al

. (2

00 8)

D es

cr ip

tiv e

2 H

S 1:

1 ,5

41 H

S 2:

4 72

SE T

Y ea

r 1:

H S

1 an

d 2 =

lo w

c ri

te ri

on SE

T Y

ea r

2: H

S 1 =

B el

C ri

te ri

on ; H

S 2 =

M et

ri te

ri on

N R

O D

R s

re du

ce d

33 %

( H

S 1:

R ed

uc ed

.7 1;

H S

2: R

ed uc

ed 1

4. 69

); IS

S re

du ce

d 97

% ; O

re du

ce d

14 %

St at

e as

se ss

m en

ts :

in cr

ea se

in s

tu de

nt s

m ee

tin g

st an

da rd

H S

Sm ol

ko w

sk i

et a

l. (2

01 6)

N R

8 N

R SC

S Fo

un da

tio ns

S ur

ve y:

cr ea

se s

in im

pl em

en ta

tio n

N R

St af

f p er

ce pt

io n

of b

ul ly

in g

an d

di sr

es pe

ct fu

ln es

s de

cr ea

se d*

; s us

pe ns

io ns

du ce

N R

V in

ce nt

T ob

(2 01

D es

cr ip

tiv e

7 M

= 1

,6 50

D =

5 33

) EB

S: in

cr ea

se s

im pl

em en

ta tio

n N

R R

ed uc

ed *

O SS

r at

es fr

om t

hr ee

h ig

h sc

ho ol

s; n

on -s

ig ni

fic an

t in

cr ea

se s

in O

ra te

s co

-o cc

ur re

d w

ith in

cr ea

se s

in E

sc or

es ; n

on cl

as sr

oo m

s et

tin gs

s ig

ni fic

an tly

* as

so ci

at ed

w ith

r ed

uc ed

O SS

w ith

as sr

oo m

s et

tin gs

s ig

ni fic

an tly

* as

so ci

at ed

ith in

cr ea

se O

N R

N ot

e. H

S =

h ig

h sc

ho ol

; F R

L =

fr ee

a nd

r ed

uc ed

-c os

t lu

nc h;

N R

= n

ot r

ep or

te d;

T =

t re

at m

en t;

C =

c on

tr ol

; R C

T =

r an

do m

iz ed

c on

tr ol

t ri

al ; C

C T

= c

om pa

ri so

n cl

us te

r tr

ia l;

PI R

S =

P ri

m ar

y In

te rv

en tio

n R

at in

g Sc

al e;

O D

R =

o ffi

ce d

is ci

pl in

e re

fe rr

al ; I

SS =

in -s

ch oo

l s us

pe ns

io n;

O SS

= o

ut -o

f- sc

ho ol

s us

pe ns

io n;

S ET

= S

ch oo

l-W id

e Ev

al ua

tio n

T oo

l; SE

T fu

ll fid

el ity

im pl

em en

ta tio

n cr

ite ri

on =

8 0%

o r

ab ov

e; B

O Q

= B

en ch

m ar

ks o

f Q ua

lit y;

B O

Q fu

ll fid

el ity

im pl

em en

ta tio

n cr

ite ri

a =

7 0%

o r

ab ov

e; E

BS =

E ffe

ct iv

e Be

ha vi

or S

up po

rt S

el f-

A ss

es sm

en t

Su rv

ey ; S

C S =

S af

e an

d C

iv il

Sc ho

ol s;

P D

= p

ro fe

ss io

na l d

ev el

op m

en t.

*S ta

tis tic

al ly

s ig

ni fic

an t

re su

lt. a C

on tr

ol p

ar tic

ip an

t da

ta w

er e

no t

pr ov

id ed

in s

tu dy

, n um

be rs

r ef

le ct

t re

at m

en t

sc ho

ol s.

b O ut

co m

e da

ta in

cl ud

ed d

iff er

en t

nu m

be rs

( i.e

., th

re e,

fi ve

, o r

ni ne

) of

h ig

h sc

ho ol

s fo

r di

ffe re

nt m

ea su

re s.

c R es

ea rc

he r

de ve

lo pe

d sc

he m

e to

d et

er m

in e

le ve

l o f i

m pl

em en

ta tio

n fid

el ity

; N I =

n o

im pl

em en

ta tio

n; P

I = p

ar tia

l i m

pl em

en ta

tio n;

F I =

fu ll

im pl

em en

ta tio

T ab

le 2

. (c

on ti

nu ed

)

294 Journal of Positive Behavior Interventions 23(4)

Sample. Across studies, samples included 1,164 total HSs serving over 1 million students. Two studies used the same data set; therefore, we counted their HSs once (Freeman et al., 2015, 2016). Four studies did not report student enrollment numbers (Barrett et al., 2008; Childs et al., 2010, 2016; Smolkowski et al., 2016). Studies including more than four HSs reported mean enrollment (Bradshaw et al., 2015; Flannery et al., 2014; Freeman et al., 2015, 2016; Vincent & Tobin, 2011). Studies with four or fewer HSs reported actual enrollment (Bohanon et al., 2006, 2012; Bohanon & Wu, 2014; Lane et al., 2007; Malloy et al., 2018; McCrary et al., 2012; Muscott et al., 2008).

Tier 1 Components

Schoolwide expectations. Of the 16 articles, seven authors explicitly stated the HSs had established schoolwide expec- tations and taught them to the student body within their Tier 1 plan (Bohanon et al., 2006, 2012; Bohanon & Wu, 2014; Childs et al., 2016; Lane et al., 2007; Malloy et al., 2018; Muscott et al., 2008). Of these seven, one study reported their expectations (Bohanon et al., 2006; for example, Be Respectful, Be Responsible, Be Academically Engaged, and Be Caring). Another study reported five schoolwide expectations, though the specific expectations were not pro- vided (Lane et al., 2007). The other nine articles reported fidelity scores using measures that assess the presence of schoolwide expectations (e.g., School-Wide Evaluation Tool [SET], Benchmarks of Quality [BOQ]). However, item-level scores were not reported and thus, without authors explicitly stating, we cannot report whether school- wide expectations were established and taught.

Acknowledgment system. Seven studies reported using an on-going acknowledgment system to reward students who met behavioral expectations (Bohanon et al., 2006, 2012; Bohanon & Wu, 2014; Childs et al., 2010; Lane et al., 2007; Malloy et al., 2018; McCrary et al., 2012). Five of these seven studies described their systems, which often included schoolwide prize drawings. Prizes included books, bags, computer software, t-shirts (Bohanon et al., 2006), preferred parking spaces, school sports passes, school dance packages, and food certificates (Lane et al., 2007). Drawings occurred weekly (Bohanon et al., 2006; Lane et al., 2007; Malloy et al., 2018), monthly (Bohanon et al., 2012), or randomly (McCrary et al., 2012). Three studies stated students could earn various items for meeting schoolwide expectations. Items included a pencil when students’ names were placed in schoolwide drawings (McCrary et al., 2012); US $0.25 vouchers to the school’s cantina, books, bags, computer soft- ware, and t-shirts (Bohanon et al., 2006); snacks, school spirit items, and school supplies (Bohanon et al., 2012).

Furthermore, one study reported schoolwide celebrations (e.g., schoolwide dance, movie tickets for all students and

staff) for meeting goals related to reducing ODRs (Bohanon et al., 2006). Another study reported HSs recognizing stu- dents for making the honor roll, hosting a thank you dinner for students and staff, and honoring student birthdays by giving students a card and pencil (Bohanon et al., 2012).

Behavior response system. Thirteen studies reported the HS had a system for collecting and responding to behavioral violations. Of these 13, all but 1 (Lane et al., 2007) relied on ODR data. In addition to using ODR data, five studies reported using a range of other data sources such as in- school suspension (ISS), out-of-school suspension (OSS), and expulsions (Childs et al., 2010, 2016; Lane et al., 2007; Smolkowski et al., 2016). Finally, Malloy et al. (2018) reported tracking data on ODRs, unexcused absences, fail- ure rates, tardies, nurse visits, and missing assignments.

Although most studies reported collecting and analyz- ing behavioral data to make decisions, few specifically described this process. Bohanon et al. (2006) stated that behavioral data (i.e., ODR and suspension) were reviewed and discussed by the PBIS leadership team monthly, whereas data were presented to the entire staff quarterly. Similarly, Flannery et al. (2014) reported SET and ODR data were reviewed quarterly. Barrett et al. (2008) reported utilizing a variety of data management programs such as Microsoft Access, Excel, and FrontPage to analyze patterns and trends in ODRs (i.e., behavior, location, time, and other individuals involved). Additional authors reported that behavioral data (i.e., ODR, ISS, and OSS) were entered into an online data entry system twice a year and reviewed by the school and district (Childs et al., 2010).

Stakeholder involvement. Six studies reported on the involve- ment of students in planning and implementation processes (Bohanon et al., 2006, 2012; Bradshaw et al., 2015; Flannery et al., 2014; Malloy et al., 2018; Muscott et al., 2008), and the degree of student involvement varied across implement- ing schools. For example, students served on the PBIS team (e.g., Bohanon et al., 2006; Muscott et al., 2008), aided in rollout activities to teach fellow students expectations (e.g., Malloy et al., 2018), or participated in interviews and sur- veys for Tier 1 development (e.g., Bohanon et al., 2006). Bohanon et al. (2006) reported students participated in development planning by presenting schoolwide data, working in small groups with faculty as recorders, and pro- viding student “reality checks.” Three studies reported on parental involvement (Bohanon et al., 2006; Lane et al., 2007; Muscott et al., 2008), which ranged from PBIS team membership to participation in surveys and interviews (e.g., Bohanon et al., 2006). Finally, three studies reported includ- ing community stakeholders in planning and implementa- tion (Bohanon et al., 2006; Bohanon & Wu, 2014; Muscott et al., 2008). Bohanon et al. (2006) stated community mem- bers were interviewed or surveyed for input in PBIS

Estrapala et al. 295

development, and a community member served on the PBIS team, though their role was not described. It was not clear how community stakeholders were utlized in the other two studies (Bohanon & Wu, 2014; Muscott et al., 2008).

HS Strategies, Practices, and Adaptations

Nine studies reported implementing contextually relevant strategies, practices, or adaptations to help ensure the Tier 1 PBIS plan was appropriate for HSs (Bohanon et al., 2006, 2012; Bohanon & Wu, 2014; Bradshaw et al., 2015; Childs et al., 2010; Flannery et al., 2014; Lane et al., 2007; Malloy et al., 2018; McCrary et al., 2012). We report these strate- gies as they relate to key contextual features: school size, organization/culture, and developmental level (Flannery et al., 2013).

School size. Strategies, practices, and adaptations for school size involved explicit descriptions of addressing (a) large enrollment numbers; (b) diverse teacher, staff, student, and community needs; and (c) the physical structure of the school (e.g., overcrowding). Strategies were implemented in two studies to account for large numbers of school mem- bers involved in PBIS development (Bohanon et al., 2010; Childs et al., 2010). In Bohanon et al. (2006), for example, researchers administered the Effective Behavior Support (EBS) survey to groups of 10 to 30 individuals and limited all school-wide PD presentations to 30 staff members to promote participation. Childs et al. (2010) stated trainers extended PBIS trainings to three consecutive days of train- ing to accommodate for larger teams.

Additional adaptations included researcher recommen- dations to modify surveys to better suit HSs. For example, Bohanon et al. (2006) modified the EBS focus more on urban and HS settings. Specifically, researchers added an “I Don’t Know” response when asking if a certain item of PBIS was in place and clarified the description of nonclass- room settings to be HS specific (e.g., hallway, cafeteria, public transit, parking lot, bathroom, and before and after school events). Finally, they added questions related to parent/guardian roles and responsibilities in developing expected behaviors. Bohanon and Wu (2014) also used sur- vey data to identify staff perceptions and needs related to school size. Participants identified the need to address “accommodations for students from diverse backgrounds” (p. 226), “varied work and extracurricular opportunities for students” (p. 226), and various concerns related to school climate in their Tier 1 plan. However, authors did not state exactly how these needs were addressed in planning and implementation efforts. Finally, one study implemented a summer pilot program to test the teaching and acknowledg- ment system on a small scale (i.e., 100 students) for feed- back and revision (Bohanon et al., 2006). Prior to the first year of full-scale implementation, the PBIS team, along

with teachers and students who participated in the summer pilot, presented the revised plan to staff members.

Organization and culture. Strategies, practices, and adapta- tions for organization and culture primarily involved explicit descriptions of addressing academic departments and communication within and across school personnel. Three studies (Bohanon & Wu, 2014; Childs et al., 2010; Flannery et al., 2014) utilized staff survey and pre-imple- mentation data to direct training and PD efforts. In another study, school staff identified inconsistencies across poli- cies, practices, procedures, and team effectiveness as spe- cific areas of need. Therefore, trainers focused on building capacity, communication, data-based decision-making, and action planning using “HS-specific examples of teach- ing, acknowledgement, and policies for handling student behavior” (Bohanon & Wu, 2014, p. 227). In Flannery et al. (2014), communication related to departmentaliza- tion was often cited by HS teams as a problem area, so adaptations included greater specification of team member roles, developing subcommittees (e.g., communication, acknowledgments, and data), clarifying existing policies, and developing systems for obtaining student, faculty, and staff feedback. In addition, staff acknowledgment proce- dures were included in the acknowledgment system. Finally, Childs et al. (2010) held frequent social skills groups for faculty and staff, although no additional infor- mation is provided.

Three studies included networking strategies between HSs implementing PBIS to learn, problem solve, and build connections. For example, school staff visited local HSs to observe and learn from their planning and implementation processes (Bohanon et al., 2012); staff attended a HS PBIS PD forum (Bohanon et al., 2012); and the research team created cross site forums for HSs involved in the study to share resources and problem solve (Flannery et al., 2014).

Developmental level. Strategies, practices, and adaptations for developmental level involved anything related to the age of students, developing student buy-in, or incorporating aca- demic and behavioral needs and preferences. In Bradshaw et al. (2015), participating HSs completed the MDS3 School Climate Survey (Bradshaw et al., 2014) to guide selection of practices for teaching expectations. Also, authors provided schools with a menu of options for addressing concerns indi- cated on MDS3 survey results, which included a program specifically designed for HSs to teach classroom social skills, self-management, and drug resistance skills (e.g., Botvin LifeSkills Training program; Botvin et al., 2006). In McCrary et al. (2012), administrators were concerned with school fail- ure rates, so they developed an after-school tutoring program to support any student who was not passing or had missing assignments. Childs et al. (2010) implemented frequent social skills instruction across all students, but did not

296 Journal of Positive Behavior Interventions 23(4)

provide the curriculum, dosage, or frequency of instruction. Finally, no studies incorporated CCR language.

Moreover, students were involved in planning and implementation in six studies (Bohanon et al., 2006, 2012; Bradshaw et al., 2015; Flannery et al., 2014; Malloy et al., 2018; Muscott et al., 2008); however, the degree of student involvement varied. For example, students served on PBIS teams (e.g., Bohanon et al., 2006; Muscott et al., 2008), aided in rollout activities to teach students expectations (e.g., Malloy et al., 2018), or participated in interviews and surveys for Tier 1 development (e.g., Bohanon et al., 2006). Bohannon et al. (2006) reported students participated by presenting schoolwide data, working in small groups with faculty as recorders, and providing the team with student “reality checks” (p.137) while developing schoolwide expectations. In Flannery et al. (2014), students served on special topic committees (e.g., bullying, use of appropriate language) and were included in decision-making. Although not explicit in their role, Childs et al. (2010) reported stu- dents were encouraged to participate “earlier” (p. 208) in PBIS teams.

Treatment fidelity. Although every study reported collecting treatment fidelity, two studies did not report scores disag- gregated at the HS level (Barrett et al., 2008; Flannery et al., 2014) and two studies did not report any fidelity scores (Freeman et al., 2014; McCrary et al., 2012). Nearly half of the studies reporting fidelity data either (a) did not report the fidelity scores (e.g., Freeman et al., 2015), (b) used mea- sures without established technical adequacy (e.g., Lane et al., 2007), or (c) did not include proficiency criteria (e.g., Lane et al., 2007; Vincent & Tobin, 2011). A variety of fidelity measures were used, including the SET Tool (Horner et al., 2004; for example, Bohanon et al., 2006), Effective Behavior Support Self-Assessment Survey (EBS; Lewis & Sugai, 1999; for example, Bohanon et al., 2012; Vincent & Tobin, 2011), BOQ (Cohen et al., 2007; for example, Childs et al., 2016), Team Implementation Check- list (Barrett et al., 2008), Coaches Checklist (Barrett et al., 2008), Implementation Phases Inventory (Barrett et al., 2008), Positive Behavior Support Plan: Primary Level— Discipline Plan and Positive Behavior Support Plan: Pri- mary Level—Social Skills (Lane et al., 2007), and Safe and Civil Schools Foundations Survey (e.g., Smolkowski et al., 2016).

Twelve studies reported fidelity outcomes; most reported increases in scores over time. In two studies, all schools met the full fidelity criterion (i.e., 80% or higher) on the SET (Bohanon et al., 2012; Malloy et al., 2018). Mixed fidelity results were found in three studies. For example, Bohanon et al. (2006) reported that after 3 years, the school met cri- terion on the SET for all but two areas (expectations taught and district level support). In studies with multiple schools in the sample, some schools met criterion while others did

not. For instance, Bohanon and Wu (2014) and Muscott et al. (2008) each had one of two HSs meeting fidelity; Bradshaw et al. (2015) reported 22 out of 31 HSs meeting criterion; and Freeman et al. (2016) reported 217 out of 865 HSs meeting full fidelity criterion. Finally, two studies reported HSs included fell below the 70% fidelity criterion on the BOQ (Childs et al., 2010, 2016).

Social Validity

Only two studies reported social validity information (Childs et al., 2010; Lane et al., 2007). One study used a web-based Attrition Survey (Childs et al., 2010) to glean information about why schools decided to stop implement- ing PBIS, and the School-Wide Implementation Factors survey to measure consumer satisfaction. Authors reported the major factors contributing to schools making the deci- sion to stop PBIS included high rates of teacher turnover, lack of time, and lack of administrative and staff commit- ment to the efforts. That said, this survey was administered state-wide with data not disaggregated by school level, thus it is unclear which factors can be attributed to HSs. Lane et al. (2007) used the Primary Intervention Rating Scale to measure social validity of Tier 1 prior to implementation. In this study, both participating schools rated Tier 1 a socially valid practice at the HS level.

Student Outcomes

Behavior. All studies measuring ODR rates reported improvements to varying degrees. One study examined when ODR reductions occurred and found a significant reduction following student booster lessons and PD activi- ties (Bohanon et al., 2012). Bohanon and Wu (2014) reported treatment schools (n = 2) decreased ODR rates by 39% while comparison schools (n = 2) reported a 10% increase. In two studies, authors found as fidelity increased, ODRs significantly decreased (Flannery et al., 2014; Free- man et al., 2016). In addition, after controlling for enroll- ment and percentage of students receiving free and reduced lunch, one study found a significant reduction in ODRs (Flannery et al., 2014). Freeman et al. (2016) found signifi- cant reductions in ODR rates over time when schools had higher initial ODR rates. Finally, Muscott et al. (2008) reported both HSs reduced ODR rates. One HS reduced their average ODR rate per day per 100 students by 14.49, while the other reduced by 0.71 (Muscott et al., 2008).

Five studies measuring ISS and OSS reported mixed results. Specifically, authors reported a reduction in ISS in three studies (Childs et al., 2010, 2016; Muscott et al., 2008) and reduction in OSS in three studies (Childs et al., 2016; Malloy et al., 2018; Muscott et al., 2008). Conversely, OSS increased in two studies (Childs et al., 2010; Vincent & Tobin, 2011), as did ISS in one study (Malloy et al., 2018).

Estrapala et al. 297

It is noteworthy that in two studies, there was a decrease in one measure (e.g., OSS) and a concurrent increase in the other measure (e.g., ISS; Childs et al., 2010; Malloy et al., 2018). Furthermore, Vincent and Tobin (2011) found increases in OSS (though statistically insignificant) occurred with increases in fidelity scores. Further analysis indicated PBIS implementation in nonclassroom settings was significantly associated with a reduction in OSS, whereas implementation in classroom settings was associ- ated with a significant increase in OSS. Although not disag- gregated by suspension type (OSS or ISS), Lane et al. (2007) found low to moderate decreases in suspension for students with high-incidence disabilities, comorbid inter- nalizing and externalizing behaviors, and students without disabilities or behavioral risk, yet a slight increase in sus- pensions for students with externalizing behaviors.

Authors reported a variety of other behavioral outcomes beyond ODR, ISS, and OSS. Interestingly, Bradshaw et al. (2015) found higher rates of bullying (i.e., verbal and physi- cal victimization) were associated with higher SET scores. Other studies reported improvements in attendance rates (Freeman et al., 2015, 2016), decreases in unexcused tar- dies (Lane et al., 2007), and improved fidelity for longer periods of time yielded significant improvements in drop- out rates (Freeman et al., 2016). Mixed results were found in disciplinary contacts (e.g., ODR, suspension, referral to alternative learning center or counseling services) in another study, with typically developing peers having a slight decrease in disciplinary contacts, and students with exter- nalizing behaviors showing a moderate increase in disci- plinary contacts (Lane et al., 2007). Finally, one study reported staff perceived decreases in bullying by 15% and student disrespect by 28% (Smolkowski et al., 2016).

Academic. Six studies reported academic outcome data with four studies reporting results from state assessment data (Childs et al., 2010; Freeman et al., 2015, 2016; Muscott et al., 2008), one reporting GPA (Lane et al., 2007) and another reporting failure rates (McCrary et al., 2012). Of the studies reporting state assessment data, three reported improvements. Specifically, Childs et al. (2010) reported a 1% increase in reading scores. Freeman et al. (2016) indi- cated that schools with high PBIS implementation fidelity had increasing trends in academic achievement. One of the two HSs in the Muscott et al. (2008) study found improve- ments on proficiency criterion for reading and math assess- ments. Two studies examined associations between state assessment data and dropout rates (Freeman et al., 2015) and fidelity scores (Freeman et al., 2016). Although there was no statistically significant relation between fidelity scores and dropout rates, schools with full PBIS implementation saw increased achievement scores (Freeman et al., 2016). Lane et al. (2007) found GPA was significantly higher for students with typical behavior when compared with students with

externalizing behavior, comorbid internalizing and external- izing behavior, and high incidence disabilities (e.g., specific learning disability, hearing/speech impairment). Also, stu- dents with internalizing behavior had significantly higher GPAs than students with high-incidence disabilities. Finally, one study reported a 71% reduction in failure rates after implementing a tutoring program (McCrary et al., 2012).

Discussion

As HSs continue to adopt Tier 1 PBIS, it is important to understand current research, practice, and limitations of the literature to drive future research and implementation efforts. Although several reviews on PBIS have been published (e.g., Gage et al., 2015, 2018; Noltemeyer et al., 2019), none have focused specifically on Tier 1 implementation in HSs. Thus, we systematically reviewed published research stud- ies on HS Tier 1 implementation by examining methodolog- ical variables, Tier 1 components, HS-specific adaptations, fidelity, social validity, behavior, and academic outcomes. Search and screening procedures yielded 16 articles included for review, which were coded and synthesized.

Prior to interpreting key findings, limitations of our systematic review procedures must be considered. First, although we searched multiple online databases for relevant articles, completed hand searches of relevant journals and the National PBIS TA Center website, it is possible some publications were missed. Second, our inclusion criteria of outcome data disaggregated by grade or school level (i.e., HS separated from middle school) eliminated several stud- ies in the screening process. Despite these limitations, find- ings of this review have important implications for future research and practice.

Key Findings, Limitations, and Recommendations for Research and Practice

Methodological characteristics. Of the 16 studies, only three were quasi-experimental and the rest were descriptive or longitudinal case studies. Due the small number of quasi- experimental studies, causal inference cannot be established, therefore limiting our understanding of effectiveness of PBIS in HSs. Furthermore, several of the included studies were not specifically targeting HS implementation, but rather, they were describing national, statewide, or district- wide K–12 PBIS efforts, and HS data were minimally included. Often, the small HS samples within the larger studies were insufficient for more sophisticated analyses; thus, they were often excluded (i.e., changes in ODRs based on race, disability, or gender; for example, Barrett et al., 2008; Muscott et al., 2008; Vincent & Tobin, 2011). To establish PBIS as an evidence-based practice for HSs and to maximize utility for practitioners, we recommend increases in research with efforts focusing on large-scale experimental

298 Journal of Positive Behavior Interventions 23(4)

studies at the HS level. Positively, the remaining 13 descrip- tive studies provide evidence that Tier 1 implementation is possible at the HS level.

HS adaptations. Experts cite school size, organizational culture, and developmental level as important HS contex- tual influences which must be addressed in Tier 1 planning, implementation, and assessment efforts (Flannery et al., 2013; Flannery & McGrath Kato, 2017). However, given the limited and varied reporting of Tier 1 components across included studies, it was difficult to determine how these contextual influences were addressed.

School size. Most often, authors reported segmenting PD meetings and schoolwide student meetings into smaller groups to accommodate large populations, and one study included a small pilot test of teaching and enforcing expec- tations. However, no authors using this adaptation provided social validity information, and it would be interesting to know if PBIS team members found this adaptation benefi- cial. Furthermore, a couple of articles utilized pre-imple- mentation surveys to better understand the school’s needs in terms of congested or less structured locations (e.g., hall- ways, cafeterias, and parking lots), and only one utilized surveys to support a diverse student population (Bohanon & Wu, 2014). Unfortunately, the results of these surveys and how they were incorporated into the Tier 1 plan were not discussed. We recommend HS teams consider survey- ing students, faculty, and staff needs to help guide Tier 1 development. This may be particularly important given the large number of stakeholders at the HS level. In future research, reporting details about how this information was used in planning and development efforts, as well the social validity of using survey results after implementation, may be helpful. Furthermore, although these ideas were not discussed in the included studies, other recommendations related to school size include hosting new student orienta- tions for incoming freshmen or students who attend after the start of the school year. These could be used to help teach students about the Tier 1 plan. Another idea is to have a designated time during the school day (e.g., 7th period) where all teachers in the building teach or review school- wide expectations.

Organization and culture. As with school size, a common method for addressing organization and culture included pre-implementation surveys, which allowed stakeholders to have a voice in the plan. Often, staff identified com- munication and departmentalization as an area of need, so team leaders established communication channels within departments and created venues for multiple HSs to dis- cuss and problem solve implementation efforts. Based on these findings, team leaders can first determine school needs related to Tier 1 (e.g., prior knowledge, experience,

concerns), conduct training to address those needs, and allow enough time to fully train all faculty, staff, and admin- istration in Tier 1 plans. Another suggestion is for school teams to provide ongoing PD to promote sustainability. During these sessions, PBIS teams can provide all faculty, staff, and administrators with relevant data (e.g., ODR, pos- itive acknowledgments) to demonstrate the impact of Tier 1 implementation efforts, as sharing data can have positive impacts on sustainability (McIntosh et al., 2013). Other potential ways to address organization and culture include (a) ensuring representation from each department on the Tier 1 team, (b) creating data systems that are accessible to all teachers at all times, (c) providing staff with a common time to address implementation needs, and (d) recognizing implementation efforts of adults similar to how students are acknowledged for positive behaviors.

Developmental level. The most common method for addressing the developmental age of students was to pro- vide a menu of options for age-appropriate reinforcers (e.g., prom tickets, parking passes) in addition to providing choices for teams to develop contextually relevant lesson plans. Three articles specifically reported including stu- dents as members of the PBIS team. In light of research suggesting that HS students desire autonomy from adult pressures and respond best to peer-driven social supports (Romer et al., 2017), future researchers and practitioners should consider ways to maximize student input. For instance, students could play a large role in developing universal supports, beyond preference and climate surveys, as active team members responsible for steering practices toward relevant needs. For example, student members could develop student-friendly definitions of universal expecta- tions, design and deliver lessons teaching expectations, or play active roles in recognizing peers following universal expectations both inside and outside school. Not only could this satisfy student need for autonomy but also reduce the amount of time PBIS teams spend on planning and prepar- ing all components of their Tier 1 plan.

Furthermore, experts in the field have suggested incor- porating CCR skills into Tier 1 plans as a means to unify faculty, staff, and administrators around a common mission and to support buy-in (Morningstar et al., 2017). That is, CCR is recommended by federal law to support a well- rounded education (e.g., Every Student Succeeds Act [ESSA], 2015) and many teachers find CCR highly rele- vant for HS students (Morningstar et al., 2017). As such, merging Tier 1 language with CCR skills may help improve efforts that address HS contextual variables (Freeman & Lombardi, 2018). Morningstar et al. (2017) offered spe- cific strategies to embed CCR skills and language (e.g., academic engagement, academic mind-sets, learning pro- cesses, critical thinking, social skills, and transition knowl- edge) into the PBIS framework; however, no included

Estrapala et al. 299

study explicitly included CCR. Future research could explore the extent to which incorporating ESSA and Morningstar et al. (2017) CCR recommendations into Tier 1 affects buy-in, sustainability, and outcomes.

Treatment fidelity. Reporting treatment fidelity scores is a recommended best practice for research studies, as fidelity is necessary for drawing accurate conclusions about inter- vention effects (Horner et al., 2005), and 10 articles reported fidelity scores and proficiency criteria using reliable and valid measures (e.g., SET, BOQ, and EBS). However, the variability of data reporting (e.g., Barrett et al., 2008) and technical adequacy of some measures used (e.g., Vincent & Tobin, 2011) indicate results of several included studies should be interpreted with caution. Interestingly, none of the 16 studies utilized the Tiered Fidelity Inventory (TFI; Algozzine et al., 2019), a reliable and valid tool for measur- ing fidelity endorsed by the National TA Center. Practitio- ners and researchers could consider using the TFI as it incorporates features of the SET, BOQ, and EBS, the mea- sure is free, and it can be used to facilitate implementation decision-making.

Of the fidelity scores reported, initial fidelity scores were often below criterion, with steady gains reported yearly. For example, Freeman et al. (2016) reported 29 schools meeting full implementation criteria based on the BOQ and SET in 2005–2006, and by 2011–2012, that num- ber increased to 217. All studies reporting fidelity score changes over time described increasing implementation trends, implying that, while initial implementation may be low, full fidelity is attainable within a couple of years of implementation. This is important information for practitio- ners, particularly as they embark on initial implementation. If HS administrators and faculty understand that effective implementation may take time, then they may be less likely to prematurely abandon potentially effective practices (Algozzine et al., 2019). Again, practitioners are encour- aged to provide faculty and staff with on-going data related to implementation efforts. Importantly, these data should include successes rather than only areas of improvement.

Furthermore, several studies examined effects of fidelity scores on student outcomes (Bradshaw et al., 2015; Childs et al., 2016; Freeman et al., 2015, 2016), or the relation between fidelity scores and student outcomes. Given the varying associations between fidelity of implementation and student outcomes, we argue that all schools implement- ing Tier 1 should carefully measure fidelity over time with a reliable and valid instrument (e.g., TFI). Another area for future research is exploring the relation between fidelity of implementation and social validity, as both could affect stu- dent outcomes (Lane et al., 2009).

Social validity. Only one study, Lane et al. (2007), directly measured and reported social validity data to assess

pre-implementation buy-in. However, no further assess- ments of social validity were reported once implementa- tion began; therefore, the social validity of the practices implemented in the study are unknown. Although Childs et al. (2010) reported several general reasons why schools discontinued PBIS, HS-specific reasons were not disaggre- gated from the state-wide data set. Therefore, barriers most relevant to HS implementation were not discussed.

The overall lack of measuring and reporting social validity across HS PBIS publications is concerning given (a) some research indicates HS teachers find PBIS inap- propriate at the HS level (Vancel et al., 2016) and (b) sev- eral included studies cited buy-in as a critical barrier to implementation (Bohanon et al., 2006; Bohanon & Wu, 2014; Bradshaw et al., 2015; Childs et al., 2010; Flannery et al., 2014; Lane et al., 2007; Malloy et al., 2018; McCrary et al., 2012). Whereas some articles addressed buy-in via pre-intervention assessments (Bohanon et al., 2006; Lane et al., 2007) or with intervention procedures (e.g., Bohanon et al., 2012; Flannery et al., 2014; dis- cussed further below), no study directly reassessed buy-in or social validity once PBIS was implemented. Second, no study reported surveying stakeholders outside of fac- ulty, staff, or other PBIS team members. Determining whether a program is socially valid extends beyond those implementing interventions (e.g., faculty, staff, adminis- trators), particularly when the program (e.g., PBIS) is intended to meet the needs of schools within larger com- munities (Marchant et al., 2012). Last, no study reported measuring student perceptions of Tier 1 planning and implementation efforts, the stakeholders most affected by PBIS efforts.

Research has shown a link between teachers’ perceptions of social validity and treatment fidelity (Lane et al., 2009), and social validity and sustainability (Coffee & Horner, 2012). Future researchers could consider measuring and reporting social validity through a variety of methods (e.g., rating scales, surveys, interviews), while also continuing to explore ways to make PBIS more acceptable for HSs, and in turn, how social validity is related to treatment fidelity and sustainability. The careful study of social validity could also help bridge the gap between research and practice, since stakeholder opinions and needs could guide the direction of future research. Relatedly, it is important to select reliable and valid measures of social validity, such as the Primary Intervention Rating Scale (PIRS; Lane et al., 2009). Whereas the PIRS has been validated for teacher use, fur- ther development of reliable and valid tools for other stake- holder use (e.g., students, parents, and community members) is warranted.

Behavioral and academic outcomes. We found reductions in ODRs across all studies, and a few studies reported increases in ISS, OSS, or rates of bullying. However, these results

300 Journal of Positive Behavior Interventions 23(4)

should be interpreted with caution given the lack of experi- mental studies in the corpus. As more HSs adopt PBIS, using experimental designs will allow for more definitive claims about effects to be made. Furthermore, nearly all studies used ODR data to demonstrate the behavioral effects of PBIS over time. Although ODRs may be a reliable and valid measure of behavior (Spaulding et al., 2010), additional types of behavioral data (e.g., ISS, OSS, school climate sur- veys, attendance) may allow researchers and practitioners to glean a more complete understanding of the effects of PBIS. Furthermore, behavioral data should be examined by loca- tion, behavior, time of day, grade level, student, and refer- ring teacher for a more detailed analysis (Clonan et al., 2007; Kim et al., 2018; Spaulding et al., 2010).

We were surprised to find that only six of the included studies reported academic data. One of the primary goals of PBIS is to improve academics in addition to behavior and school climate (Horner et al., 2010); however, the majority of research did not include academic data. Because schools are already required to track achievement data by federal and state governments (VanGronigen & Meyers, 2019), schools likely have a variety of data sources readily available (e.g., state assessments, GPAs, and failure rates). Researchers and practitioners could analyze these data for evidence of PBIS impact and report these data in future research.

Despite these limitations, Tier 1 PBIS implementation continues to emerge as a promising practice in HSs. Our findings from these 16 studies suggest that some simple strategies (e.g., greater student involvement in planning efforts, age-appropriate reinforcers, collaborating with other HSs) might improve buy-in, treatment fidelity, and student outcomes at the HS level. As researchers and prac- titioners move forward, documenting the effects of PBIS with specific attention to HS contextual variables might reveal important moderators for enhancing outcomes and advancing the field.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

*Indicates articles included in the review. Algozzine, B., Barrett, S., Eber, L., George, H., Horner, R., Lewis,

T., Putnam, B., Swain-Bradway, J., McIntosh, K., & Sugai, G. (2019). School-wide PBIS Tiered Fidelity Inventory. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. www.pbis.org

*Barrett, S. B., Bradshaw, C. P., & Lewis-Palmer, T. (2008). Maryland statewide PBIS initiative: Systems, evaluation, and next steps. Journal of Positive Behavior Interventions, 10, 105–114. https://doi.org/10.1177/1098300707312541

*Bohanon, H., Fenning, P., Carney, K. L., Minnis-Kim, M. J., Anderson-Harriss, S., Moroz, K. B., . . . Pigott, T. D. (2006). Schoolwide application of positive behavior support in an urban high school: A case study. Journal of Positive Behavior Interventions, 8, 131–145. https://doi.org/10.1177/10983007 060080030201

*Bohanon, H., Fenning, P., Hicks, K. J., Weber, S., Thier, K., Hoeper, L., & Irvin, L. (2012). A case example of the imple- mentation of schoolwide positive behavior support in a high school setting using change point test analysis. Preventing School Failure, 56, 91–103. https://doi.org/10.1080/10459 88X.2011.588973

*Bohanon, H., & Wu, M. J. (2014). Developing buy-in for posi- tive behavior support in secondary settings. Preventing School Failure, 58, 223–229. https://doi.org/10.1080/10459 88X.2013.798774

Botvin, G. J., Griffin, K. W., & Nichols, T. R. (2006). Preventing youth violence and delinquency through a universal school-based prevention approach. Prevention Science, 7, 403–408.

*Bradshaw, C. P., Koth, C. W., Thornton, L. A., & Leaf, P. J. (2009). Altering school climate through school-wide posi- tive behavioral interventions and supports: Findings from a group-randomized effectiveness trial. Prevention Science, 10, 100–115. https://doi.org/10.1007/s11121-008-0114-9

Bradshaw, C. P., Pas, E. T., Debnam, K. J., & Johnson, S. L. (2015). A focus on implementation of positive behavioral interven- tions and supports (PBIS) in high schools: Associations with bullying and other indicators of school disorder. School Psychology Review, 44, 480–498. https://doi.org/10.17105/ spr-15-0105.1

Bradshaw, C. P., Waasdorp, T. E., Debnam, K. J., & Lindstrom Johnson, S. (2014). Measuring school cli- mate: A focus on safety, engagement, and the environ- ment. Journal of School Health, 84, 593–604. https://doi.org/10.1111/josh.12186

*Childs, K. E., Kincaid, D., & George, H. P. (2010). A model for statewide evaluation of a universal positive behavior sup- port initiative. Journal of Positive Behavior Interventions, 12, 198–210. https://doi.org/10.1177/1098300709340699

*Childs, K. E., Kincaid, D., & George, H. P. (2016). The rela- tionship between school-wide implementation of positive behavior intervention and supports and student discipline outcomes. Journal of Positive Behavior Interventions, 18, 89–99. https://doi.org/10.1177/1098300715590398

Clonan, S. M., McDougal, J. L., Clark, K., & Davidson, S. (2007). Use of office discipline referrals in school-wide decision making: A practical example. Psychology in the Schools, 44, 19–27. https://doi.org/10.1002/pits.20202

Coffee, J. H., & Horner, R. H. (2012). The sustainability of schoolwide positive behavior interventions and supports. Exceptional Children, 78, 407–422. https://doi.org/10.1177 /001440291207800402

Cohen, R., Kincaid, D., & Childs, K. E. (2007). Measuring school- wide positive behavior support implementation: Development

Estrapala et al. 301

and validation of the benchmarks of quality. Journal of Positive Behavior Interventions, 9, 203–213.

Every Student Succeeds Act, 20 U.S.C. § 6301. (2015). https:// www.congress.gov/114/plaws/publ95/PLAW-114publ95.pdf

*Flannery, K. B., Fenning, P., McGrath Kato, M., & McIntosh, K. (2014). Effects of school-wide positive behavioral interven- tions and supports and fidelity of implementation on problem behavior in high schools. School Psychology Quarterly, 29, 111–124.

Flannery, K. B., Frank, J. L., McGrath Kato, M., Doren, B., & Fenning, P. (2013). Implementing schoolwide positive behav- ior support in high school settings: An analysis of eight high schools. The High School Journal, 96, 267–282. https://doi. org/10.1037/spq0000039

Flannery, K. B., Hershfeldt, P., & Freeman, J. (2018). Lessons learned on implementation of PBIS in high schools: Current trends and future directions. Center for Positive Behavioral Interventions and Supports (funded by OSEP, US Department of Education), University of Oregon Press.

Flannery, K. B., & McGrath Kato, M. (2017). Implementation of SWPBIS in high school: Why is it different? Preventing School Failure, 61, 69–79. https://doi.org/10.1080/10459 88X.2016.1196644

Flannery, K. B., Sugai, G., & Anderson, C. M. (2009). School- wide positive behavior support in high school. Journal of Positive Behavior Interventions, 11, 177–185. https://doi. org/10.1177/1098300708316257

Freeman, J., & Lombardi, A. (2018). Promoting college and career readiness through PBIS [Conference session]. International Conference on Positive Behavior Support, San Diego, CA, United States.

*Freeman, J., Simonsen, B., McCoach, D. B., Sugai, G., Lombardi, A., & Horner, R. H. (2015). An analysis of the relationship between implementation of school-wide positive behavior interventions and supports on high school dropout rates. The High School Journal, 98(4), 290–315. https://doi. org/10.1353/hsj.2015.0009

*Freeman, J., Simonsen, B., McCoach, D. B., Sugai, G., Lombardi, A., & Horner, R. H. (2016). Relationship between school- wide positive behavior interventions and supports and aca- demic, attendance, and behavior outcomes in high schools. Journal of Positive Behavior Interventions, 18, 41–51. https:// doi.org/10.1177/1098300715580992

Gage, N., Sugai, G., Lewis, T. J., & Brzozowy, S. (2015). Academic achievement and school-wide positive behavior supports. Journal of Disability Policy Studies, 15, 99–209. https://doi.org/10.1177/1044207313505647

Gage, N., Whitford, D. K., & Katsiyannis, A. (2018). A review of schoolwide positive behavior interventions and supports as a framework for reducing disciplinary exclusions. The Journal of Special Education, 52, 142–151. https://doi.org/10127274/ 060629214686796178874678

Gresham, F. M., & Lopez, M. F. (1996). Social validation: A unifying concept for school-based consultation research and practice. School Psychology Quarterly, 11, 204–227.

Horner, R. H., Carr, E. G., Halle, J., Mcgee, G., Odom, S. L., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education.

Exceptional Children, 71(2), 165–179. https://doi.org/.1177 /001440290507100203

Horner, R. H., Sugai, G., & Anderson, C. M. (2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptional Children, 42(8), 1–14.

Horner, R. H., Todd, A. W., Lewis-Palmer, T., Irvin, L., Sugai, G., & Boland, J. B. (2004). The school-wide evaluation tool (SET): A research instrument for assessing school-wide positive behavior support. Journal of Positive Behavior Interventions, 6, 3–12.

Kim, J., McIntosh, K., Mercer, S. H., & Nese, R. N. T. (2018). Longitudinal associations between SWPBIS fidelity of implementation and behavior and academic outcomes. Behavioral Disorders, 43, 357–369. https://doi.org/10.1177 /0198742917747589

Lane, K. L., Kalberg, J. R., Bruhn, A. L., Driscoll, S. A., Wehby, J. H., & Elliott, S. N. (2009). Assessing social validity of school-wide positive behavior support plans: Evidence for the reliability and structure of the Primary Intervention Rating Scale. School Psychology Review, 38, 135–144.

*Lane, K. L., Wehby, J. H., Robertson, E. J., & Rogers, L. A. (2007). How do different types of high school students respond to schoolwide positive behavior support programs? Characteristics and responsiveness of teacher-identified stu- dents. Journal of Emotional and Behavioral Disorders, 15, 3–20.

Lewis, T., & Sugai, G. (1999). Effective behavior supports: A sys- tems approach to proactive schoolwide management. Focus on Exceptional Children, 31, 24–37.

*Malloy, J. M., Bohanon, H., & Francoeur, K. (2018). Positive behavioral interventions and supports in high schools: A case study from New Hampshire. Journal of Educational and Psychological Consultation, 28, 219–247. https://doi.org/10 .1080/10474412.2017.1385398

Marchant, M., Heath, M. A., & Miramontes, N. Y. (2012). Merging empiricism and humanism: Role of social validity in the school-wide positive behavior support model. Journal of Positive Behavior Interventions, 14, 221–230. https://doi. org/10.1177/1098300712459356

McCrary, D., Lechtenberger, D., & Wang, E. (2012). The effect of schoolwide positive behavioral supports on chil- dren in impoverished rural community schools. Preventing School Failure, 56, 1–7. https://doi.org/10.1080/10459 88X.2010.548417

McDaniel, S., & Bruhn, A. L. (2016). Using a changing-criterion design to evaluate the effects of check-in/check-out with goal modification. Journal of Positive Behavior Interventions, 18(4), 197–208. https://doi.org/10.1177/10983007155 88263

McIntosh, K., Mercer, S. H., Hume, A. E., Frank, J. L., Turri, M. G., & Mathews, S. (2013). Factors related to sustained implementation of schoolwide positive behavior support. Exceptional Children, 79(3), 293–311.

Morningstar, M. E., Lombardi, A., Fowler, C. H., & Test, D. W. (2017). A college and career readiness framework for sec- ondary students with disabilities. Career Development and Transition for Exceptional Individuals, 40, 79–91. https://doi. org/10.1177/2165143415589926

302 Journal of Positive Behavior Interventions 23(4)

*Muscott, H. S., Mann, E. L., & LeBrun, M. R. (2008). Positive behavioral interventions and supports in New Hampshire. Journal of Positive Behavior Interventions, 10, 190–205. https://doi.org/10.1177/1098300708316258

Noltemeyer, A., Palmer, K., James, A. G., & Wiechman, S. (2019). School-wide positive behavioral interventions and supports (SWPBIS): A synthesis of existing research. International Journal of School & Educational Psychology, 7, 253–262. https://doi.org/10.1080/21683603.2018.1425169

OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. (2017). Positive Behavioral Interventions & Supports [Website]. www.pbis.org

Romer, D., Reyna, V. F., & Satterthwaite, T. D. (2017). Beyond stereotypes of adolescent risk taking: Placing the adolescent brain in developmental context. Developmental Cognitive Neuroscience, 27, 19–34. https://doi.org/10.1016/j.dcn.2017 .07.007

*Smolkowski, K., Strycker, L., & Ward, B. (2016). Scale-up of safe & civil schools’ model for school-wide positive behav- ioral interventions and supports. Psychology in the Schools, 53, 339–358. https://doi.org/10.1002/pits.21908

Spaulding, S., Irvin, L. K., Horner, R. H., May, S. L., Emeldi, M., Tobin, T. J., & Sugai, G. (2010). Schoolwide social-behav- ioral climate, student problem behavior, and related admin- istrative decisions: Empirical patterns from 1,510 schools nationwide. Journal of Positive Behavior Interventions, 12, 69–85. https://doi.org/10.1177/1098300708329011

Swain-Bradway, J., Pinkney, C., & Flannery, K. B. (2015). Implementing schoolwide positive behavior interventions and supports in high schools. Teaching Exceptional Children, 47, 245–255. https://doi.org/10.1177/0040059915580030

Tobin, T. J. (2008). Will functional interventions in versatile envi- ronments reduce dangerous behaviors? Journal of Behavior Analysis of Offender and Victim: Treatment and Prevention, 1, 171–186. https://doi.org/10.1037/h0100442

Vancel, S. M., Missall, K. N., & Bruhn, A. L. (2016). Teacher ratings of the social validity of Schoolwide Positive Behavior Interventions and Supports: A comparison of school groups. Preventing School Failure, 60, 320–328. https://doi.org/10.1 080/1045988X.2016.1157784

VanGronigen, B. A., & Meyers, C. V. (2019). How state educa- tion agencies are administering school turnaround efforts: 15 years after No Child Left Behind. Educational Policy, 33, 423–452. https://doi.org/10.1177/0895904817691846

*Vincent, C. G., & Tobin, T. J. (2011). The relationship between implementation of school-wide positive behavior support (SWPBIS) and disciplinary exclusion of students from various ethnic backgrounds with and without disabili- ties. Journal of Emotional and Behavioral Disorders, 19, 217–232. https://doi.org/10.1177/1063426610377329

What Works Clearinghouse. (2015). Procedures and standards handbook, version 3.0. https://ies.ed.gov/ncee/wwc/Docs/ referenceresources/wwc_procedures_v3_0_standards_hand- book.pdf