Assignment: Course Project – Collaboration for Effective Implementation of School-wide Support Systems

O R I G I N A L P A P E R

A Supportive Beliefs Intervention to Facilitate the Implementation of Evidence-Based Practices Within a Multi-Tiered System of Supports

Clayton R. Cook • Aaron R. Lyon •

Dajana Kubergovic • Diana Browning Wright •

Yanchen Zhang

Published online: 21 January 2015

� Springer Science+Business Media New York 2015

Abstract The current study represents a preliminary

examination of school-level beliefs and attitudes as they

relate to the implementation of universal evidence-based

practices (EBPs) within a multi-tiered system of support

(MTSS) targeting student’s social, emotional, and behav-

ioral problems. This study was organized around three

primary objectives: (a) assess coaches’ perceptions of the

utility and importance of targeting educator beliefs to

improve adoption and use of EBPs, (b) demonstrate the

association between educator beliefs and degree of MTSS

implementation, and (c) conduct a preliminary pre–posttest

of a supportive belief intervention (SBI) to enhance edu-

cators’ beliefs and examine whether changes in beliefs

were associated with improved implementation. To

accomplish these objectives, data were collected from 62

elementary schools across five school districts involved in

a collaborative consultative partnership to design and

implement a school-wide MTSS framework. Collectively,

the results provided preliminary support for the importance

of beliefs: (a) coaches reported beliefs were critical to

implementation and facilitative of their roles working with

teachers, (b) educator beliefs predicted initial implemen-

tation fidelity on a global measure of MTSS practices and

specific measure of school-wide positive behavior inter-

ventions and supports, and (c) the SBI was associated with

significant changes in educator beliefs and these changes

were associated with improved implementation. The

implications, limitations, and future directions of this

research are discussed.

Keywords Implementation � Social, emotional, and behavioral (SEB) � Evidence-based practices (EBPs) � Multi-tiered systems of support (MTSS)

Introduction

Studies indicate that approximately one out of every five

students has a diagnosable mental health disorder (Cos-

tello, Mustillo, Erkanli, Keler, & Angold, 2003). Many

more exhibit milder forms of social, emotional, and

behavioral (SEB) problems that do not reach clinical lev-

els, but nevertheless negatively impact academic achieve-

ment (Goodman, Joyce, & Smith, 2011), are associated

with an increased likelihood of short- and long-term neg-

ative outcomes (Beesdo & Knappe, 2012), and, in some

circumstances, may also impede their classmates’ learning

(Trout, Nordness, Pierce, & Epstein, 2003). As a result,

there is burgeoning interest in school mental health to

prevent social, emotional, and educational problems

(Durlak, Weissberg, Dymnicki, Taylor & Schellinger,

2011).

Multi-Tiered Systems of Support (MTSS)

A multi-tiered system of support (MTSS) has been advo-

cated as a way to organize and deliver a continuum of

evidence-based practices (EBPs) to address students’ SEB

needs (EBPs; Cook, Burns, Browning-Wright, & Gresham,

2010). MTSS is a proactive, prevention-oriented service

delivery framework that aims to meet all students’ needs

through the implementation of a continuum of EBPs via

C. R. Cook (&) � A. R. Lyon � D. Kubergovic � Y. Zhang University of Washington, Seattle, WA, USA

e-mail: [email protected]

D. Browning Wright

Diana Browning Wright Consulting, Inc., Sierra Madre, CA,

USA

123

School Mental Health (2015) 7:49–60

DOI 10.1007/s12310-014-9139-3

data-driven decision making (Strein, Hoagwood, & Cohn,

2003). The continuum of EBPs consists of delivering uni-

versal (i.e., Tier 1) supports for all students, selected (i.e.,

Tier 2) interventions for some students, and indicated (i.e.,

Tier 3) treatments for a few students. Although research

has demonstrated that numerous EBPs can be implemented

within a MTSS framework, multiple barriers interfere with

their uptake and sustained use under natural educational

conditions (Forman et al., 2012). Barriers include insuffi-

cient post-training consultation (i.e., ‘‘train and hope’’)

(Joyce & Showers, 2002), unsupportive leadership (Aa-

rons, 2006), and policies that are counterproductive to the

implementation of new innovations (Slavin, 2002).

Bridging the Implementation Gap

Increasingly, research is focusing on understanding

implementation processes and identifying strategies that

help transfer research findings into community-based social

service settings (Powell et al., 2012). Given the limited use

of EBPs in schools, researchers and educational leaders

have also called for the identification of multi-level strat-

egies that facilitate the use of effective practices in the

education sector (Forman et al., 2013; Owens et al., 2014).

In particular, multiple components of the inner organiza-

tional context (i.e., the settings and individuals involved in

implementation efforts) have been identified that influence

implementation success and represent specific pinpoints for

implementation enhancement interventions. These include

organizational climate—defined as personnel’s perceptions

of, and emotional reactions to, the characteristics of their

work setting (Aarons & Sawitzky, 2006), principal lead-

ership (Elias et al., 1997), and organizational citizenship—

defined as behaviors that go beyond the standard ‘‘call of

duty’’ or core job aspects (Organ et al., 2006).

The characteristics of the school organizational context

described above can be distinguished from the steps or

approaches needed to actually install an EBP. For instance,

effective professional development has been shown to be a

key ingredient for quality installation of innovative

educational programs (Penuel, Fishman, Yamaguchi, &

Gallagher, 2007). Studies have demonstrated that, to be

successful, professional development should also incorpo-

rate coaching to facilitate EBP implementation in schools

(Joyce & Showers, 2002). Coaching represents a form of

facilitative process that helps implementers comprehend

the purpose of the intervention, internalize intervention

delivery, and adapt the intervention to the context (Du-

senbury et al., 2005). Coaching has emerged as a promising

strategy that enhances relevant implementation (i.e.,

appropriateness and fidelity) and student (e.g., behavioral

and academic) outcomes in the context of EBP imple-

mentation over and above those observed via standard

professional development activities (Bradshaw et al., 2012;

Haskins & Loeb, 2007). Unfortunately, findings demon-

strate that many implementers are ambivalent or resistant

to coaching interactions and that implementation may

suffer as a result (Dart, Cook, Gresham, Collins, & Che-

nier, 2012). For these reasons, coaching alone may be

insufficient to produce high levels of implementation

(Fixsen, Blase, Naoom, & Wallace, 2009).

Educator Beliefs and Attitudes

Research supports the view that the beliefs and attitudes of

purveyors or implementers are likely to influence the

uptake and use of EBPs (Nelson & Steele, 2007).

Researchers in numerous fields, including education, have

also narrowed in on the importance of implementer beliefs

and attitudes to the utilization of EBPs (Aarons & Palinkas,

2007; Haney, Czerniak, & Lumpe, 1996). In recognition of

the role of beliefs and attitudes, Lyon, Stirman, Kerns, and

Bruns (2011) advocated for the development of effective

methods of increasing professionals’ motivation to imple-

ment new innovations in their practice. Specifically, they

identified existing motivational enhancement strategies that

take direction from contemporary theories of motivation

(e.g., Fishbein et al., 2001) to identify specific areas for

improvement in order to build provider commitment (e.g.,

Feifer et al., 2006). However, limited to no attention has

focused on intervening with educators on their beliefs prior

to and throughout implementation.

Beliefs and attitudes among educators have been argued

to be a prerequisite to significant change in practices and

improvement in outcomes within schools (Guskey, 1986).

Both qualitative and quantitative researches support the

view that educators’ beliefs toward certain practices and

their own professional roles are likely to influence the

uptake and use of EBPs (Bowden, Lanning, Pippin, &

Tanner, 2003; Parcel, O’Hara-Tompkins, Harrist, & Basen-

Engquist, 1995). For example, teacher beliefs have been

shown to correlate with SWPBIS implementation fidelity

(Kincaid et al., 2007) and their willingness to adopt and

implement social–emotional learning curricula (Brackett,

Reyes, Rivers, Elbertson, & Salovey, 2012). Moreover,

educators who possess a belief that components of some

EBPs are detrimental to students (e.g., that extrinsic rein-

forcement—a key component of SWPBIS and some SEL

programs—harms intrinsic motivation) may have limited

intentions to implement those EBP (Maag, 2001). Unfor-

tunately, the extant literature is correlational in nature, with

few studies explicitly focusing on ways to promote sup-

portive beliefs and attitudes as an implementation

enhancement strategy. Pre-implementation interventions

that focus on altering beliefs and attitudes prior to actual

implementation are likely to pay significant dividends

50 School Mental Health (2015) 7:49–60

123

during the subsequent phases of the implementation pro-

cess (e.g., implementation and sustainment) when profes-

sionals’ attitudes about EBP become particularly salient

(Aarons et al., 2011).

Researchers have advocated for the use of theory in

designing interventions to improve implementation, as

theoretically informed interventions are likely to be more

effective if they focus on causal determinants of human

behavior. Social-cognitive theories, in particular the theory

of planned behavior (TPB), have been argued to be useful

in the formation of implementation enhancement inter-

ventions (Eccles et al., 2005). The TPB was developed to

predict and explain human behavior in specific contexts

(Ajzen & Manstead, 2007). The TPB states that an indi-

vidual’s intention to carry out a given behavior is the most

potent predictor of behavior (Fishbein & Ajzen, 1975).

According to the TPB, the recipe for successful adoption

and implementation of EBP is an implementer who has

supportive beliefs about the intervention, perceives there to

be social expectations/pressure to implement the practices,

and believes in one’s capability to implement the practice

and that doing so will make a relevant difference (Eccles

et al., 2007). However, to date, limited to no research has

examined the potential impact of integrating this theory

into an intervention to promote better implementation.

Purpose of the Present Study

Considering the role of implementer beliefs and the

importance of developing theoretically informed interven-

tions, the purpose of this study was to examine the

importance of educator beliefs and attitudes as they relate

to the adoption and use of EBPs targeting students’ SEB

functioning. This study was conducted in the context of a

collaborative partnership with several school districts to

develop and implement a behavioral multi-tiered system of

support (B-MTSS) framework in an effort to promote

better social, emotional, and academic outcomes. We

organized this research according to three specific objec-

tives. The first objective was to examine coaches’ per-

ceptions of the importance (i.e., represents an important

aspect of the implementation process) and utility (i.e.,

makes coaching easier and more effective) of addressing

educator beliefs and attitudes to facilitate implementation

of practices associated with a B-MTSS framework. The

second objective was to examine whether educators’

beliefs collected pre- and post-PD and coaching were

associated with the degree to which elements of the

B-MTSS framework were being implemented with fidelity.

The last objective was to conduct a pre–posttest to examine

whether a supportive belief intervention (SBI) was able to

produce significant, positive changes in educators’ beliefs

and attitudes over time, and whether changes in beliefs

were correlated with global and specific measures assessing

fidelity of implementation. It is important to note that this

represents a preliminary study examining the role of edu-

cator beliefs as they relate to implementation of MTSS

practices.

Based on the research objectives outlined above, we had

several hypotheses. First, we postulated that coaches would

report that beliefs are critical to implementation success.

Second, we hypothesized that educator beliefs and attitudes

would be predictive of school-level MTSS implementation

fidelity broadly and of SW-PBIS fidelity specifically.

Third, we predicted that we would observe a significant

change in educators’ beliefs across schools following the

SBI and this change would be associated with improved

implementation outcomes.

Methods

Participating Schools

A total of 62 elementary schools across five school districts

were included in this study. These schools were drawn

from districts that were participating in system-wide

reform efforts to implement a B-MTSS. Districts were

located in different geographic regions of the USA: Mid-

west, Southern California, and Northern California. In

total, there were a total of 24,118 students enrolled in these

schools. The demographic data on the students in the

participating elementary schools were as follows: gender

(51 % male and 49 % female), ethnicity (48 % White,

19 % Hispanic/Latino, 18 % Asian/Pacific Islander, 13 %

African American, and 2 % Other), and socioeconomic

status (38 % free and reduced lunch). Moreover, 13.7 % of

the students in the schools were receiving special education

services.

There were a total of 1,181 educators (94 administrators,

1,071 teachers, and 16 coaches) who participated in the

study. As for educator demographics, the average years in

the profession were 7.6 (SD = 4.1) for teachers, 8.3

(SD = 5.2) for administrators, and 9.8 (SD = 3.5) for

coaches. The ethnic breakdown of educators was 78 %

Caucasian, 8 % Asian/Pacific Islander, 6 % African

American, 3 % Hispanic/Latino, and 6 % other. The school

districts experienced an average staff attrition rate of 24 %

per year (range 14–35 %). At the outset of the study, no

school districts were implementing a system-wide

approach to addressing student SEB needs.

Coaches were existing employees within the school

district whose positions were repurposed to provide

coaching supports to individual schools on the implemen-

tation of the B-MTSS, including providing performance-

based feedback on the implementation of practices,

School Mental Health (2015) 7:49–60 51

123

modeling certain practices when needed, and engaging in a

problem-solving process with school teams to overcome

barriers to implementation. These coaches were selected

given their prior experience and training in consultation

and coaching methods and received specialized training

and support (e.g., two additional days of professional

development, attendance at coaching conferences/work-

shops, and maintained ongoing communication with the

outside consultants) to facilitate the development of their

content expertise. The two days of coaching professional

development consisted training them in data-based deci-

sion making (e.g., fidelity rubrics to track implementation

practices and inform performance-based feedback), the

problem-solving method (problem identification, problem

analysis, generate plan, plan implementation, and plan

evaluation), and effective coaching techniques (e.g.,

motivational interviewing techniques, rapport building,

group facilitation). Coaches were primarily school psy-

chologists (n = 10), but also included former site admin-

istrators (n = 3) and school social workers (n = 3).

Procedures

The school districts contacted the first and last authors to

engage in a multi-year, district-wide initiative to imple-

ment a B-MTSS targeting students’ SEB needs. All the

data were collected as part of the B-MTSS initiative to

facilitate program evaluation and create a feedback system

to improve future implementation. IRB approval was

obtained and no identifying student information was

gathered as part of the data collection process. The

B-MTSS initiative was designed based on a horizontal

(within tiers of support) and vertical (across tiers of sup-

port) integration of EBPs framework (Domitrovich et al.,

2010) to facilitate implementing a continuum of supports

and making ongoing decisions about students and practices

based on data. This model is discussed in greater detail in

other sources (Sprague, Cook, Browning Wright, & Sadler,

2008; Cook, Browning Wright, Gresham, & Burns, 2010).

Consistent with system change literature, this initiative

was designed as a three-year project in which participating

schools build capacity over time to ultimately build the

model to scale (Aarons, Hurlburt, & Horwitz, 2011). There

were a total of four PD sessions during the first year. Each

PD session entailed 2 days, each lasting 6 h (total of 8 six-

hour days). The number of PD days was systematically

decreased to 6 days during the second year and 4 days

during the third year. The initiative was grounded in a

train-the-trainer model of implementation with embedded

coaching provided to each of the participating schools

between each of the professional development sessions.

External consultants (a researcher first author and profes-

sional educational consultant last author) trained each of

the site-based teams to be indigenous trainers and dis-

seminators back at their site in order to build internal

expertise within each school. Coaches attended all PD

sessions and supported teams’ efforts to disseminate

information and implement practices back at their school

site.

The teams included site administrators, professional

support staff (school psychologist, counselor, and/or

behavior specialist), two general education teachers, and

two special education teachers. School administrators were

instructed to select influential members of their building

who were open and willing to support implementation

efforts back at the school site (Atkins et al., 2008). Time

was reserved at the end of each professional development

session to create an action plan (what will be done, who

will do it, what resources are needed to do it, and by when

will it be done) based on each school’s identified priorities

to incrementally implement the content and practices

associated with the B-MTSS. The main content for the first

PD session were modeled activities to promote educator

beliefs with regard to supporting student SEB functioning

(see below description of SBI), while only a little time was

devoted to new SBI activities during the remaining PD

sessions. However, for all PD sessions, teams were

instructed to review their school-level beliefs and develop

action items to deliver aspects of the SBI.

The initiative emphasized a sequential yet recursive

process of establishing readiness and scaling up imple-

mentation of key practices of the B-MTSS framework. The

first step in this process was to establish beliefs and atti-

tudes that are supportive of the adoption and use of

effective practices. Knowledge and skills represented the

next component of the system change process. The

knowledge component of the system change process was to

deliver specific content that focused on developing the

conceptual and practical understanding of the practices that

populate the B-MTSS framework. The skills component of

the system change approach consisted of breaking down

each of the practices into its constituent practices (e.g.,

progressive system of responding to problem behavior—

proximity, redirection tactics, ongoing monitoring to rein-

force desirable behavior, effective command, and teaching

interaction with the student) and teaching educators how to

implement practices with fidelity using a tell-show-do

approach. The last component of the system change pro-

cess was to develop specific procedures that would facili-

tate the sustainment of implementation. These procedures

included periodic administrative walk-through observa-

tions to monitor implementation, structured professional

learning community meetings about specific practices

included within the B-MTSS framework, and developing

specific policy that outlines expectations to implement

particular practices.

52 School Mental Health (2015) 7:49–60

123

A pre–post-design was used to address the three

research objectives outlined above. Baseline data were

collected in the Fall prior to the initial B-MTSS PD ses-

sion. All post data were collected in the Spring at the end of

the academic year.

Interventions

Behavioral MTSS Framework

The B-MTSS framework was based on the seven key

concepts of MTSS: (1) multiple tiers of support, (2) evi-

dence-based practices, (3) universal screening, (4) progress

monitoring, (5) data-driven decision making, (6) fidelity of

implementation, and (7) problem-solving teaming (see

Cook et al., 2010). Moreover, as discussed above, the

B-MTSS framework emphasizes the integration of theo-

retically different practices within certain tiers of support

(e.g., SW-PBIS and SEL within Tier 1) that serve different

purposes yet are complementary of one another, as well as

the integration of supports across tiers of support to facil-

itate better needs-driven programming. For the purposes of

this study, only the Tier 1 support system was emphasized,

considering that the main objective of Year 1 of the ini-

tiative was to facilitate implementation of the Tier 1 sup-

ports for all students. The following practices comprised

the Tier 1 level: SW-PBIS (positive, structured, and safe

environment), adoption of an SEL curriculum (teaching

critical skills), proactive classroom management (positive,

structured, and safe environment), intentional practices to

establish, maintain, and restore positive relationships with

students (positive relationships), and universal screening

practices (needs-driven programming). For a greater dis-

cussion of each of these components, see Sprague et al.

(2008) and Cook et al. (2010).

Supportive Belief Intervention (SBI)

The TPB provided the overarching theoretical framework

to design and deliver the SBI to alter, shift, or solidify

supportive beliefs and attitudes among educators to facili-

tate the adoption and use of EBPs, which has been shown

to predict and explain human behavior in specific contexts

(Ajzen & Manstead, 2007). Thus, the authors developed

and integrated a range of supportive belief activities

throughout the professional development sessions that

would facilitate positive attitudes, increase social expec-

tations and pressure around the implementation of EBPs,

and improve self-efficacy. Moreover, social influence and

persuasion tactics were embedded throughout (Cialdini,

2001; Pratkanis, 2007), such as the saying is believing (i.e.,

generating and advocating a persuasive message to a

receptive audience; Aronson, 1999), social proofing (i.e.,

people act a certain way because they observe or hear about

others acting this way; Cialdini, 2009), and commitment

and consistency (i.e., people behave consistently with their

commitments; Sundle, Cialdini, Griskevicius, & Kenrick,

2012). The content included relevant video clips, small

group reflective exercises (e.g., what are anti-relationship

strategies that can be used in schools? Develop a mindset

device that enables the perfect belief system when working

with challenging students—what beliefs and attitudes

would you program into this machine?), engaging exam-

ples to highlight important research findings (e.g., using

humor and interesting examples), testimonial stories from

other educators (e.g., testimonial from teacher indicating

the importance of proactive supporting students), and

metaphors were integrated into a presentation.

Measures

Beliefs About Behavior Survey (BABS)

The BABS (Browning Wright & Cook, 2008) was used to

measure school-level beliefs and attitudes relevant to

practices targeting students’ SEB functioning. Consistent

with effective scale construction, the content of the BAB

was originally developed based on expert consensus. Five

experts in SEB and MTSS were asked to generate a list of

common beliefs that were either facilitate of (positively

stated belief) or interfered with (negatively stated belief)

the adoption and implementation of EBPs that target

improving students’ SEB performance. A total of 92 beliefs

were generated as the initial item pool. This list was then

narrowed down using a sorting process that consisted of

ranking items from most to least important and eliminating

redundant beliefs. This resulted in 35 negatively and pos-

itively stated beliefs that were measured on a 5-point Likert

scale ranging from ‘‘strongly disagree’’ to ‘‘strongly

agree.’’

A favorable or supportive belief is coded as either

agreeing with a positively stated belief (e.g., ‘‘even without

parental involvement and support, schools can effectively

teach students’ behavioral expectations and social–emo-

tional skills’’) or disagreeing with a negatively stated belief

(e.g., ‘‘teaching students how to behave appropriately at

school is the parents’ or students’ responsibility—not

mine’’). To capture school-level beliefs, an index was

calculated based on the number of beliefs that 80 % or

more staff in a school provided ratings in the supportive or

favorable direction (agree with a positively stated belief

and disagree with a negatively stated one). The greater the

value on this index, the greater the number of supportive

beliefs educators’ possessed collectively in a given school.

The 80 % criterion was based on research that has identi-

fied this percentage as a tipping point for implementation

School Mental Health (2015) 7:49–60 53

123

(Fixsen & Blase, 1993; Horner et al., 2004). Thus, a value

of 15 indicated that 80 % or more of the staff had sup-

portive beliefs on 15 of the 35 items. Prior research has

evaluated the technical adequacy of the BABS, and find-

ings indicate that it possesses adequate test–retest reli-

ability (r = .88) and convergent and divergent validity

(Cook & Browning Wright, 2012).

Multi-Tiered System of Support for Behavior Evaluation

Rubric (MTSS-BER)

The MTSS-BER (Cook & Browning Wright, 2012) was

developed to serve as a global measure of the adoption and

implementation of key practices within the B-MTSS

framework that was completed by site-based teams with

the assistance of their assigned coach. The MTSS-BER has

been shown to be predictive of school-level outcome data,

including academic achievement, behavioral data, and

school climate (Cook, 2013). Each item on the MTSS-BER

is rated on a four-point scale to reflect the degree of

implementation: exploring and planning, partially imple-

menting, moderately implementing, and fully implement-

ing. The MTSS-BER includes a total of 28 items that are

broken down into six domains: (1) vision, beliefs, and

objectives, (2) data-based decision making, (3) Tier 1

implementation, (4) Tier 2 implementation, (5) Tier 3

implementation, and (6) equity practices. Each domain was

measured by multiple items to capture the degree to which

a school disseminates and implements key components of

the B-MTSS Framework. For the purposes of this study

and given the scope of Year 1 of this project, only the first

three domains from the MTSS-BER were utilized for

analyses. Thus, a total score was computed across the three

categories to represent an overall degree of follow through

and fidelity of implementation for Year 1.

School-Wide Evaluation Tool (SET)

To include a gold standard measure of implementation

fidelity, the SET (Sugai et al., 2000) was used to capture

fidelity of SW-PBIS practices. The SET is a direct obser-

vation measure of seven core features of universal SWPBIS

fidelity (behavioral expectations defined, behavioral

expectations taught to students, rewards delivered for

appropriate behavior, predictable consequences, formal

systems to collect and use data, administrator support, and

district support). Scores range from 0 and 100 %,

(100 % = perfect fidelity). The SET has documented

inter-observer agreement (r = .99), internal consistency

(a = .96), test–retest reliability (r = .97), and construct validity (r = .75 with staff-reported effective behavior

support systems; Horner et al., 2004). Trained coaches

within each school district conducted the SET observations

and completed the ratings. No inter-observer agreement

data were collected as part of this study.

Coach Survey

For the purposes of this study, a coach survey was devel-

oped to capture coaches’ perceptions regarding the

importance of addressing beliefs and attitudes to facilitate

the implementation of EBPs according to a MTSS frame-

work, as well as the extent to which they perceived that the

SBI content facilitated their coaching efforts surrounding

implementation of the Tier 1 level of supports. A total of

three items were constructed to measure these perceptions:

(1) To what extent was the focus on altering beliefs helpful

to remove resistance/ambivalence among educators to

implement particular EBPs associated with the B-MTSS?;

(2) To what extent have the professional development

activities targeting educators’ beliefs made it easier and

more effective for you to do your job of coaching or

consulting with them?; and (3) How important is it for

school systems to provide specific information or trainings

to educators’ to alter their mindsets or belief system before

actually training them on implementing an evidence-based

practice? The coach survey possessed acceptable internal

consistency (a = 0.72) (Nunnally & Bernstein, 1994).

Results

Data analyses were conducted in step with the research

objectives described above. Coaches’ perceptions of the

utility and importance of addressing educator beliefs were

examined first. Following, descriptive statistics were cal-

culated and correlational analyses performed to examine

the association between the independent variable (i.e.,

educator beliefs) and the dependent variables (global and

specific implementation fidelity). Last, a pre–posttest of the

impact of the SBI was performed (t test on change scores)

followed up by correlational and regression analyses to

examine whether changes in beliefs over time were asso-

ciated with implementation outcomes. All analyses were

conducted using SPSS version 21.

Research Objective #1

The first research objective focused on gathering data on

coaches’ perceptions of the importance and utility of

implementation enhancement efforts that intentionally

focus on altering educators’ beliefs and attitudes related to

the implementation of SEB EBPs. Data were obtained from

a total of 16 coaches spread across the five districts who

were assigned specific schools to support the implemen-

tation of the B-MTSS framework (4–7 schools depending

54 School Mental Health (2015) 7:49–60

123

on the district). Descriptive statistics indicated that the

mean ratings across the three items were 4.87 (range 4–5),

4.77 (range 4–5), and 4.95 (range 4–5) for items 1, 2, and 3,

respectively. As one can see, there was very little vari-

ability between the coaches’ ratings, with the average rat-

ings indicating that coaches endorsed the utility and

importance of addressing educators’ beliefs. Specifically,

the findings indicated that coaches reported that addressing

educators’ beliefs was (a) helpful to alleviate staff resis-

tance and ambivalence to implementing practices to pro-

mote student SEB well-being (item 1), (b) useful to make

their job of coaching and consulting with educators easier

and more effective (item 2), and (c) extremely important to

facilitate the adoption and implementation of EBPs (item

3).

Research Objective #2

To examine whether school-level educators’ beliefs were

predictive of relevant global and specific implementation

outcomes, we first computed descriptive statistics to cap-

ture the means and standard deviations of the independent

(BABS scores) and dependent variables (MTSS-BER and

SET scores), see Table 1. Descriptive statistics for the

BABS scores indicated that the average for pre and post

was 14.32 (SD = 3.46; range 8–23) and 18.69 (SD = 4

.89; range 9–28), respectively. These data indicated that

participating schools had an average of 14 out of the 35

beliefs upon which 80 % or more of the staff endorsed

supportive beliefs at pre (i.e., agree with a positively stated

belief or disagree with a negatively stated belief) and

nearly 19 out of 35 beliefs at post. The MTSS-BER and

SET were measured at post (i.e., end of Year 1), and the

descriptive statistics are included in Table 1. As one can

see, there was significantly variability across all the

implementation indices, with ranges indicating that there

were schools with low implementation, moderate imple-

mentation, and high implementation.

Correlational analyses were then performed to examine

the relationship of school-level BABS scores and the

implementation variables (see Table 2). We first examined

the data to determine whether the assumptions of correla-

tional analyses (e.g., normality, continuous data, linear

relationship, no major outliers), and the basic assumptions

were met. The results indicated that both pre- and post-

BABS scores correlated significantly and positively with

the implementation outcome variables. Comparison of the

correlations across data collection time points indicated

that post-BABS scores had consistently stronger correla-

tions with implementation outcome variables than the pre-

BABS scores. The pre-BABS correlations indicated that

schools with higher initial beliefs were associated with

higher scores at post across all the implementation vari-

ables. Specifically, the pre-BABS scores correlated sig-

nificantly with the four MTSS-BER scores (vision/beliefs/

objectives, data-based decision making, Tier 1 global

Table 1 Descriptive statistics for the independent and dependent variables

Variables Mean SD Minimum Maximum

Pre-beliefs about behavior survey a

14.32 3.46 8 23

Post-beliefs about behavior survey b

18.69 4.89 9 28

Vision/beliefs/objectives* ,c

5.15 1.94 2 9

Data-based decision making* ,d

5.73 2.11 3 10

Tier 1 implementation* ,e

7.88 2.47 4 12

MTSS-BER total score f

18.76 5.21 9 29

School-wide evaluation tool g

67.68 15.64 30 86

MTSS-BER Multi-tiered System of Support Behavior Evaluation Rubric; *subscales of the MTSS-BER. The ranges for all of the scales are: a

0–35, b

0–35, c

0–9, d

0–12, e

0–15, f

0–36, and g

0–100

Table 2 Bivariate correlations between pre- and post-BABS scores and implementation outcome variables

MTSS-BER

vision and objectives

MTSS-BER

data-based decision making

MTSS-BER

Tier 1

MTSS-BER

total score

SET score

Pre school-level BABS .47*** .24* .36*** .62*** .41***

Post school-level BABS .54*** .33** .43*** .74*** .57***

D School-level BABS .41*** .25* .48*** .67*** .51***

* p \ .05; ** p \ .01; *** p \ .001

School Mental Health (2015) 7:49–60 55

123

implementation, and total score) and degree of imple-

mentation of SW-PBIS (SET score). The pre-BABS scores

were correlated the strongest with the MTSS-BER total

score (r = .62) followed by the vision/beliefs/objectives

(r = .47) dimension. With regard to post-BABS scores, a

similar pattern of relationships was uncovered, suggesting

that schools with higher scores at post were associated with

significantly better implementation. The post-BABS score

had the largest correlation with the MTSS-BER total score

(r = .72), followed by the SET scores (r = .57). Both the

pre- and post-BABS score had the weakest correlation,

albeit significant, with the data-based decision making

variable from the MTSS-BER.

Research Objective #3

The last research question examined the extent to which the

SBI was associated with significant changes in school-level

beliefs and attitudes from pre to post and whether the

changes were predictive of implementation. The first step

consisted of examining the impact of the SBI to change

educators’ beliefs over time. To answer this question, we

conducted a paired sample t test on the pre–post data. The

null hypothesis was that there would be no difference

between pre- and post-BABS scores. Results indicated a

significant effect across time, t(61) = 7.28 (p \ .001), with the means revealing significant improvements in the aver-

age number of items with 80 % or more of staff endorsing

beliefs/attitudes in the favorable or supportive direction of

implementation from pre (m = 14.32) to post (m = 18.69).

The standardized mean difference effect size for this was

1.03, indicating a large effect according to Cohen’s (1988)

guidelines.

The next step was to examine whether the pre–post

changes in BABS scores were predictive of implementa-

tion outcomes. Unlike the analyses under Objective 2,

these analyses focused specifically on determining whether

the degree of change in beliefs was associated with

implementation outcomes at post. The results from these

correlational analyses revealed that BABS change scores

were significantly related to all five implementation out-

come variables (see Table 2). These findings suggested that

schools that changed the most on beliefs were associated

with higher-quality implementation across both global and

specific fidelity measures. These associations were the

strongest for the MTSS-BER total score and SET scores.

The final analysis consisted of performing a hierarchical

multiple regression with baseline beliefs being entered first

and then post-beliefs regressed on the MTSS-BER total

score. The results indicated that for the full model, baseline

and post-beliefs accounted for 64 % (R 2 ) of the variance in

the MTSS-BER score [F(2, 60) = 42.00, p \ .001]. Moreover, after controlling for the effects of baseline

beliefs, post-beliefs were able to significantly explain

variance in the implementation outcome variable (b = .66; t(1) = 6.58, p \ .001).

Discussion

The purpose of this study was to investigate the relevance

and importance of addressing educator beliefs and attitudes

in order to support the implementation of EBPs included

within a B-MTSS framework targeting students’ SEB

functioning. Several noteworthy findings emerged from this

research that have important implications for research and

practice. Perhaps most noteworthy were the findings from a

preliminary pre–posttest of the SBI which indicated that

educators’ beliefs significantly improved, and the changes

were associated with indicators of implementation fidelity.

Additional evidence in support of the importance of educator

beliefs to the implementation process was obtained from data

on coaches’ perceptions of the utility and importance of such

an undertaking, as well as prospective and concurrent rela-

tionships demonstrating the link between educator beliefs

and implementation outcomes.

Although the coaches did not directly implement the SBI,

they did help work with school teams to prepare and delivery

aspects of it to promote more supportive beliefs. As a result,

they had direct experience with the school team’s use of and

staff response to the SBI content. Coaches’ data indicated

that they perceived efforts the SBI to be helpful to reduce

educator resistance and ambivalence to change, extremely

useful to facilitate their own efforts to improve fidelity of

implementation, and important to promote a school context

that is supportive of EBP implementation. This finding was

important considering that coaches are often offered as one

of the main solutions to overcome barriers with regard to

implementation (Kretlow & Bartholomew, 2010). However,

prior research has shown that resistance or ambivalence to

coaching and consultation is commonplace (Gonzalez,

Nelson, Gutkin, & Shwery, 2004), underscoring the impor-

tance of identifying implementation interventions that can

facilitate effective coaching.

The correlational analyses revealed that school-level

beliefs and attitudes were associated with a global, team-

completed measure of implementation fidelity, as well as a

specific gold standard, observational measure of SW-PBIS

fidelity—the SET. This is notable considering the prior

research linking SET scores to relevant student outcomes

(Horner et al., 2004). Results indicated that, overall,

schools with higher levels of beliefs at pre were likely to be

associated with higher levels of implementation fidelity at

post. Moreover, findings indicated that schools with higher

beliefs at post were associated with higher implementation

by the end of Year 1 of the B-MTSS initiative. Considered

56 School Mental Health (2015) 7:49–60

123

together, these findings provided support for the link

between beliefs and implementation behaviors, and the

utility of such measures to better understand readiness for

the installation of EBPs, as well as effectively plan system

change efforts.

Findings also provided initial support for the SBI, as a

significant and relatively large change in school-level

beliefs from pre to post was observed. Specifically,

invoking belief and attitude change through a non-con-

frontational manner using social persuasion tactics repre-

sents a potentially promising way to change educator

behavior. Evidence also supported the importance of this

shift, as changes in beliefs from pre to post were signifi-

cantly and positively correlated with posttest fidelity,

indicating that schools that experienced the most change

were associated with the highest levels of implementation

at the end Year 1 of the initiative.

Despite the significant effect demonstrating change in

beliefs over time, there were still several schools with low

beliefs and low implementation outcomes at post (15 %; 10

of the 62 schools). It is important to uncover the factors

that serve as barriers to implementation efforts, particularly

when substantial resources (professional development,

coaching, and materials) were devoted to supporting

installation and inner organizational factors (beliefs of

administrators and teachers) were factored into the process.

One simple explanation is that the content never made its

way back to the school, because dissemination rested pri-

marily on the shoulders of the identified building-level

teams. This represents a known limitation of the train-the-

trainer approach to dissemination and implementation

(Pancucci, 2007), suggesting a need to better monitor and

support building-level teams with the dissemination pro-

cess. Another potential explanation is that the B-MTSS

work was not prioritized in relation to other initiatives

within the systems or that additional inner organizational

context factors influenced the implementation process. For

example, the roll out of Common Core and teacher eval-

uation systems were actively occurring in all school dis-

tricts, potentially creating time conflicts or competing

demands for participating teachers. Regardless of the

explanation, these schools may benefit from additional

implementation support enhancements, such as dedicated

leadership interventions (e.g., Aarons et al., under review),

closer monitoring and feedback for accountability, more

intensive coaching resources, and/or use of incentives to

increase motivation to implement practices.

Implications

One of the main implications that can be drawn from this

study is the importance of measuring and targeting edu-

cator beliefs and attitudes to reduce the implementation

gap. There is a need for well-validated and feasible mea-

sures that not only assess beliefs and attitudes, but capture

other factors associated with the school organizational

context. A suite of such measures could be used to better

examine readiness for implementation, as well as support

ongoing implementation and sustainment of school-based

EBPs. In alignment with the larger implementation science

literature, careful assessment of the organizational imple-

mentation context of schools should consider multiple

system levels and include perspectives of individual

teachers and administrators, as well as organizational

processes at the school and district levels (e.g., Aarons,

Erhart, & Farahnak, 2014). Ultimately, there is likely no

single implementation strategy that can produce successful

Tier 1 implementation in schools, but rather a collection of

strategies that need to be used to ensure high quality

implementation.

Although assessing this was not a primary goal of the

current project, beliefs and attitudes may also function

across multiple levels within the inner organizational

context of a school. For example, improved beliefs and

attitudes among both teachers and administrators may

result in a more supportive implementation climate, which

enhances the overall context for implementation. It is also

quite possible that improved attitudes and beliefs result in

more supportive leadership, which in turn triggers a

recursive process that leads to more consistent messaging

about implementation, greater time allocation for dissem-

ination, and enhanced accountability. The current study did

not attempt to differentiate beliefs according to different

groups of educators and measure other relevant inner

organizational factors potentially related to implementation

(e.g., strategic leadership, implementation climate).

Limitations and Future Research

As with most studies, this research is not without its lim-

itations. First, this was not a randomized control study and,

as such, is unable to provide causal inferences regarding

the effects of the SBI and the relationship between educator

beliefs and implementation of EBPs. Given the scope and

nature of the study, the intention was not to perform a

rigorous experimental evaluation, but rather to demonstrate

the importance of beliefs and attitudes to fidelity and the

potential promise of the SBI. Future studies should use

more tightly controlled procedures to replicate the findings

reported here and to demonstrate the causal effects of

implementation enhancement interventions that target

educators’ beliefs and attitudes.

Second, this study did not involve the collection and

analysis of student outcome data. As a result, it is unclear

whether educator beliefs and attitudes are predictive of

student outcomes or whether higher fidelity was associated

School Mental Health (2015) 7:49–60 57

123

with student improvement in this particular project. Future

research should conduct meditational analyses that exam-

ine the direct and indirect effects of educator beliefs and

attitudes on student outcomes. Third, the findings indicat-

ing that coaches found the SBI to be relevant and helpful in

facilitating their dissemination and implementation efforts

might be expected in light of cognitive dissonance and

social desirability. Moreover, no data were collected on

coaches’ adherence to procedures. Future research should

examine whether belief-enhancing interventions are caus-

ally linked to coaching effectiveness.

Fourth, although the MTSS-BER and SET have dem-

onstrated acceptability reliability in prior studies, no reli-

ability data were collected as part of this study, given that

school-based coaches were assisting with data collection.

The use of indigenous school staff as coaches, however,

represented a significant strength of this study, Finally, data

were only analyzed at the school level. Given the multi-

level structure of schools, future research should gather

individual (teacher and administrator) and school-level

data to examine factors that moderate educator beliefs and

ultimately differentially impact implementation outcomes.

Although the degree and ways in which educator beliefs

and attitudes contribute to the adoption and implementation

of school-based EBPs still remains somewhat unclear, this

study provided preliminary support for their role and

malleability during training and coaching efforts. This

research, along with other existing studies (Brackett,

Reyes, Rivers, Elbertson, & Salovey, 2012), suggests that

educator beliefs should be included as key variables in

theoretical models attempting to predict successful instal-

lation of EBPs and as strategies in approaches designed to

improve implementation success.

Acknowledgments This publication was made possible in part by funding from Grant Number K08 MH095939, awarded by the

National Institute of Mental Health (NIMH) to the second author. Dr.

Lyon is an investigator with the Implementation Research Institute

(IRI), at the George Warren Brown School of Social Work, Wash-

ington University in St. Louis, through an award from the National

Institute of Mental Health (R25 MH080916) and the Department of

Veterans Affairs, Health Services Research & Development Service,

Quality Enhancement Research Initiative (QUERI).

References

Aarons, G. A. (2006). Transformational and transactional leadership:

Association with attitudes toward evidence-based practice.

Psychiatric Services, 57(8), 1162–1169.

Aarons, G. A., Erhart, M. G., & Farahnak, L. R. (2014). The

implementation leadership scale (ILS): Development of a brief

measure of unit level implementation leadership. Implementation

Science, 9(1), 45.

Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a

conceptual model of evidence-based practice implementation in

public service sectors. Administration and Policy in Mental

Health and Mental Health Services Research, 38, 4–23.

Aarons, G. A., & Palinkas, L. A. (2007). Implementation of evidence-

based practice in child welfare: Service provider perspectives.

Administration and Policy in Mental Health & Mental Health

Services Research, 34, 411–419.

Aarons, G. A., & Sawitzky, A. C. (2006). Organizational climate

partially mediates the effect of culture on work attitudes and staff

turnover in mental health services. Administration and Policy in

Mental Health and Mental Health Services Research, 33(3),

289–301.

Ajzen, I., & Manstead, A. S. R. (2007). Changing health-related

behaviors: An approach based on the theory of planned behavior.

In K. van den Bos, M. Hewstone, J. de Wit, H. Schut, & M.

Stroebe (Eds.), The scope of social psychology: Theory and

applications (pp. 43–63). New York: Psychology Press.

Aronson, E. (1999). The power of self-persuasion. American

Psychologist, 54, 875–884.

Atkins, M. S., Frazier, S. L., Leathers, S. J., Graczyk, P. A., Talbott,

E., Jakobsons, L., … & Bell, C. C. (2008). Teacher key opinion leaders and mental health consultation in low-income urban

schools. Journal of Consulting and Clinical Psychology, 76,

905–908.

Beesdo-Baum, K., & Knappe, S. (2012). Developmental epidemiol-

ogy of anxiety disorders. Child and Adolescent Psychiatric

Clinics of North America, 21(3), 457–478.

Bowden, R. G., Lanning, B. A., Pippin, G., & Tanner, Jr., J. (2003).

Teachers’ attitudes towards abstinence-only sex education

curricula. Education, 123(4), 780–788.

Brackett, M. A., Rivers, S. E., Reyes, M. R., & Salovey, P. (2012).

Enhancing academic performance and social and emotional

competence with the RULER feeling words curriculum. Learn-

ing and Individual Differences, 22(2), 218–224.

Bradshaw, C. P., Pas, E. T., Bloom, J., Barrett, S., Hershfeldt, P.,

Alexander, A., … & Leaf, P. J. (2012). A state-wide partnership to promote safe and supportive schools: The PBIS Maryland

initiative. Administration and Policy in Mental Health, 39(4),

225–237.

Browning-Wright, D. & Cook, C. R. (2008). Behavior and RTI:

Integrating behavioral and academic support [PowerPoint

Slides]. Retrieved from http://www.pent.ca.gov/pos/rti/behavior

andrti_wright-cook.pdf.

Cialdini, R. (2001). Harnessing the science of persuasion. Harvard

Business Review, 79, 72–79.

Cialdini, R. (2009). Influence: Science and practice (5th ed.).

Pearson: Boston.

Cook, C. R. (2013). Measuring implementation of a comprehensive,

integrated MTSS framework targeting student behavior. Paper

presented at the National Association of School Psychologists

Annual Conference in Washington, DC.

Cook, C. R., Burns, M., Browning-Wright, D., & Gresham, F. M.

(2010a). Transforming school psychology in the RTI era: A guide

for administrators and school psychologists. Palm Beach: LRP

Publications.

Cook, C. R., Mayer, G. R., Browning-White, D., Kraemer, B., Gale,

B., & Wallace, M. D. (2012). Exploring the link between

evidence-based quality of behavior intervention plans, treatment

integrity and student outcomes under natural educational con-

ditions. The Journal of Special Education, 46, 3–16.

Cook, C. R., Volpe, R. J., & Livanis, A. (2010b). Constructing a

roadmap for future universal screening research beyond aca-

demics. Assessment for Effective Intervention, 35, 197–205.

Costello, E. J., Mustillo, S., Erkanli, A., Keeler, G., & Angold, A.

(2003). Prevalence and development of psychiatric disorders in

childhood and adolescence. Archives of General Psychiatry, 60,

837–844.

58 School Mental Health (2015) 7:49–60

123

Dart, E. H., Cook, C. R., Collins, T. A., Gresham, F. M., & Chenier, J.

S. (2012). Test-driving interventions to increase treatment

integrity and student outcomes. School Psychology Review,

41(4), 467–481.

Domitrovich, C. E., Bradshaw, C. P., Greenberg, M. T., Embry, D.,

Poduska, J. M., & Ialongo, N. S. (2010). Integrated models of

school-based prevention: Logic and theory. Psychology in the

Schools, 47, 71–88.

Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., &

Schellinger, K. B. (2011). The impact of enhancing students’

social and emotional learning: A meta-analysis of school-based

universal interventions. Child Development, 82(1), 405–432.

Dusenbury, L., Brannigan, R., Hansen, W. B., Walsh, J., & Falco, M.

(2005). Quality of implementation: Developing measures crucial

to understanding the diffusion of preventive interventions.

Health Education Research, 20(3), 308–313.

Eccles, M., Grimshaw, J., Walker, A., Johnston, M., & Pitts, N.

(2005). Changing the behaviour of healthcare professionals: The

use of theory in promoting the uptake of research findings.

Journal of Clinical Epidemiology, 58, 107–112.

Eccles, R., Pedersen, A., Regberg, D., Tulento, H., Borum, P., &

Stjarne, P. (2007). Efficacy and safety of topical combinations of

ipratropium and xylometazoline for the treatment of symptoms

of runny nose and nasal congestion associated with acute upper

respiratory tract infection. American Journal of Rhinolology,

21(1), 40–45.

Elias, M. J., Zins, J. E., Weissberg, R. P., Frey, K. S., Greenberg, M.

T., Haynes, N. M., … & Shriver, T. P. (1997). Promoting social and emotional learning: Guidelines for educators. Alexandria,

VA: Association for Supervision and Curriculum Development.

Feifer, C., Ornstein, S. M., Jenkins, R. G., Wessell, A. M., Corley, S.

T., Nemeth, L. S., … & Liszka, H. (2006). The logic behind a multi-method intervention to improve adherence to clinical

practice guidelines in a nationwide network of primary care

practices. Evaluation and the Health Professions. 29(1), 65–88.

Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention, and

behavior: An introduction to theory and research. Reading, MA:

Addison-Wesley.

Fishbein, M., Triandis, H. C., Kanfer, F. H., Becker, M., Middlestadt,

S. E., & Eichler, A. (2001). Factors influencing behavior and

behavior change. In A. Baum, T. A. Revenson, & J. E. Singer

(Eds.), Handbook of health psychology. Mahwah, New Jersey:

Lawrence Erlbaum Associates.

Fixsen, D. L., & Blase, K. A. (1993). Creating new realities: Program

development and dissemination. Journal of Applied Behavior

Analysis, 26, 597–615.

Fixsen, D. L., Blase, K. A., Naoom, S. F., & Wallace, F. (2009). Core

implementation components. Research on Social Work Practice,

19, 531–540.

Forman, S. G., Fagley, N. S., Chu, B. C., & Walkup, J. T. (2012).

Factors influencing school psychologists’ willingness to imple-

ment interventions. School Mental Health, 4, 207–218.

Forman, S. G., Shapiro, E. S., Codding, R. S., Gonzales, J. E., Reddy,

L. A., Rosenfield, S. A., … & Stoiber, K. C. (2013). Implemen- tation science and school psychology, 28(2), 77–100. doi:10.

1037/spq0000019.

Gonzalez, J. E., Nelson, J. R., Gutkin, T. B., & Shwery, C. S. (2004).

Teacher resistance to school-based consultation with school

psychologists: A survey of teacher perceptions. Journal of

Emotional and Behavioral Disorders, 12, 30–37.

Goodman, A., Joyce, R., & Smith, J. P. (2011). The long shadow cast

by childhood physical and mental problems on adult life.

Proceedings of the National Academy of Sciences, 108,

6032–6037.

Guskey, T. R. (1986). Staff developmental and the process of teacher

change. Educational Researcher, 15(5), 5–12.

Haney, J. J., Czerniak, C. M., & Lumpe, A. T. (1996). Teacher beliefs

and intentions regarding the implementation of science educa-

tion reform strands. Journal of Research in Science Teaching,

33, 971–993.

Haskins, R., & Loeb, S. (2007). A plan to improve the quality of

teaching in American schools. Future of Children, 17(1), 1–8.

Horner, R. H., Todd, A. W., Lewis-Palmer, T., Irvin, L. K., Sugai, G., &

Boland, J. B. (2004). The school-wide evaluation tool (SET): A

research instrument for assessing school-wide positive behavior

support. Journal of Positive Behavior Interventions, 6(1), 3–12.

Joyce, B., & Showers, B. (2002). Student achievement through staff

development (3rd ed.). Alexandria, VA: Association for Super-

vision and Curriculum Development.

Kincaid, D., Childs, K., Blase, K. A., & Wallace, F. (2007).

Identifying barriers and facilitators in implementing school-

wide positive behavior support. Journal of Positive Behavior

Interventions, 9, 174–184.

Kretlow, A. G., & Bartholomew, C. C. (2010). Using coaching to

improve the fidelity of evidence-based practices: A review of

studies. Teacher Education and Special Education, 33, 279–299.

Lyon, A. R., Stirman, S. W., Kerns, S. E. U., & Bruns, E. J. (2011).

Developing the mental health workforce: Review and applica-

tion of training strategies from multiple disciplines. Administra-

tion and Policy in Mental Health and Mental Health Services

Research, 38, 238–253.

Maag, J. W. (2001). Management of surface behavior: A new look at an

old approach. Counseling and Human Development, 33(9), 1–10.

Nelson, T. D., & Steele, R. F. (2007). Predictors of practitioner self-

reported use of evidence-based practices: Practitioner training,

clinical setting, and attitudes toward research. Administration

and Policy in Mental Health and Mental Health Services

Research, 34, 319–330.

Nunnally, J. S., & Bernstein, I. H. (1994). Psychological theory

(3rd ed.). New York: McGraw-Hillrobust.

Organ, D. W., Podsakoff, P. M., & MacKenzie, S. B. (2006).

Foundations for organizational science: Organizational citizen-

ship behavior: Its nature, antecedents, and consequences.

Thousand Oaks, CA: SAGE.

Owens, D. K., Lohr, K. N., Atkins, D., Treadwell, J. R., Reston, J. T.,

Bass, E. B., … & Helfand, M. (2014). Grading the strength of a body of evidence when comparing medical interventions: AHRQ

and the effective health care program. In Agency for Healthcare

Research and Quality. Methods Guide for Comparative

Effectiveness Reviews: Rockville, MD.

Pancucci, S. (2007). Train the trainer: The bricks in the learning

community scaffold of professional development. International

Journal of Humanities and Social Science, 2(1), 14–21.

Parcel, G. S., O’Hara-Tompkins, N. M., Harrist, R. B., & Basen-

Engquist, K. M. (1995). Diffusion of an effective tobacco

prevention program. Part II: Evaluation of the adoption phase.

Center for Health Promotion Research and Development, 10(3),

297–307.

Penuel, W., Fishman, B., Yamaguchi, R., & Gallagher, L. (2007).

What makes professional development effective? Strategies that

foster curriculum implementation. American Educational

Research Journal, 44(4), 921–958.

Powell, B. J., McMillen, J. C., Proctor, E. K., Carpenter, C. R.,

Griffey, R. T, Bunger, A. C., … & York, J. L. (2012). A compilation of strategies for implementing clinical innovations

in health and mental health. Medical Care Research and Review,

69(2), 123–157.

Pratkanis, A. R. (2007). The science of social influence: Advances and

future progress. New York: Psychology Press.

Slavin, R. (2002). Evidence-based education policies: Transforming

educational practice and research. Educational Researcher,

31(7), 15–21.

School Mental Health (2015) 7:49–60 59

123

Sprague, J., Cook, C. R., Browning-Wright, D., & Sadler, C. (2008).

Response to intervention for behavior: Integrating academic and

behavior supports. Palm Beach: LRP Publications.

Strein, W., Hoagwood, K., & Cohn, A. (2003). School psychology: A

public health perspective: Prevention, populations, and system

change. Journal of School Psychology, 41, 23–38.

Sugai, G., Horner, R. H., Dunlap, G., Hieneman, M., Lewis, T. J.,

Nelson, C. M., … & Ruef, M. (2000). Applying positive behavior support and functional behavioral assessment in schools. Journal

of Positive Behavior Interventions, 2(3), 131–143.

Sundle, J., Cialdini, R., Griskevicius, V., & Kenrick, D. (2012). The

world’s (truly) oldest profession: Social influence in evolution-

ary perspective. Social Influence, 7, 134–153.

Trout, A. K., Nordness, P. D., Pierce, C. D., & Epstein, M. H. (2003).

Research on the academic status of children with emotional and

behavioral disorders: A review of the literature from 1961 to

2000. Journal of Emotional and Behavioral Disorders, 11(4),

198.

60 School Mental Health (2015) 7:49–60

123