Discussion: Evidence-Based Practices: How Do You Know They Are Working?

The Journal of Special Education 2016, Vol. 50(1) 27 –36 © Hammill Institute on Disabilities 2015 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/0022466915613592 journalofspecialeducation.sagepub.com

Article

In the field of special education, a commitment to the logic and ethics of using research to inform decisions about prac- tice has been reflected in the field’s efforts to identify and use evidence-based practices (EBPs) as a standard for the profession (Council for Exceptional Children, 2014; Odom et al., 2005). As in other fields, this focus has led inexorably back to what some commentators have termed the “wicked” problem of implementation (Cook & Odom, 2013).

Fixen and his colleagues (following Rittel & Webber, 1973) described wicked problems as those that are “difficult to define and fight back when you try to solve them” (Fixen, Blaze, Metz, & Van Dyke, 2013, p. 218). Indeed, the obser- vation that “interests vested in the system-as-is suddenly appear and typically deter attempts to change the system” (Fixen et al., 2013, p. 218) has been made by ecologically oriented observers of human behavior since time of Marx (Bronfenbrenner, 1979; Lewin, 1951; Marx, 1888/1984). One implication of this view, of course, is that the problem of (non)implementation of EBP may be most usefully viewed not simply as a “deficit” in the knowledge, skills, or ideological commitments of practitioners but as a product of the set of social, organizational, and material conditions

that operate in a given human service setting. In this article, we draw on interviews conducted with special education practitioners to investigate how these kinds of contextual factors (and others) may affect the ways in which practitio- ners interpret and respond to contemporary press for imple- mentation of EBP.

We are by no means the first to recognize the importance of seeking practitioner perspectives in understanding the challenges of implementing EBP in special education. For example, Landrum, Cook, Tankersley, and Fitzgerald (2002) surveyed 127 teachers (60 special educators, 67 general edu- cators) to assess their views about the value of four sources of information about practice: university coursework,

613592 SEDXXX10.1177/0022466915613592The Journal of Special EducationHudson et al. research-article2015

1University of Washington, Seattle, USA 2Northern Illinois University, DeKalb, USA 3Central Michigan University, Mount Pleasant, USA 4American Institutes for Research, Washington, DC, USA

Corresponding Author: Roxanne F. Hudson, Area of Special Education, University of Washington, P.O. Box 353600, Seattle, WA 99195, USA. E-mail: [email protected]

A Socio-Cultural Analysis of Practitioner Perspectives on Implementation of Evidence-Based Practice in Special Education

Roxanne F. Hudson, PhD1, Carol A. Davis, EdD1, Grace Blum, MEd1, Rosanne Greenway, MEd1, Jacob Hackett, MEd1, James Kidwell, MEd1, Lisa Liberty, PhD1,2, Megan McCollow, PhD1,3, Yelena Patish, MEd1, Jennifer Pierce, PhD1,4, Maggie Schulze, MEd1, Maya M. Smith, PhD1, and Charles A. Peck, PhD1

Abstract Despite the central role “evidence-based practice” (EBP) plays in special education agendas for both research and policy, it is widely recognized that achieving implementation of EBPs remains an elusive goal. In an effort to better understand this problem, we interviewed special education practitioners in four school districts, inquiring about the role evidence and EBP played in their work. Our data suggest that practitioners’ responses to policies that press for increased use of EBP are mediated by a variety of factors, including their interpretations of the EBP construct itself, as well as the organizational conditions of their work, and their access to relevant knowledge and related tools to support implementation. We interpret these findings in terms of their implications for understanding the problem of implementation through a more contextual and ecological lens than has been reflected in much of the literature to date.

Keywords evidence-based practices, implementation, special education practitioners

28 The Journal of Special Education 50(1)

research journals, teaching colleagues, and in-service/pro- fessional development workshops. Their data indicated that research journals and university courses (presumably sources of relatively reliable information about EBP) were viewed as less useful, less trustworthy, and less accessible than either information from colleagues or information received via professional development. Similarly, Boardman, Argüelles, Vaughn, Hughes, and Klingner (2005) reported that teachers often expressed the belief that the extant research was not relevant to the populations they served in their classrooms, and reported relying on colleagues for rec- ommendations about practice.

In a more recent study, Jones (2009) investigated the views of 10 novice special educators regarding EBP. Based on interview, classroom observation, and rating scale data, Jones suggested that the novice teachers she studied fell into three broad groups. “Definitive supporters” expressed clear and positive views about the importance of research in decisions about classroom practice. “Cautious consumers” felt research could be useful, but often did not reflect char- acteristics and needs of their individual students. A third group, “The Critics,” expressed skepticism about the value of research for decisions about classroom practice.

Taken together, these studies (and others) provide a rather robust picture of the tensions between research and practice in special education. While significant variation exists among special education practitioners in their views about the value and relevance of research to their work in the classroom, many express more confidence in the knowl- edge and expertise of local colleagues than in information they might receive from university coursework and/or researchers. This result is consistent with research from other fields and suggests that much remains to be learned about the conditions under which practitioners utilize knowledge from research in decisions about practice (Aarons & Palinkas, 2007; Glasgow, Lichtenstein, & Marcus, 2003).

In our review of the special education research on this topic, we noted that most researchers have framed their analysis of practitioner perspectives related to implementa- tion of EBP in essentially individualistic and personological terms—placing teachers (and, in some cases, administra- tors) in the center of their analysis of the implementation process. For example, as noted earlier, Jones (2009) parsed individual teachers into groups such as “the Critics” and “the Supporters.” Also focusing on individual practitioners, Landrum et al. (2002) argued,

Only when we have confidence that teachers learn about empirically sound practice in both their initial preparation and ongoing professional development, and that their skills reflect this training, can we predict that students with disabilities will be afforded the most appropriate learning opportunities available. (p. 48)

We do not entirely disagree with these conclusions, and others like them that underscore the importance of persono- logical variables (e.g., practitioner knowledge, prior train- ing, attitudes) affecting implementation of EBP. But we would also argue that in foregrounding characteristics of individual practitioners as a focus of analysis, these studies reflect a set of implicit assumptions about the nature of practice and how it is constructed that narrows our view of the problems of implementation, and the range of actions to be considered in engaging those problems. In the present study, we follow recent recommendations (Harn, Parisi, & Stoolmiller, 2013; Klingner, Boardman, & McMaster, 2013; Peck & McDonald, 2014) in undertaking a more holistic and contextual approach to understanding how practitioner perspectives on EBP are shaped by the conditions in which they work.

Theoretical Framing

In conceptualizing “a more contextual” approach to under- standing practitioner interpretation and implementation of EBP, we drew on some of the precepts of sociocultural the- ory as a general framework for investigating ways in which social and material conditions shape workplace learning and practice (Billett, 2003; Engeström, 2001; Scribner, 1997; Vygotsky, 1978). Our choice of a sociocultural per- spective was based on several of its key precepts that we believed would be useful in understanding practitioner per- spectives on implementation of EBP. First, sociocultural theory foregrounds analysis of relationships between indi- vidual and collective dimensions of social practice—in this case, the analysis of the transactions that take place between individual practitioners and the organizations in which they work (Engeström, 2001). Second, this view assumes that human thought processes (including, of course, one’s views about EBP) are shaped by the demands of the practical activities in which people are regularly engaged. A third assumption of this stream of sociocultural theory is that par- ticipation in social practice is affected by the affordances and constraints of the conceptual and material tools avail- able (e.g., the characteristics and representations of EBP available in local school districts and other professional resources; Falmagne, 1995; Leontev, 1975/1978; Scribner, 1997). Overall, the sociocultural perspective suggests the value of undertaking a more focused analysis of the social and organizational conditions in which decisions about practice are made than has been reflected in much of the extant research on the problem of implementation. We used the following research questions to guide our inquiry:

Research Question 1: How do special education practi- tioners interpret the meaning of EBP in the context of decisions they make about curriculum and instruction?

Hudson et al. 29

Research Question 2: What contextual factors are asso- ciated with practitioner interpretations of the role EBP can and should play in their decisions about instruction?

Method

We used a qualitative methodology (Merriam, 2009) to investigate the perspectives—that is, the values, beliefs, and attitudes—held by special education practitioners with regard to their views about EBP, and the role research played in their decisions about curriculum and instruction. We elected this methodological approach because of the hypothesis-generating, rather than hypothesis-testing, pur- poses of the study (Glaser & Strauss, 1967).

Participants

A total of 27 special education practitioners participated in our study. We contacted directors of special education via email and invited participation from four school districts in the Seattle/Puget Sound area. Demographics for these dis- tricts are presented in Table 1.

Teacher participants were nominated by special educa- tion directors, who were asked to identify individuals they believed would be interested in being interviewed for the study. In each district, we requested nominations of teachers working in three types of programs or settings: resource rooms serving students with a wide range of disability labels placed primarily in general education classrooms, self-contained classes serving students with emotional/ behavioral disabilities (EBD), and self-contained class- rooms serving students with low-incidence developmental disabilities. Table 2 reports the number, working context, and experience level of study participants in each of the dis- tricts in which we collected data.

Data Collection and Analysis

Interviews. The primary data source for our study consisted of face-to-face interviews we conducted individually with the 27 special educators who agreed to participate in the study. We used semistructured interview protocols for each of the four types of practitioners we interviewed: special

education directors, resource room teachers, EBD teachers, and teachers of students with low-incidence developmental disabilities. While the protocols for administrators and teachers varied in some ways, both were structured to pro- ceed from general, context-descriptive questions such as “Tell me about the work you do,” to more focused questions about daily practice (“Tell me about a typical day in your classroom”). We asked each informant to define the term EBP and tell us what it meant to them in terms of their deci- sions about curriculum and instruction. Interview protocols also included a series of questions about district policies related to EBP in both general and special education, and how these affected the decisions our informants made in the classroom. Interviews were generally between 45 min to an hour in length. Interviews were audio-recorded and subse- quently transcribed verbatim for analysis. Transcripts were entered into a web-based platform for qualitative and mixed-method data analysis (http://www.dedoose.com).

Data analysis. We used the standard procedures for induc- tive data analysis described by Charmaz (2002), Strauss and Corbin (1997), and others. Thus, we began our analysis by having each of the 11 members of our research team read through the interview transcripts, identifying text segments of potential relevance to our research questions. Each of these segments was tagged using low inference descriptors, such as “classroom assessment” or “progress monitoring.” Members of the research team then met to discuss examples of the text segments they had tagged, identifying and defin- ing codes emerging from individual analysis to be formal- ized and used collectively. The remainder of the interviews were then coded, followed by an additional round of team meetings in which examples of each code were discussed, with some codes combined, others modified or deleted based on their perceived value relative to our research ques- tions. A set of interpretive categories were developed through this process which were used to aggregate coded data segments and which became the basis for further anal- ysis. These categories were then used as a basis for develop- ing a series of data displays (Miles & Huberman, 1994) organized by district and by each type of participant (i.e., resource room teachers, special education directors, etc.). Team members met to discuss the implications of these analyses and to develop a set of analytic memos which inte- grated the categorical data into larger and more interpretive case summaries. These summaries were used to develop the set of cross-case findings described below.

Results

Our findings suggest that personal characteristics (particu- larly values and beliefs about EBP), the features of organi- zations (particularly practitioner positionality within these organizations), and access to relevant tools all affected the

Table 1. School District Characteristics.

District Enrollment Special education enrollment (%)

Students eligible for free or reduced-price

meals (%)

A 18,123 9.70 22.10 B 20,659 13.60 35.10 C 17,973 13.60 66.90 D 8,920 12.40 26.00

30 The Journal of Special Education 50(1)

ways practitioners interpreted the relevance of the EBP to decisions they made about practice. We interpreted these as dimensions of practical activity that were inseparable and mutually constitutive (Billett, 2006). As depicted in Figure 1, our data suggest these factors operate in a highly interdependent manner. We use this conceptual model to understand both the points of the triangle and the interac- tions that take place between points as represented by the lines of the triangle.

In the following sections, we will present findings both related to the points of the triangle and the intersections of elements. First, we use excerpts from our interviews to illustrate how the practitioners we interviewed interpreted the idea of EBP, the organizational contexts they worked in, and the tools and resources available to them. Second, we present findings that illuminate the connections and interac- tions between them.

People: Practitioner Definitions of EBP

We asked each of our informants how they defined EBP in the context of their work in special education. The predomi- nance of responses to this question reflected the notion that EBP meant that “someone” had researched a specific pro- gram or practice and found it to be effective:

There’s obviously been research and studies so what I picture in my mind is that they have a curriculum and they conduct a study where they have kids who participate in the study and then they probably have some pre- and posttest to see if they’ve made gains.

I’d say evidence-based would be like, that it’s been tried in lots of different settings, across you know lots of different populations and there’s been demonstrated success using that curriculum or whatever the thing is you’re talking about, you know, the social skills sheet or something. So it’s used with lots of people and over different settings.

We noticed that our participants typically defined EBP in ways that emphasized its external origins, and its ostensive function as a “prescription” for their practice, rather than as

a resource for their own decision making (Cook & Odom, 2013). In some cases, this interpretation was also congruent with the stance taken by district administrators:

We have adults that want to continue to do what they’ve done in the past. And it is not research-based nor if you look from a data perspective has it been particularly effective and that’s not going to happen and we say, “This is the research, this is what you’re going to do.” (Special Education Director, District A)

This strong ideological commitment to use of EBP in the classroom was shared by some teachers:

I believe that by using research-based instruction, and teaching with fidelity, then you’re more likely to have an outcome that is specific to the research, as long as we use the curriculum as it’s designed. Um, I think it’s vital, I think it’s vital that we are not pulling things out of a hat, that we are using. (Resource Room Teacher, District A)

More often, however, we found that practitioner views about research in general, and the value of EBP in decision making about classroom practice in particular, were more ambivalent. Perhaps the most widely shared concern about EBP expressed by our informants had to do with the ten- sions they perceived between the “general case” and the specifics of local context, including the special needs of the children served in the schools and classrooms in which they worked (Cook, Tankersley, Cook, & Landrum, 2008). While the value of research and the relevance of EBP were often acknowledged in abstract terms, both teachers and administrators were quick to identify what they perceived to be limitations in the relevance of research for local deci- sion making and special populations:

. . .well what makes me question it—I’m always curious about what the norm population is because it is talking about typically developing kids and research-based practices that are used for those types of kids. It’s very different for my kids. So when I’m looking at an evidenced-based practice I want to be clear on what evidence [is about] Gen Ed versus the Special Ed population. (Self-Contained Classroom Teacher, District B)

Table 2. Participant Characteristics.

Participants Number of

participants per district

Median years in position District

Special education director

EBD teacher

Resource room teacher

Self-contained teacher

A 1 1 2 2 6 7 B 1 1 2 3 7 10 C 1 2 2 2 7 6 D 1 2 2 2 7 6

Note. EBD = emotional/behavioral disabilities.

Hudson et al. 31

For many teachers, ambivalence about EBP included particular tension about who makes decisions about the rel- evance of evidence to their classroom practice. These teach- ers often made reference to local perspectives as “forgotten” or “overlooked” in decisions about practice:

. . . evidence-based is very important because you do need to look at what you’re doing but there is just the day-to-day knowledge that is overlooked in the evidence-based piece. (Self-Contained Classroom Teacher, District B)

Most of the teachers and many administrators we inter- viewed appeared to locate the authority for making evi- dence-based decisions about curriculum and instruction with the district central office or with “general education.” For example, one director of special education reported, “for our resource room students . . . we always start with the Gen Ed and then if we have curriculum, if a part of that cur- riculum has a supported intervention component to it we start with that.” Many teachers similarly viewed the locus of decisions about curriculum and instruction as external to their classrooms. As a Resource Room Teacher in District D puts it,

They tell us what to teach and when to teach it. I mean, we have a calendar and a pacing guide. We can’t, we really don’t make the decisions too much. I would hope . . . that it’s supported and making sure that students learn but I don’t really know.

In some cases, these teachers expressed confidence that the judgments of district curriculum decision makers were grounded in appropriate evidence:

I kind of just trust that the district is providing me with evidence-based stuff. So I’m trusting that the curriculum that they’ve chosen and that my colleagues have done test work on is really what they say it is. (EBD Teacher, District D)

However, in other cases, teachers expressed more skepti- cal views about the trustworthiness of the data district offi- cials used to make decisions about curriculum:

. . . over the years we’ve had so many evidence-based, research based and so many changes, that . . . if you just want my honest [opinion] . . . I know that there’s data behind it, but if it’s evidence based or research based, why are we always changing? (EBD Teacher, District B)

Figure 1. Relationships between people, organizations, and tools. Adapted from McDiarmid & Peck (2012).

32 The Journal of Special Education 50(1)

To summarize, similar to earlier studies (Boardman et al., 2005; Jones, 2009; Landrum et al., 2002), we found that the personal characteristics of practitioners—that is, their experiences, values, beliefs, and attitudes—functioned as a powerful filter through which they interpreted the meaning of EBP and evaluated the relevance of this con- struct for their decision making about curriculum and instruction. Practitioner definitions of EBP often reflected the assumption that the locus of authority regarding EBP lies outside the classroom, and the ostensive function of EBP was to provide prescriptions for classroom practice.

In the following sections, we report findings related to our second research question, describing ways in which contextual features such as organization and access to tools and resources may influence the way practitioners interpret the value and relevance of EBP in their daily work.

Organizational Contexts of EBP

Our data suggest that our interviewees’ views about the value and relevance of evidence in decision making about practice were often part of a larger process of coping with the organizational conditions of their work. Several specific issues were salient in the interviews we conducted. One of these, of course, had to do with district policies about evi- dence-based decision making (Honig, 2006). In some dis- tricts, special education administrators described strong district commitments related to the use of research evidence in decision making:

In this district, it’s [EBP] becoming really big. You don’t ever hear them talk about any initiative without looking at the research and forming some sort of committee to look at what practices are out there and what does the research tell us about it. And then identifying what are the things we’re after and how well does this research say they support those specific things we want to see happen. I would say that work has started, and that is the lens that comes from Day One of anything we do. (Special Education Administrator, District C)

However, strong district commitments to evidence-based curriculum decisions in general education were sometimes viewed as raising dilemmas for special education teachers. A teacher of students with emotional and behavioral prob- lems in District B described the problem this way:

I try to make our classroom setting as much like a general Ed classroom as I can, because the goal is to give them strategies to work with behaviors so that they can function in Gen Ed classrooms. Which is a challenge, because they were in Gen Ed classrooms before they came here, so something wasn’t working.

Some special education teachers described being caught between curriculum decisions made in general education and practices they saw as more beneficial for their students with disabilities:

. . . if (general education teachers) change their curriculum then I need to follow it so my kids can be a part of it. Especially with my kids being more part of the classroom. So you know 4 years ago I was not doing the Math Expressions, and now I am doing the Math Expressions and it’s hard because I’m trying to follow the Gen Ed curriculum and there’s times where the one lesson ends up being a 2- or 3-day lesson with some added worksheets because they just aren’t getting the skill and, being a spiraling curriculum, it goes pretty fast sometimes too. (Self-Contained Teacher, District B)

While the tensions between curriculum models in gen- eral education and special education (with each claiming its own evidentiary warrants) were problematic for many of the resource room teachers we interviewed, these dilemmas were less salient to teachers of children in self-contained classrooms, including those serving students with EBD and those serving students with low-incidence disabilities. Instead, the challenges that these teachers described had to do with isolation and disconnection from colleagues serv- ing students like theirs. A Self-Contained Classroom Teacher, District B, said, “Well, this job is very isolating. I’m the only one that does it in this building . . . so uh I’m kind of alone in the decisions I make.” Another Self- Contained Classroom Teacher, District D, said,

Sometimes it makes me feel like it’s less than professional . . . I don’t know, I just sometimes wish that, I feel like there’s not always a lot of oversight as far as what am I doing. Is there a reason behind what I’m doing? And did it work? I wish there was more.

In cases where teachers felt isolated and disconnected from district colleagues, they often reported relying on other self-contained classroom teachers for support and consultation, rather than resources in their district or from the research literature:

When you’re in an EBD class you can get pretty isolated . . . but the other beauty of being in an EBD room is you have other adults in the room that you can talk to, or they can have other ideas, or you can call other teachers. (EBD Teacher, District B)

Resources and Tools Related to EBP

District commitments to use of EBPs in the classroom were in many cases accompanied by allocation of both material and conceptual resources. The resources and supports most often cited by both administrators and teachers in this con- nection were focused on curriculum materials:

Susan, who is our Special Ed curriculum developer, and Louisa, who’s our curriculum facilitator . . . they recognize that we’ve shifted practice to a really research-based practice . . . We never really did this [before] we just bought books and

Hudson et al. 33

stuff. And I said, “Well, I don’t operate that way. We’re going to shift practice and I’m going to help you” . . . and she has actually been very, very successful at reaching out and capturing the attention of folks, authors that publish . . . and some other materials and some other research and then digging deep. (Special Education Director, District A)

I think there’s something really powerful about having that scope and sequence and that repetition that gradually builds on itself. Yeah so instead of me trying to create it as I go, having that research-based program, and of course I’m going to see if it’s not working, then I’m flexible to change it, but I’m going to have a good base to at least start with. (Resource Room Teacher, District B)

While district curriculum resources (which were often assumed to be evidence-based) were important tools for many resource room teachers we interviewed, both teachers and administrators expressed frustration about what they viewed as a paucity of evidence-based curriculum and instructional resources for students in self-contained pro- grams, particularly those serving students with low-inci- dence disabilities:

There’s not a lot of curriculum out there for the self-contained classrooms. So we do have some, some for our more mild self- contained programs, specific reading, writing, math curriculums. But we also made our own library we call it “The Structured Autism Library” . . . it’s a library of materials we have online that we’ve developed ’cause you can’t really pull anything off the shelves for those kinds of kids. (Special Education Director, District D)

. . . in self-contained settings in Ocean View, I think that I am expected to . . . sort of use Gen Ed assessments, but in Kindergarten they already test out of those, they already fail so miserably at those. I am then the expert because there’s no assessments that assess these kids’ growth so I make my own assessments. (Self-Contained Classroom Teacher, District D)

In the context of these perceptions about the lack of relevant evidence-based resources, we found that only a few teach- ers undertook individual efforts to locate and use available research. Those who did often encountered considerable difficulty in locating relevant research resources:

If I’m implementing something I’m not so good at then I’ll go do some reading on it. The Autism modules online are helpful. I’d say mostly every time I try and type something in for a problem I’m having without knowing if there is any research on it, I can never find it, hardly ever. (Self-Contained Classroom Teacher, District C)

More often, teachers adopted a “progress monitoring” approach to evidence and decision making: “We take what we learn about the kids and we change how we instruct them, so that’s kind of what we do every day . . . based on

each kid and their individual performance” (Self-Contained Classroom Teacher, District B).

After our examination of each separate element of the system, we analyzed the interactions between elements to understand the transactional, contextual nature of practitio- ners’ understanding of EBPs.

A Holistic View: Relationships Between People, Tools, and Organizations

Our data suggest that the positions practitioners occupied within their schools and districts had considerable influence on their access to useful resources and tools related to EBP. The nature of the tools available to them, in turn, affected practitioner experiences with and beliefs about the value and relevance of EBPs in the context of decisions they were making. Our findings are summarized below in terms of some specific hypotheses regarding the ways in which these dimensions of practice are related to one another.

Hypothesis 1: Practitioner beliefs about the value and relevance of EBP are shaped by the affordances and con- straints of the tools and resources they use for decision making.

Similar to other researchers (e.g., Boardman et al., 2005; Landrum et al., 2002), we found some skepticism about the practical relevance of research for local decision making across all of the practitioner groups we interviewed. However, we also noted that this view was most prevalent among self-contained classroom teachers and particularly among teachers of students with low-incidence disabilities. In many cases, both teachers and administrators working with students with low-incidence disabilities expressed the belief that research on “kids like ours” did not exist. For example, a Self-Contained Classroom Teacher in District B explained her stance about research and practice this way:

Well it’s just that there’s not a lot out there for me and maybe it’s hard to find. I feel like through my multiple programs, I’ve looked at a lot and there’s just not a lot out there. And I know why—this is a small fraction of the population.

In some cases, teachers in our study described themselves as having access to curriculum tools they considered to be evidence-based for some populations, but inappropriate for the specific students they served.

Hypothesis 2: Practitioner access to relevant tools and resources is affected by the position they occupy within the school and the district.

The self-contained classroom teachers we interviewed often described themselves as being extremely isolated, and as

34 The Journal of Special Education 50(1)

having relatively little access to tools and resources (research studies, evidence-based curriculum, and relevant evidence- based professional development) they viewed as useful for the students they taught. We found that teachers who worked in positions that were relatively close to general education often had access to more tools and resources (evaluation instruments, curriculum guides, professional development) related to EBP, but were also more likely to encounter ten- sions between mandates from general education, and prac- tices they viewed as appropriate for students with special education needs. The effects of organizational position were also mediated by district policies and practices around col- laboration. For example, one district had strategically devel- oped specific organizational policies and practices to support collaboration among their self-contained classroom teachers. One of the teachers in this district commented on the value of these practices as a support for implementation of EBPs:

I definitely believe in the PLC (professional learning community) model and just sharing ideas and having, taking on new strategies, and monitoring our own use of them. I think that is kind of the future. I think you’re going to get more buy-in, than a sit and get PD session on evidence-based practice. I think its been proven actually, that you just sit and get it and then you don’t have to use it, no one checks in with you to make sure you are using it. So making groups of people accountable (to each other) makes a ton of sense to me. (Self- Contained Classroom Teacher, District B)

More often, however, district policies, resources, and sup- ports related to EBP were reported to be focused on curricu- lum and instructional programs for students with high-incidence disabilities.

Teachers in self-contained classrooms often found these tools and resources, and the idea of EBP to be of little direct value to their daily work.

Hypothesis 3: How practitioners define EBP affects how they interpret the value and relevance of tools and other organizational resources available to them related to EBP.

As we explain further later on, most of the teachers and administrators we interviewed defined EBP primarily in terms of prescriptions for practice that were made by exter- nal authorities. Many of these informants were also those who expressed ambivalence, and often outright skepticism, about the value of EBPs for their work. In contrast to this pattern, a few of the practitioners we talked with appeared to have a more nuanced way of defining EBP. In these cases, EBP was defined less as a prescription for practice than as a resource for decisions they would make in the classroom:

I think it would be wonderful to be informed of that research, and the best teacher would have all that information, and be able to look at the kid, and provide them an opportunity with a program that is research-based and validated and everything,

and look at how the child is responding to the program, give it a little bit of time, make sure you’re delivering it with authenticity and the way it’s supposed to be delivered, you know, give it 3–4 weeks, and if it’s not working you need to find something else. (Resource Room Teacher, District B)

Discussion

Over the last decade, the notion of EBP has become one of the most influential policy constructs in the field of special education. In this study, we sought to improve our under- standing of the ways practitioners define the idea of EBP and interpret its relevance in the contexts of their daily practice. In addition, we hoped to extend previous research by learn- ing more about some of the contextual factors that might influence practitioner views about EBP. To investigate these two issues, we conducted interviews with special education professionals in four local school districts, including those occupying positions as directors of special education, resource room teachers, self-contained classroom teachers of students with emotional and behavioral disabilities, and teachers of students with low-incidence developmental dis- abilities. Our analysis of these data was guided by some gen- eral precepts drawn from sociocultural theory (Chaiklin & Lave, 1993; Vygotsky, 1978), particularly the idea that social practice can be understood as a process in which individuals are continually negotiating ways of participating in collec- tive activity (Nicolini, Gherardi, & Yanow, 2003).

We found that the practitioners we interviewed often defined EBP in ways that externalized the locus of authority for what constituted relevant evidence for practice as the results of “studies someone had done.” Ironically, this view also appeared to be the focus of considerable tension and con- cern, as many of the practitioners we interviewed expressed the idea that research did not adequately reflect the characteris- tics of the students they served nor the contexts of the decisions about practice they were charged with making on a daily basis.

Our findings extend previous research (e.g., Boardman et al., 2005; Jones, 2009) by identifying some specific hypotheses regarding the ways practitioner interpretations and responses to contemporary pressures for use of EBPs may be shaped by the organizational contexts of their work, as well as the professional resources (research, curriculum tools, training) that are available to them. We consider some implications of these general ideas below.

Context, Participation, and Practice

In the interviews we conducted, we were continually impressed with the ways in which practitioners’ ideas and decisions about their ways of doing things were shaped by the resources available to them, the policies and practices of the schools and districts in which they worked, and the values and norms of their colleagues. The view which emerges from these interviews suggests many ways in which decisions

Hudson et al. 35

about practice—that is, decisions about how to do things in the classroom—are distributed across multiple participants and multiple settings. We believe this perspective is in tension with much of the literature on implementation of EBP, which tends to reflect a much more individualistic view of the prob- lem, with an accompanying set of intervention strategies focused primarily on individual practice (e.g., Jones, 2009; Landrum et al., 2002). Just to be clear, we are not suggesting that researchers are naïve about what are often conceptualized as “contextual variables” affecting individual practice.

The hypothesis is actually more provocative—that is, that practice itself may be understood as essentially social, relational, and distributive in nature (Edwards, 2012). Such a view does not suggest that changing individual behavior is unimportant, much less that the “real” issues reside only in the “system” (Fixen et al., 2013). Our data suggest, rather, that our understanding of the problems of implementing EBP may be improved by more focused investigation of the ways in which individual and collective dimensions of pro- fessional practice create and sustain one another (Nicolini et al., 2003; Peck, Gallucci, Sloan, & Lippincott, 2009).

A recurring theme in the interview data we collected had to do with the ways that practitioner definitions of EBP, and their views of the relevance of this construct for their work, were confounded with their concerns about the locus of power and control over decision making about classroom practice. In some cases, these concerns appeared to be more salient to our informants than those about the substance of the decisions themselves, and reflected a sense of practitio- ner alienation from researchers (and, in some instances, administrators). Our interpretation of these concerns was that they represented fundamental and perennial tensions about power and authority over decisions in the classroom. Issues of power and control are by no means limited to the problems of implementation of EBP, but are, rather, one of the most pervasive challenges to any kind of systemic change effort in educational organizations (van den Berg, 2002). We hasten to point out that we do not interpret this finding in personological terms, that is, we do not see these practitioners as simply “burned out” or “resistant.” Rather, we view their responses to be, at least in part, a reflection of the tensions they experience between the organizational contexts of their work, the tools and resources available to them, and the demands of their work in the classroom. These hypotheses appear worthy of further investigation.

Professional Development and Implementation of EBP

The interviews we conducted raise several questions about training and professional development related to implemen- tation of EBP. First, we found that most of the 27 teachers and administrators we interviewed were ill-prepared to understand and respond to the essential epistemological tensions between the “general case” and the “local case”

that inevitably accompany decisions about practice (Harn et al., 2013). As we noted above, this sample of practitioners more often interpreted research as a prescription for practice, rather than as a source of hypotheses about what might work with their students (Cook & Odom, 2013). This “top-down” view of EBP appeared to be distributed equally across teach- ers and administrators in our study. As we have noted, this interpretation of the EBP construct was often associated with skeptical views about the value of EBPs for decisions about classroom practice. This finding, if replicated through addi- tional research with larger and more diverse samples, may be an important focus for professional development.

Neither the teachers nor the administrators we inter- viewed appeared well prepared to access tools that were not immediately available in the districts in which they worked. For example, very few of the teachers and administrators we interviewed seemed to be aware of the tremendous changes in online accessibility of library resources that has taken place in the past few years, as well as the explosion of online training resources related to EBPs in special educa- tion (e.g., http://iris.peabody.vanderbilt.edu/index.html). While the restricted sample of practitioners we interviewed mandates caution in generalizing from these findings, these data do suggest the value of undertaking a broader investi- gation of the kinds of preparation and support practitioners receive via both preservice and professional development activities related to online access to tools for evidence- based decision making about curriculum and instruction.

We conclude by noting the congruence of our emerging hypotheses regarding the socially negotiated and distributed nature of practice with findings from other educational set- tings (Ingram, Seashore Louis, & Schroeder, 2004; Peck & McDonald, 2014; Spillane, Halverson, & Diamond, 2001) and other fields of practice (Nicolini et al., 2003). These studies, and our present findings, suggest that achieving implementation of EBP may be viewed most productively not simply as a problem of individuals acquiring new knowledge, skills, and dispositions but rather as a broader social process in which individual practitioners participate in the collective negotiation of decisions about practice in the context of the tools available to them and the organiza- tional conditions in which they work.

Acknowledgments

We are thankful to the teachers and principals who generously shared their thinking with us.

Authors’ Note

The content is solely the responsibility of the authors and does not necessarily represent the official views of the Office of Special Education Research.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

36 The Journal of Special Education 50(1)

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Research reported here was supported by Award Number H325D100072A.

References

Aarons, G. A., & Palinkas, L. A. (2007). Implementation of evi- dence-based practice in child welfare: Service provider per- spectives. Administration and Policy in Mental Health and Mental Health Services Research, 34, 411–419.

Billett, S. (2003). Sociogeneses, activity and ontogeny. Culture & Psychology, 9, 133–169.

Billett, S. (2006). Constituting the workplace curriculum. Journal of Curriculum Studies, 38, 31–48.

Boardman, A. G., Argüelles, M. E., Vaughn, S., Hughes, M. T., & Klingner, J. (2005). Special education teachers’ views of research-based practices. The Journal of Special Education, 39, 168–180.

Bronfenbrenner, U. (1979). Contexts of child rearing: Problems and prospects. American Psychologist, 34, 844–850.

Chaiklin, S., & Lave, J. (1993). Understanding practice: Perspectives on activity and context. Cambridge, UK: Cambridge University Press.

Charmaz, K. (2002). Qualitative interviewing and grounded theory analysis. In J. F. Gubrium & J. A. Holstein (Eds.), Handbook of interview research: Context & method (pp. 675–694). London, England: SAGE.

Cook, B. G., & Odom, S. L. (2013). Evidence-based practices and implementation science in special education. Exceptional Children, 79, 135–144.

Cook, B. G., Tankersley, M., Cook, L., & Landrum, T. J. (2008). Evidence-based practices in special education: Some practical considerations. Intervention in School and Clinic, 44, 69–75.

Council for Exceptional Children. (2014). Standards for evidence- based practices in special education. Reston, VA: Author.

Edwards, A. (2012). The role of common knowledge in achieving collaboration across practices. Learning, Culture and Social Interaction, 1, 22–32.

Engeström, Y. (2001). Expansive learning at work: Toward an activity theoretical reconceptualization. Journal of Education and Work, 14, 133–156.

Falmagne, R. (1995). The abstract and the concrete. In L. Martin, K. Nelson, & E. Tobach (Eds.), Sociocultural psychology: Theory and practice of doing and knowing (pp. 205–228). Cambridge, UK: Cambridge University Press.

Fixen, D., Blaze, K., Metz, A., & Van Dyke, M. (2013). Statewide implementation of evidence-based programs. Exceptional Children, 79, 213–230.

Glaser, B., & Strauss, A. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine Press.

Glasgow, R. E., Lichtenstein, E., & Marcus, A. C. (2003). Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. American Journal of Public Health, 93, 1261–1267.

Harn, B., Parisi, D., & Stoolmiller, M. (2013). Balancing fidel- ity with flexibility and fit: What do we really know about fidelity of implementation in schools? Exceptional Children, 79, 181–193.

Honig, M. (2006). Complexity and policy implementation. In M. Honig (Ed.), New directions in education policy implemen- tation: Confronting complexity (pp. 1–25). Albany: State University of New York Press.

Ingram, D., Seashore Louis, K., & Schroeder, R. (2004). Accountability policies and teacher decision making: Barriers to the use of data to improve practice. Teachers College Record, 106, 1258–1287.

Jones, M. L. (2009). A study of novice special educators’ views of evidence-based practices. Teacher Education and Special Education, 32, 101–120.

Klingner, J. K., Boardman, A. G., & McMaster, K. L. (2013). What does it take to scale up and sustain evidence-based prac- tices? Exceptional Children, 79, 195–211.

Landrum, T. J., Cook, B. G., Tankersley, M., & Fitzgerald, S. (2002). Teacher perceptions of the trustworthiness, usabil- ity, and accessibility of information from different sources. Remedial and Special Education, 23, 42–48.

Leontev, A. N. (1978). Activity, consciousness, and personality (M. J. Hall, Trans.). Englewood Cliffs, NJ: Prentice Hall. (Original work published 1975)

Lewin, K. (1951). Field theory in social science: Selected theo- retical papers. Oxford, UK: Harpers.

Marx, K. (1984). Thesis on Feuerbach. In K. Marx & F. Engels (Eds.), The individual and society (pp. 5–7). Moscow, Russia: Progress. (Original work published 1888)

McDiarmid, G. W. & Peck, C. (2012, April). Understanding change in teacher education programs. Paper presented at the Annual Meeting of the American Educational Research Association, Vancouver, B.C., Canada.

Merriam, S. B. (2009). Qualitative research: A guide to design and implementation. Hoboken, NJ: John Wiley.

Miles, M. B., & Huberman, A. M. (1994). Qualitative data analy- sis: An expanded sourcebook. Thousand Oaks, CA: SAGE.

Nicolini, D., Gherardi, S., & Yanow, D. (2003). Knowing in organi- zations: A practice-based approach. Armonk, NY: M.E. Sharpe.

Odom, S. L., Brantlinger, E., Gersten, R., Horner, R. H., Thompson, B., & Harris, K. R. (2005). Research in special education: Scientific methods and evidence-based practices. Exceptional Children, 71, 137–148.

Peck, C. A., Gallucci, C., Sloan, T., & Lippincott, A. (2009). Organizational learning and program renewal in teacher edu- cation: A socio-cultural theory of learning, innovation and change. Educational Research Review, 4, 16–25.

Peck, C. A., & McDonald, M. (2014). What is a culture of evi- dence? How do you get one? And . . . should you want one? Teachers College Press, 116(4), 1–27.

Rittel, H. W., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4, 155–169.

Scribner, S. (1997). Mind and social practice. Selected writings of Sylvia Scribner. Cambridge, UK: Cambridge University Press.

Spillane, J. P., Halverson, R., & Diamond, J. B. (2001). Investigating school leadership practice: A distributed per- spective. Educational Researcher, 30, 23–28.

Strauss, A., & Corbin, J. M. (1997). Grounded theory in practice. Thousand Oaks, CA: SAGE.

van den Berg, R. (2002). Teachers’ meanings regarding educa- tional practice. Review of Educational Research, 72, 577–625.

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge, UK: Harvard University Press.