Research Based Practices

Learning Disabilities Research & Practice, 30(2), 91–101 C© 2015 The Division for Learning Disabilities of the Council for Exceptional Children

Using the 6S Pyramid to Identify Research-Based Instructional Practices for Students with Learning Disabilities

Tanya Santangelo Arcadia University

Leslie C. Novosel and Bryan G. Cook University of Hawai´i at Mānoa

Meredith Gapsis Arcadia University

To optimize students’ learning outcomes, educators are increasingly expected to use instruc- tional practices shown to be effective by credible research. To help make this possible, or- ganizations and scholars are producing resources that summarize research related to various instructional practices. However, as the collection of resources grows in size and complexity, it can be difficult and frustrating for practitioners to locate and utilize this information. In this article, we describe the 6S Pyramid (DiCenso, Bayley, & Haynes, 2009), a framework for efficiently and accurately prioritizing different types of research evidence. We also identify and briefly summarize Internet resources corresponding with each level of the Pyramid that can be used to identify research-based practices for students with learning disabilities. To illustrate how the 6S Pyramid can be used in school and classroom settings, we begin and end the article with vignettes featuring a pair of co-teachers seeking to improve instruction for their students who struggle with reading.

MS. MARKLOFF AND MS. HUTCHINSON: ACT I

Ms. Markloff is a second-year, general education teacher. This year, she is teaching a diverse, inclusive class of 24 fourth graders, six of whom have learning disabilities (LD), and struggle with reading. An additional four students in the class have not been identified with LD, but consistently score below grade-level expectations on the state reading assess- ment. During the language arts period, Ms. Hutchinson, a special education teacher, provides push-in support for the students who struggle with reading. The progress monitor- ing data Ms. Markloff and Ms. Hutchinson gathered during the first 2 months of school suggest that their core literacy instruction is effective for most of the class. However, the teachers are very concerned because several students, in- cluding those with LD, are not making adequate gains.

As the teachers begin to brainstorm how to improve reading instruction for the students who are struggling, they recall a workshop they attended last year, where the presenter encouraged them to use the Internet as a tool to identify effective instructional practices. Ms. Markloff and Ms. Hutchinson cannot recall the specific Web sites the presenter recommended, so they reach out and ask several colleagues, as well as a couple of their former college professors, for suggestions. They set aside an hour and a

Requests for reprints should be sent to Tanya Santangelo, Arcadia Uni- versity. Electronic inquiries should be sent to [email protected].

half after school the following Friday to check out their leads. When Friday afternoon arrives, they compile the recommendations they received into a list that includes more than 10 Web sites, such as What Works Clearinghouse and National Center on Intensive Intervention (see Table 1).

As Ms. Markloff and Ms. Hutchinson navigate the Web sites, they become increasingly frustrated and overwhelmed with the magnitude, variability, and complexity of informa- tion across the different websites they visit. Many sites have far more information than they expected—much of which they find to be confusing and not directly related to their students. They also discover a lack of consistency in the types of re- sources available. For example, some sites offer narrative summaries of the research related to a particular topic area, others rate the effectiveness of particular practices and pro- grams using specific criteria, and one site evaluates partic- ular elements of individual studies. When the teachers finish exploring the last Web site on their list, they realize nearly two hours has passed; yet they are still unsure of how to improve their instruction to help their students who struggle with reading. Disappointed and frustrated after their expe- rience, Ms. Markloff and Ms. Hutchinson find themselves questioning whether Internet resources really DO offer use- ful information for teachers.

We suspect that many teachers find themselves in situ- ations similar to Ms. Markloff and Ms. Hutchinson. That is, they want to find the most effective practices for improving the outcomes of their struggling learners, and

92 SANTANGELO ET AL.: 6S PYRAMID

TABLE 1 Summary of Internet Resources

Levels of Information Offereda

Name of Internet Resource Web Address 2 3 4 5

Best Evidence Encyclopedia www.bestevidence.org X Center on Instruction www.centeroninstruction.org X X Current Practice Alerts www.teachingld.org/alerts X National Center on Intensive Intervention www.intensiveintervention.org X National Dissemination Center for Children with Disabilities www.parentcenterhub.org X National Secondary Transition Technical Assistance Center www.nsttac.org X What Works Clearinghouse www.ies.ed.gov/ncee/wwc X X X

aRefers to the levels of evidence in the 6S Pyramid (DiCenso et al., 2009), as shown in Figure 1.

they like the idea of using practices shown to be effective by credible research evidence. However, despite the grow- ing number of resources available on the Internet related to instructional practices supported by scientific research (i.e., research-based practices (RBPs); further defined in the next section), information from the Internet can be mislead- ing, overwhelming, and difficult to translate into practice. In this article, we describe a model for guiding special ed- ucators through the maze of Internet resources related to RBPs for students with LD. Specifically, after providing a brief overview of RBPs and their importance for special ed- ucators, we discuss the 6S Pyramid (DiCenso, Bayley, & Haynes, 2009) and how it can help practitioners efficiently and accurately prioritize RBPs for students with LD that can be found on a number of existing Internet resources. We con- clude with a description of Ms. Markloff and Ms. Hutchinson using the 6S Pyramid and Internet resources described in this article to identify RBPs for their students who struggle with reading.

RBPs

Although no instructional program or technique works for every student, some practices are generally more effective than others for improving student outcomes (Cook, Smith, & Tankersley, 2012; Hattie, 2009). Because scientific research has an unparalleled track record of determining which prac- tices work best (Odom et al., 2005; Slavin, 2002, 2008), re- cent educational reforms and laws emphasize identifying and implementing practices shown by sound, scientific research to be effective (Yell & Rozalski, 2013). In this article, we use the term RBPs to refer generally to instructional strate- gies, techniques, and programs shown by credible scientific research to be effective. We distinguish RBPs from evidence- based practices which, according to Cook and Cook (2013), are a subset of RBPs with research bases that meet rigorous standards related to research design, quality, and quality (e.g., Gersten et al., 2005; Horner et al., 2005). Thus, in this arti- cle we use RBPs synonymously with “empirically validated practices” to refer to practices supported as effective by sci- entific research, but with research bases that do not necessar- ily meet the rigorous standards required for evidence-based practices.

Although RBPs can help improve outcomes for all learn- ers, they are especially important for students with LD, who require highly effective instruction to reach their goals and potential (e.g., Swanson, Harris, & Graham, 2013). The re- search base on interventions for students with LD, however, is large, varied, and technical (Cook et al., 2012). Miech, Nave, and Mosteller (2005) referred to this as the 20,000 article problem because approximately 20,000 education-related ar- ticles are published annually and, clearly, educators do not have time to read and make sense of that much information. Moreover, even if time were not a barrier, many educators do not have the advanced training necessary to critically analyze the research base. In response to this problem, ed- ucational organizations and scholars have begun to appraise and synthesize research findings to help practitioners and other stakeholders readily identify RBPs (Odom et al., 2005; Slavin, 2002, 2008). However, like the research bases they intend to simplify, the growing array of resources has be- come sizeable, diverse, and complicated. As Ms. Markloff and Ms. Hutchinson discovered, rather than streamlining the process of determining what works, these resources may un- intentionally confound the problem by adding another layer of information through which practitioners must sift.

IDENTIFYING INTERNET RESOURCES FOR RBPs

We used the following process to locate and analyze the re- sources described in this article. First, we conducted a search of Internet resources that we and our colleagues identified as potentially including information about RBPs for students with LD to determine which ones offer free and publicly available information corresponding with different levels of the 6S Pyramid. Specifically, we focused on practices targeting academic (e.g., reading, math, writing, content area knowledge) and other college- and career-readiness outcomes (e.g., functional life and transition skills) for K-12 students. Thus, assessment (e.g., curriculum-based measurement) and behavioral interventions (e.g., school- wide positive behavioral support) were not included. We also restricted our focus to resources that synthesize and appraise evidence from studies using research designs generally recognized as establishing cause and effect

LEARNING DISABILITIES RESEARCH 93

relationships: true random experiments (e.g., randomized control trials), quasi-experiments, and single-case studies (Cook, Tankersley, Cook, & Landrum, 2008; Odom et al., 2005). We originally planned to only include information derived from research conducted exclusively with students with LD. However, given the scarcity of such resources, we decided to also include information derived from research with samples that include students with LD along with other groups, such as students determined to be at risk of academic failure. In subsequent sections of this article, we note when the participants involved in research supporting a practice are not just students with LD. Through our initial search, we identified seven relevant Internet resources that provide information on RBPs for students with LD that correspond with one or more levels of the 6S Pyramid (see Table 1).

Next, two authors independently examined all of the RBP- related resources available at the seven web sites and docu- mented critical characteristics of each relevant publication found (e.g., title, level/type of evidence, percentage of stu- dents with LD in the sample, targeted outcome area, pri- mary findings). The data collected by the two authors were then compared and the few differences found were resolved through discussion (inter-rater reliability, calculated as to- tal% agreement, >99 percent). Our review and analysis of the Web sites took place during the fall and spring of 2013 and was updated in September of 2014.

THE 6S PYRAMID

The 6S Pyramid (DiCenso et al., 2009; see Figure 1) is a framework for categorizing and prioritizing resources that appraise and synthesize research findings. The 6S Pyramid was developed in the field of medicine to help doctors and other professionals make decisions based on the best avail- able research evidence (e.g., Robeson, Dobbins, DeCorby, & Tirilis, 2010). The Pyramid represents a hierarchy of ev- idence in which higher levels provide increasingly accurate and efficient types of information. Therefore, when search- ing for RBPs using the 6S Pyramid, practitioners should start at the highest possible level and only work down to lower levels as necessary. For some well-researched practices, the highest level of available evidence might be a summary of the research (Level 2). In contrast, the highest level of evidence for a practice that is not yet well-researched might be a single study (Level 6). In the following sections, we describe each level in the 6S Pyramid and summarize Inter- net resources currently available to educators. Because each level of the Pyramid builds on information from lower levels, we begin our description at the bottom and move up.

Level 6: Studies

The lowest level in the 6S Pyramid is studies. Although indi- vidual studies are the building blocks of research bases, a sin- gle study, by itself, offers relatively little value to practitioners looking to identify RBPs (DiCenso et al., 2009). No study is perfect or represents “the truth.” Indeed, many studies con- tain bias and may report misleading findings (Cook, 2014).

As such, examining the effectiveness of a practice should in- volve consideration of findings from multiple, high-quality, experimental studies. However, searching for, critically ana- lyzing, and synthesizing the findings of multiple studies are incredibly time-consuming tasks and, thus, are not practical for busy educators to do on their own (Haynes, 2001). Given the availability of more efficient and accurate information at higher levels of the Pyramid, examining original studies is not the preferred method to determine the effectiveness of a practice and should only be used when higher-level resources are unavailable.

Level 6 Resources

Individual studies examining the impact of instructional prac- tices for students with LD were not offered at any of the Web resources we examined. However, they are readily available in professional journals, such as Learning Disabilities Research & Practice. Google Scholar (www.scholar.google.com) and ERIC (www.eric.ed.gov) are two search engines educators can use to search for individual studies, and they are also available through many public and university-based libraries.

Level 5: Synopses of Studies

Synopses of studies are concise, accessible descriptions of individual studies written by experts (DiCenso et al., 2009). Synopses of studies are beneficial to practitioners because they summarize the critical elements of the original research into a much shorter format, typically using nontechnical lan- guage. Most synopses of studies also include an evaluative commentary that highlights important strengths and limita- tions of the study. Therefore, reading a synopsis of a study is more accurate and efficient than reading the original study. Nonetheless, study synopses are limited because they only provide evidence from one study. Confidence in the effective- ness of a practice is greatly increased when multiple studies are considered and the results are consistent (Moonesinghe, Khoury, & Janssens, 2007). Because of the important limi- tations associated with considering evidence one study at a time, all higher levels of the 6S Pyramid consider findings across multiple studies.

Level 5 Resources

We identified two Internet resources that currently offer Level 5 information related to instructional practices for students with LD: What Works Clearinghouse (WWC) and National Center on Intensive Intervention. The study synopses pro- duced by the WWC are called “Single Study Reviews” and can be accessed by clicking the “Single Study Reviews” link located on WWC’s home page. Each Single Study Review includes a concise summary of the targeted study’s features and findings, as well as a rating of research quality (i.e., meets WWC evidence standards without reservations, meets WWC evidence standards with reservations, or does not meet WWC evidence standards). Although the WWC had produced more

94 SANTANGELO ET AL.: 6S PYRAMID

FIGURE 1 The 6S Pyramid (DiCenso, Bayley, & Haynes, 2009).

than 80 Single Study Reviews at the time of our review, only one was a synopses of a study that included students with LD (i.e., Same-Language-Subtitling [SLS]: Using Subtitled Music Video for Reading Growth). Because WWC releases several new Single Study Reviews each month, we anticipate additional LD-related Level 5 resources will be available from WWC in the future.

The study synopses produced by National Center on Intensive Intervention are organized into a “Tools Chart” that can be accessed using the “Tools Charts” dropdown menu at the top of the home page or directly at the URL: www.intensiveintervention.org/chart/instructional- intervention-tools. Each synopsis includes: (1) a summary of the intervention (e.g., targeted students, cost); (2) an evaluation and description of key elements related to research quality (e.g., design, fidelity); (3) details related to implementation (e.g., group size, duration, training); (4) a description of the findings (e.g., effect size for all participants as well as targeted subgroups, such as students with LD); and (5) related research (i.e., number of additional studies evaluating the practice and whether they have been reviewed by WWC). At the time of our review, 59 study synopses were available from National Center on Intensive Intervention. Consistent with the organization’s mission, all of the synopsized studies focused on students with academic difficulties and 19 specifically included students with LD in the sample (16 studies targeted literacy and 3 focused on math). Practitioners should prioritize practices that are supported as effective by multiple, high-quality studies for a target population (e.g., elementary students with LD). For example, in the “Tools Chart,” we found three studies evaluating Stepping Stones to Literacy, all of which involved kindergarten participants, received positive

ratings for study quality, and showed positive effects on reading and prereading outcomes. Thus, educators can have considerable confidence in the effectiveness of this practice for this population.

Level 4: Syntheses

Syntheses draw conclusions about the effectiveness of a practice by systematically considering findings across mul- tiple experimental studies. A research synthesis can, then, be thought of as a “study of studies.” Because it is unclear whether the results from any individual study are accurate or generalizable until they are (or are not) confirmed by other studies, replication is one of the foundational tenets of scien- tific inquiry (Jasny, Chin, Chong, & Vignieri, 2011). Since syntheses consider the findings of multiple studies, they rep- resent a higher level of evidence than individual studies. Reading a synthesis is also more efficient than reading all of the individual studies included in the review. Because of their advantages, syntheses are being conducted with increasing frequency to determine what works in education, including for students with LD (e.g., Swanson et al., 2013).

A common type of synthesis used to identify RBPs is a meta-analysis. In a meta-analysis, researchers establish crite- ria for what studies will be included in their review, and then calculate the average impact (i.e., effect size) of the practice across the studies reviewed (Cooper, Hedges, & Valentine, 2009; see Banda & Therrien, 2008, for a practitioner-friendly introduction to meta-analysis). An evidence-based review is another type of synthesis used to identify RBPs (Cook et al., 2012). Similar to meta-analyses, evidence-based reviews an- alyze multiple studies, but the latter (1) consider only studies

LEARNING DISABILITIES RESEARCH 95

that meet quality standards and (2) use predetermined stan- dards to categorize the effectiveness of the practice based on the findings from high-quality studies.

Despite their strengths, research syntheses are also sub- ject to limitations that can impact the accuracy and efficiency of their results. For example, published syntheses can be lengthy and, without an understanding of advanced statisti- cal methods, the details and nuances of the findings can be challenging to interpret. Also, just like with individual stud- ies, low quality methods can negatively impact the accuracy and utility of the findings (Cooper et al., 2009). Moreover, most practitioners have neither the time nor expertise needed to critically examine and evaluate a synthesis. Additionally, because syntheses conducted by different organizations and researchers do not necessarily use the same methods (e.g., different studies are included), they can produce conflicting results about the effectiveness of a practice (Briggs, 2008; Cook & Cook, 2013; Slavin, 2008). Finally, syntheses have not been conducted on all practices, and they can quickly fall out of date as new research is completed.

Level 4 Resources

We identified five Web sites that currently offer syntheses. Meta-analyses are available at the Center on Instruction and evidence-based reviews can be found at WWC, Best Evi- dence Encyclopedia, National Secondary Transition Tech- nical Assistance Center, and Current Practice Alerts. The Center on Instruction has produced two meta-analyses that include research with samples containing some or all students with LD: Interventions for Adolescent Struggling Readers: A Meta-Analysis with Implications for Practice; and Math- ematics Instruction for Students with Learning Disabilities or Difficulty Learning Mathematics: A Synthesis of the In- tervention Research. They focus on grades 4–12 and 1–12, respectively. These syntheses can be found by clicking the “Special Education” link on the Center on Instruction’s home page and then selecting “Research: Meta-analyses and Sum- maries.” Although Center on Instruction is not producing new publications, all their resources are archived and remain available.

The evidence-based reviews offered by WWC, called “In- tervention Reports,” are produced by a team of experts and include: (1) an overview of the targeted practice, (2) a de- tailed description of the methods and results for each in- cluded study, and (3) a list of studies excluded from the review along with the reasons why. WWC syntheses cate- gorize practices—based on the number, quality, and effects of reviewed studies—as having either positive, potentially positive, mixed, no discernible, potentially negative, or neg- ative effects (Effectiveness Rating). An Improvement Index (which summarizes the impact of the intervention in per- centile points) and an Extent of Evidence Rating (either small or medium to large) are also provided. WWC Intervention Reports can be accessed by clicking the “Intervention Re- ports” link on the home page. At the time of our review, WWC had published 17 syntheses targeting students with LD (i.e., included research with samples comprised of at least 50 percent of students with LD). As shown in Table 2,

TABLE 2 What Works Clearinghouse Ratings for Practices with Reviewed Research Studies Involving Students with Learning Disabilities

Rating Practice and Grade Level by Outcome Area

General Reading Achievement

No Discernible Effects Project Read Phonology (K-4) Repeated Reading (5–12)

Alphabetics Potentially Positive Effects Lindamood Phoneme Sequencing (4) No Discernible Effects Reading Mastery (2–4)

Repeated Reading (5–12) Reading Fluency Potentially Positive Effects Lindamood Phoneme Sequencing (4)

Peer-Assisted Learning Strategies (2–6) No Discernible Effects Read Naturally (4–6)

Repeated Reading (5–12) Reading Comprehension Potentially Positive Effects Peer-Assisted Learning Strategies (2–6)

Repeated Reading (5–12) No Discernible Effects Lindamood Phoneme Sequencing (4)

Reading Mastery (2–4) Mathematics Potentially Positive Effects Lindamood Phoneme Sequencing (4) No Discernible Effects Peer-Assisted Learning Strategies (2–6) Writing Potentially Positive Effects Read Naturally (4–6)

Spelling Mastery (2–4) Potentially Negative Effects Lindamood Phoneme Sequencing (4) No Studies Meet Evidence Standards

Alphabetic Phonics, Barton Reading & Spelling System, Dyslexia Train- ing Program, Fundations, Herman Method, Read 180, Reciprocal Teaching, Unbranded Orton–Gillingham-based Interventions, Voyager Reading Pro- grams, Wilson Reading System.

seven of these syntheses include at least one study that met WWC evidence standards and, thus, these practices received an Effectiveness Rating.

Whereas WWC syntheses focus on specific practices, the evidence-based reviews produced by Best Evidence Encyclo- pedia analyze research related to broader topic areas, such as elementary mathematics instruction. The rating scale used by the Best Evidence Encyclopedia to describe the effectiveness of a practice has six categories: strong evidence of effective- ness, moderate evidence of effectiveness, limited evidence of effectiveness—strong evidence of modest effects, limited evidence of effectiveness—weak evidence with notable ef- fects, insufficient evidence of effectiveness, and no quali- fying studies. Each “Best Evidence Synthesis” completed by the Best Evidence Encyclopedia results in two publica- tions: a “Full Report,” which comprehensively describes the synthesis methods and findings, and an “Educator’s Sum- mary,” which is a short and easy-to-read summary designed to help practitioners understand and utilize the synthesis re- sults. Syntheses can be accessed via the topic-specific links on the left-hand side of the home page. At the time of our review, the Best Evidence Encyclopedia offered one synthe- sis that included research with (K-5) students with LD, titled Effective Programs for Struggling Readers: A Best Evidence Synthesis.

96 SANTANGELO ET AL.: 6S PYRAMID

TABLE 3 National Secondary Transition Technical Assistance Center Ratings for Practices with Reviewed Research Studies Involving Students with

Learning Disabilities

Level of Evidence Practice and Grade/Age Level by Outcome Area

Academic Skills

Strong Mnemonics (13–17) Peer Assistance (13–17) Self-Management (13–16) Technology (12–22) Visual Displays (13–16)

Functional Life and Transition Skills Moderate Self-Advocacy strategy (to teach student involvement in the IEP meeting, 12–18)

Self-Determined Learning Model (to teach goal attainment, 14–19) Self-Directed IEP (to teach student involvement in the IEP meeting, 12–21) Simulations (to teach social skills, 12–21) Training Modules (to promote parent involvement in the transition process, 15+) Whose Future is it Anyway? (to teach knowledge of transition planning, 12–16) Whose Future is it Anyway? (to increase self-determination skills, 12–16)

Potential Computer-Assisted Instruction (to teach participation in the IEP process, 12–18) Mnemonics (to teach completing a job application, 15–16)

The evidence-based reviews published by National Sec- ondary Transition Technical Assistance Center, called “Practice Descriptions,” are written specifically for practitioners. Each includes a brief summary of the prac- tice, a short description of the research included in the re- view, a list of additional references, and recommendations for learning more about implementation (including links to sample lesson plans). The level of evidence supporting a practice is rated as strong, moderate, potential, or low. Al- though National Secondary Transition Technical Assistance Center’s synthesis process is generally consistent with the way other organizations (e.g., WWC and Best Evidence En- cyclopedia) conduct evidence-based reviews, it is unique in that, sometimes, an existing synthesis (e.g., a meta-analysis published in a peer-reviewed journal) is used to develop a Practice Description (rather than independently locating and analyzing the individual studies). At the time of our review, 14 Practice Descriptions included research with stu- dents with LD (see Table 3). They can be accessed from the “Evidence-based Practices” tab at the top of their home page or directly from http://nsttac.org/content/evidence-based- practices-secondary-transition.

The Council for Exceptional Children’s Division for Learning Disabilities and Division for Research collabora- tively produce a series of “Current Practice Alerts”. Like Na- tional Secondary Transition Technical Assistance Center’s Practice Descriptions, Current Practice Alerts are written specifically for educators and include a concise description of the targeted practice and research. Current Practice Alerts rate the effectiveness of practices reviewed as either Go for It (evidence suggests a high likelihood of effectiveness) or Use Caution (either insufficient evidence to determine effective- ness or evidence indicating small, no, or negative effects). Current Practice Alerts differ from the systematic synthe- ses produced by WWC, Best Evidence Encyclopedia, and National Secondary Transition Technical Assistance Center because a uniform set of predetermined standards is not used

to categorize the effectiveness of each practice reviewed. Rather, authors, who are experts in the field, make a decision based on their own interpretation of the research (which is then reviewed and verified by the Alerts Editorial Commit- tee). As shown in Table 4, 16 Current Practice Alerts that include research with students with LD were available at the time of our review.

In addition to the Web resources described above, re- search syntheses examining the effectiveness of instructional practices for students with LD are frequently published in professional journals. As previously described with Level 6 resources, Google Scholar, ERIC, and other search engines can be used to locate relevant syntheses published in journals. For example, a teacher who wants to improve the written language skills of his/her students with LD would find several syntheses by conducting a search using the key words “learning disabilities,” “meta-analysis,” and “writing instruction.”

Level 3: Synopses of Syntheses

Synopses of syntheses are brief and accessible reviews of existing syntheses. They include a summary of the synthesis’ methods, findings, and implications and, in most cases, an expert evaluation of the research quality (DiCenso et al., 2009). Synopses of syntheses preserve the strengths of syntheses (e.g., determining effectiveness across multiple studies), while addressing some of their limitations (e.g., significant time and expertise required to critically analyze methods and interpret results). Nonethe- less, it is important to recognize, like Level 5 and Level 4 evidence, Level 3 synopses are only as good as the original research on which they are based (Windish, 2013). Additionally, they are relatively rare in education and, as with syntheses, they become outdated as more research is conducted.

LEARNING DISABILITIES RESEARCH 97

TABLE 4 Current Practice Alerts Ratings for Reviewed Practices

Rating Practice and Grade Levela by Outcome Area

Multiple Academic Domains and Content Areas

Go For It • Direct Instruction (K-12) • Class Wide Peer Tutoring (K-12) • Cognitive Strategy Instruction (K-12) • Graphic Organizers (K-12) • Mnemonics (K-12) • Self-Determined Learning Model of Instruction (not specified) • Vocabulary Instruction (K-12)

Use Caution • Cooperative Learning (K-12) • Co-teaching (K-12) • Learning Styles (not specified)

Reading Go For It • Fluency Instruction (not specified)

• Phonics Instruction (beginning readers and older students with reading difficulties) • Phonological Awareness Acquisition and Intervention (beginning readers) • Reading Comprehension Strategy Instruction (not specified)

Use Caution • Reading Recovery (not specified) Writing Go For It • Self-Regulated Strategy Development for Writing (upper-elementary through middle school)

aMost Current Practice Alerts indicate a relevant age range, rather than the specific grades represented in the research used to determine the rating.

Level 3 Resources

Center on Instruction and National Dissemination Center for Children with Disabilities are the two Internet resources we identified that currently offer synopses of syntheses related to students with LD. Center on Instruction’s synopses are written for practitioners and include an easy-to-understand summary of the synthesis methods and findings, as well as a section devoted to implications for practice. They do not include an explicit evaluation of the synthesis methods; how- ever, the syntheses were all chosen to be synopsized because of their quality. Center on Instruction syntheses can be lo- cated by clicking the “Special Education” link on the left- hand side of the home page, and then selecting “Research: Meta-analyses and Summaries” or by typing “Synopsis” into the “Search Terms” box at the top of the page. Center on In- struction offers one synopsis of a synthesis examining ways to improve expository text comprehension for students with LD, and two others related to writing instruction for all stu- dents, including those with LD.

National Dissemination Center for Children with Disabil- ities offers a number of synopses of syntheses, which they call “Research Summaries.” Each synopsis uses the same, practitioner-friendly format and offers a summary of the syn- thesis methods, findings, and implications. As with Center on Instruction, National Dissemination Center for Children with Disabilities synopses do not include an evaluative com- ponent, but syntheses are selected because of their generally high quality. Eighteen of National Dissemination Center for Children with Disabilities’ synopses are of syntheses that included research with samples comprised of students with LD. They represent a variety of outcome areas (e.g., reading, math, writing) and span grades K-12. Because of

discontinued funding, National Dissemination Center for Children with Disabilities’ original Web site is no longer active; however, all the publications are archived and remain available through The Center for Parent Information and Resources (www.parentcenterhub.org/resources).

Level 2: Summaries

Summaries are publications typically produced by a group of experts, who seek to integrate all available research ev- idence (e.g., original research and syntheses) related to a particular topic area; therefore, they provide highly reliable recommendations (DiCenso et al., 2009; Haynes, 2006). To the degree that summaries reflect an entire research base, they represent the most accurate form of evidence that can be used to identify RBPs. Additionally, reading a single sum- mary is much more efficient than reading the large body of studies and syntheses on which the recommendations in the summary are based. In education, Level 2 summaries are currently the highest level of evidence available and, thus, this is the level where educators should begin their search for information, examining lower levels only when relevant Level 2 summaries cannot be found. Although summaries do not exist for all academic areas, they are available for many key skill areas, such as reading, writing, and mathematics. Despite their many strengths, it is important to recognize that, like syntheses, summaries are only as valid as the stud- ies and the process used to create them (Windish, 2013). Finally, summaries are not available for all populations (e.g., students with LD) in all topic areas. And, as with syntheses, summaries need to be reviewed and revised periodically so they do not become outdated.

98 SANTANGELO ET AL.: 6S PYRAMID

Level 2 Resources

“Practice Guides” are the only example of summaries we identified related to LD. Each Practice Guide is written by a panel of experts, which includes both researchers and practi- tioners, and is focused on a specific topic (e.g., reading com- prehension) and (in most cases) specific student population (e.g., elementary-aged students; English language learners). Based on the panel’s extensive and rigorous review of the available, high quality research, a set of recommendations for school/classroom practice is offered. Each recommen- dation is rated as being supported by a strong, moderate, or minimal level of evidence. Although Practice Guides are comprehensive, they are written so that practitioners can un- derstand the panel’s methods and findings and can use them to guide school/classroom practice. Practices Guides can be accessed by clicking the “Practice Guides” link on WWC’s home page. At the time of our review, none of the 17 avail- able Practice Guides focused specifically on students with LD. However, nine included research involving at least some students with LD: Assisting Struggling Students with Mathe- matics; Dropout Prevention; Developing Effective Fractions Instruction for Kindergarten Through 8th Grade; Effective Literacy and English Instruction for English Learners in the Elementary Grades; Improving Adolescent Literacy: Effec- tive Classroom and Intervention Practices; Improving Math- ematical Problem Solving in Grades 4 Through 8; Organizing Instruction and Study to Improve Student Learning; Reduc- ing Behavior Problems in the Elementary School Classroom; and Teaching Elementary School Students to be Effective Writers.

Level 1: Systems

Representing the highest tier of the 6S Pyramid, systems are computerized decision support tools that summarize all rel- evant research evidence about a particular practice or topic (Cullum, Ciliska, Haynes, & Marks, 2008; DiCenso et al., 2009). Systems provide recommendations—based primar- ily on Level 2 summaries—in response to practitioner input (Haynes, 2001, 2006). For example, in medicine, clinicians can enter patient characteristics and symptoms, and the elec- tronic system will provide research-based recommendations aligned to the patient’s specific needs and characteristics. In medicine, enthusiasm for systems is growing (e.g., Hlyva et al., 2010; Osheroff et al., 2007). Although systems do not yet exist in education, we envision them being developed and utilized in the future. Once this happens, an educator could enter information about a learner’s characteristics (e.g., grade, age, disability information) and target area (e.g., be- ginning reading skills, math problem solving) via a pulldown menu on a Web site, and the system would generate a list of recommended practices based on the best available research evidence. Importantly, the goal of systems is not to replace professional judgment, but rather to provide individualized, authoritative, and timely research-based recommendations to inform and support practitioners’ decision making (Hlyva et al., 2010).

Considerations and Limitations

When using the 6S Pyramid as a guide for identifying effective instructional practices, it is important that special educators bear in mind that different resources use different procedures and criteria, the levels of the 6S Pyramid may overlap, RBP-related terminology is used inconsistently, and implementation of RBPs needs to be supported. Con- sideration of these issues will inform and optimize special educators’ use of the 6S Pyramid.

Educators should not assume that RBPs at a given level of the 6S Pyramid are equivalent across Web sites/organizations. For example, at Level 4, some syntheses use stringent criteria applied to a comprehensive search of the literature as the basis for their conclusions, whereas oth- ers use no systematic criteria or criteria that are less rigorous. Similarly, synopses vary in terms of their clarity for practi- tioners, the depth of information provided, and the presence and quality of an evaluation. In fact, resources at the same level may occasionally provide conflicting recommendations (e.g., one synthesis indicates a practice is effective, but an- other reports it is not). In such cases, the higher quality re- sources should carry more weight. If practitioners encounter conflicting results from resources at the same level of the 6S Pyramid, we recommend they consult with individuals with advanced research training and expertise to determine which is more credible.

Organizations providing resources related to RBPs have not, to our knowledge, designed their products to align di- rectly with one or more levels of the 6S Pyramid. Rather, the 6S Pyramid is a heuristic framework that we have applied to existing resources in order to better understand and apply those resources. Although we believe that the different levels of the 6S Pyramid make useful distinctions between differ- ent types of resources, it is important to recognize that it may be difficult to neatly classify some resources into one of the six levels. For example, an exhaustive meta-analysis that is conducted by a group of scholars and includes an expanded discussion of practical recommendations might reasonably be considered either a Level 2 (summary) or Level 4 (syn- thesis). Despite this potential difficulty, it is important to remember that the guiding principle of the 6S Pyramid is for practitioners to prioritize resources that incorporate entire re- search bases, regardless of how we or others may categorize them.

Terminology related to RBPs is used inconsistently in the field of special education (Cook & Cook, 2013). For example, National Secondary Transition Technical Assistance Center refers to a practice as evidence-based (with potential evidence) when supported as effective by a single study. In contrast, other synthesis authors (e.g., Baker, Chard, Ketterlin-Geller, Apichatabutra, & Doabler, 2009; Chard, Ketterlin-Geller, Baker, Doabler, & Apichatabutra, 2009) only categorized practices as evidence-based for students with LD when supported by multiple, high-quality studies. Additionally, sources use different terms and criteria to classify the effectiveness of practices. For instance, the WWC refers to practices supported by the highest level of evidence as having “positive effects,” whereas National Secondary Transition Technical Assistance Center uses

LEARNING DISABILITIES RESEARCH 99

“evidence-based practices with strong evidence” and the Current Practice Alerts use “Go For It.” Because these labels correspond with different criteria, it is important to understand what a particular term designates.

Although we recommend using the 6S Pyramid to identify effective practices, it is important to recognize identifying RBPs is but the first (and perhaps easiest) step in implementing effective instruction and improving learner outcomes. As Fixsen, Blase, Horner, and Sugai (2009) noted, identifying an RBP “is one thing, implementation of that practice is another thing altogether” (p. 5). Implementing RBPs successfully over time can be very difficult in schools and needs to be supported by ongoing training and a supportive school culture (see Cook & Odom, 2013; Fixsen, Blase, Metz, & Van Dyke, 2013). Moreover, some recom- mended practices/programs may entail costs and training not available to some educators. Additionally, special educators will need to balance implementing effective practices as designed (i.e., with fidelity) with adapting them in ways to optimize their positive impact on students with LD. We suggest interested educators read Johnson and McMaster’s (2013) excellent chapter for guidelines on adapting RBPs by implementing critical elements of RBPs as designed, while tailoring other aspects of interventions to meet the unique characteristics and needs of groups and individuals. Finally, it is important to recognize that even practices with significant research support will not work for every learner; there will be “nonresponders” to every practice. As such, even when using practices at the highest levels of the 6S Pyramid, special educators should systematically monitor student progress to evaluate the effects of the practice on individual learners.

Notwithstanding these limitations and considerations, the 6S Pyramid can be a useful tool for educators to efficiently identify and prioritize RBPs for students with LD using ex- isting Internet resources. We conclude with the second act of our vignette, in which Ms. Markloff and Ms. Hutchin- son demonstrate how the 6S Pyramid can be applied in practice.

Ms. Markloff and Ms. Hutchinson: Act II

After reading about the 6S Pyramid, Ms. Markloff and Ms. Hutchinson are excited to conduct a second search for RBPs; this time, they will use their understanding of hierarchical levels of evidence to search strategically and efficiently. The teachers plan to start searching at the top of the Pyramid and work down only as necessary. Because there are no resources at Level 1 in education, they begin their search browsing through the only Level 2 resources available for students with LD, the Practice Guides, and decide that Improving Adolescent Literacy: Improving Classroom and Intervention Practices is the most relevant because it focuses on students in 4th–12th grade and includes some research with students with LD.

In the guide, they find five recommendations. The first four relate to core reading instruction for all students: provide ex- plicit vocabulary instruction, provide direct and explicit com- prehension instruction, provide opportunities for extended discussion of text meaning and interpretation, and increase

student motivation and engagement in literacy learning. Af- ter discussing these recommendations, Ms. Markloff and Ms. Hutchinson conclude that they already incorporate these practices as part of their instructional routine, though the descriptions and examples in the Practice Guide do provide them with several new and important ideas for optimizing their instruction. The teachers recognize they do not imple- ment the fifth recommendation (make available intensive and individualized interventions for struggling readers that can be provided by trained specialists), so they read this section of the guide very carefully and take notes on the recommen- dations they would like to implement. These include: using assessments to precisely identify each student’s strengths and weaknesses, aligning instruction and intervention with stu- dents’ individual needs, establishing learning goals, utiliz- ing learning aids, such as graphic organizers, and explicitly teaching reading strategies.

Ms. Markloff and Ms. Hutchinson decide to search for ad- ditional resources to help them deepen and extend what they learned by reading the Practice Guide. Having exhausted the relevant Level 2 resources, the teachers search next for Level 3 resources. To their surprise—between National Dissemination Center for Children with Disabilities and Center on Instruction—they locate more than 10 synopses of syntheses related to reading instruction for students with LD. While reading these publications, they expand and refine their notes. For example, reading National Dissem- ination Center for Children with Disabilities’ synopsis of How Reading Outcomes of Students with Disabilities are Related to Instructional Grouping Formats: A Meta-Analytic Review convinces them to use small group instruction more frequently, because a solid body of research indicates it improves reading outcomes for students with disabilities, including those with LD. They also decide to implement peer- mediated programs, such as peer tutoring and reciprocal teaching, because they are supported as effective in the syn- opsis. From National Dissemination Center for Children with Disabilities’ synopses of Fluency and Comprehension Gains as a Result of Repeated Reading and A Synthesis of Research on Effective Interventions for Building Reading Fluency with Elementary Students with Learning Disabilities, they learn about the positive outcomes associated with repeated reading and decide to also incorporate repeated reading into their instructional routine. Center on Instruction’s synopsis of Improving Comprehension of Expository Text in Students with Learning Disabilities: A Research Synthesis and several similar synopses produced by National Dissemination Cen- ter for Children with Disabilities reinforce what they learned from the Practice Guide about the effectiveness of con- tent/text enhancements and cognitive strategy instruction, and also provide additional suggestions for implementation.

At this point, Ms. Markloff and Ms. Hutchinson review all of their notes and decide that they have identified a sufficient number of RBPs to implement based on what they learned from the Level 2 and Level 3 resources. Rather than try to implement everything at once, they select two RBPs with which they feel comfortable and that align well with the needs of their students: graphic organizers and repeated reading. They each take one practice to research and set a goal of finding or making a checklist of the critical elements of each

100 SANTANGELO ET AL.: 6S PYRAMID

practice by the end of next week that they will use to guide their use of the practices.

REFERENCES

Baker, S. K., Chard, D. J., Ketterlin-Geller, L. R., Apichatabutra, C., & Doabler, C. (2009). The basis of evidence for Self-Regulated Strat- egy Development for students with or at risk for learning disabilities. Exceptional Children, 75, 303–318.

Banda, D. R., & Therrien, W. J. (2008). A teacher’s guide to meta-analysis. Teaching Exceptional Children, 41(2), 66–71.

Briggs, D. C. (2008). Synthesizing causal inferences. Educational Re- searcher, 37(1), 15–22. doi:10.3102/0013189×08314286

Chard, D. J., Ketterlin-Geller, L. R., Baker, S. K., Doabler, C., & Apichatabu- tra, C. (2009). Repeated reading interventions for students with learning disabilities: Status of the evidence. Exceptional Children, 75, 263–281.

Cook, B. G. (2014). A call for examining replication and bias in special education research. Remedial and Special Education, 35, 233–246. doi:10.1177/0741932514528995

Cook, B. G., & Cook, S. C. (2013). Unraveling evidence-based prac- tices in special education. Journal of Special Education, 47, 71–82. doi:10.1177/0022466911420877

Cook, B. G., & Odom, S. L. (2013). Evidence-based practices and im- plementation science in special education. Exceptional Children, 79, 135–144.

Cook, B. G., Smith, G. J., & Tankersley, M. (2012). Evidence-based practices in education. In K. R. Harris, S. Graham, & T. Urdan (Eds.), APA educational psychology handbook (Vol. 1, pp. 495–528). Washington, D.C.: American Psychological Association.

Cook, B. G., Tankersley, M., Cook, L., & Landrum, T. J. (2008). Evidence-based practices in special education: Some practical con- siderations. Intervention in School & Clinic, 44, 69–75. doi:10.1177/ 1053451208321452

Cooper, H., Hedges, L. V., & Valentine, J. C. (2009). The handbook of research syntheses and meta-analysis (2nd ed.). New York, NY: Russell Sage.

Cullum, N., Ciliska, D., Haynes, R. B., & Marks, S. (Eds.) (2008). Evidence- based nursing: An introduction. Oxford, UK: Blackwell.

DiCenso, A., Bayley, L., & Haynes, R. B. (2009). Accessing pre-appraised evidence: Fine-tuning the 5S model into a 6S model. Evidence Based Nursing, 12, 99–101. doi:10.1136/ebn.12.4.99-b

Fixsen, D. L., Blase, K. A., Horner, R., & Sugai, G. (2009). Concept paper: Developing the capacity for scaling up the effective use of evidence-based programs in state departments of education. Retrieved from http://www.uconnucedd.org/lend/readings/2011/pdfs/Session %2025%20-%20Mar%2025,%202011/Concept_Paper_SISEP_0409_ WEB.pdf

Fixsen, D., Blase, K., Metz, A., & Van Dyke, M. (2013). Statewide im- plementation of evidence-based programs. Exceptional Children, 79, 213–230.

Gersten, R., Fuchs, L., Compton, D., Coyne, M., Greenwood, C., & In- nocenti, M. S. (2005). Quality indicators for group experimental and quasi-experimental research in special education. Exceptional Chil- dren, 71, 149–164.

Hattie, J. A. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York, NY: Routledge.

Haynes, R. B. (2001). Of studies, syntheses, synopses, and systems: The ‘‘4S’’ evolution of services for finding current best evidence. Evidence Based Medicine, 6, 36–38. doi:10.1136/ebm.6.2.36

Haynes, R. B. (2006). Of studies, syntheses, synopses, summaries, and systems: The “5S” evolution of information services for evidence- based healthcare decisions. Evidence- Based Medicine, 11, 162–164.

Hlyva, O., Cotoi, C., Hobson, N., Parrish, R., Jedraszewski, D., Wilczyn- ski, N., et al. (2010). MacPLUS Federated Search: A new era tool for evidence-informed clinical decision making. McMaster University Medical Journal, 7, 57–60.

Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165–179.

Jasny, B. R., Chin, G., Chong, L., & Vignieri, S. (2011). Again, and again, and again . . . Science, 334(6060), 1225. doi:10.1126/ science.334.6060.1225

Johnson, L. D., & McMaster, K. L. (2013). Adapting research-based prac- tices with fidelity: Flexibility by design. In B. G. Cook, M. Tankersley, & T. J. Landrum (Eds.), Advances in learning and behavioral disabili- ties (Vol. 26, pp. 65–91). Bingley, UK: Emerald.

Miech, E. J., Nave, B., & Mosteller, F. (2005). The 20,000 article problem: How a structured abstract can help practitioners sort out educational research. Phi Delta Kappan, 86, 396–400.

Moonesinghe, R., Khoury, M. J., & Janssens, A. C. J. (2007). Most published research findings are false—But a little replication goes a long way. PLoS Medicine, 4(2), e28. doi:10.1371/journal.pmed.0040028

Odom, S. L., Brantlinger, E., Gersten, R., Horner, R. H., Thompson, B., & Harris, K. R. (2005). Research in special education: Scientific methods and evidence-based practices. Exceptional Children, 71, 137–148.

Osheroff, J. A., Teich, J. M., Middleton, B., Steen, E. B., Wright, A., & Detmer, D. E. (2007). A roadmap for national action on clinical decision support. Journal of the American Medical Informatics Association, 14, 141–145. doi:10.1197/jamia.M2334

Robeson, P., Dobbins, M., DeCorby, K., & Tirilis, D. (2010). Facilitating access to pre-processed research evidence in public health. BMC Public Health, 10, 95. doi:10.1186/1471-2458-10-95

Slavin, R. E. (2002). Evidence-based education policies: Transforming edu- cational practice and research. Educational Researcher, 31(7), 15–21. doi:10.3102/0013189×031007015

Slavin, R. E. (2008). Perspectives on evidence-based research in ed- ucation: What works? Issues in synthesizing educational pro- gram evaluations. Educational Researcher, 37(1), 5–14. doi:10.3102/ 0013189×08314117

Swanson, H. L., Harris, K. R., & Graham, S. (2013). Handbook of learning disabilities (2nd ed.). New York, NY: Guilford.

Windish, D. (2013). Searching for the right evidence: How to answer your clinical questions using the 6S hierarchy. Evidence-Based Medicine, 18(3), 93–97. doi:10.1136/eb-2012-100995

Yell, M. L., & Rozalski, M. (2013). The peer-reviewed requirement of the IDEA: An examination of law and policy. In B. Cook, M. Tankersley, & T. J. Landrum (Eds.), Evidence-based practices. Advances in learn- ing and behavioral disabilities (Vol. 26, pp. 149–172). Bingley, UK: Emerald.

About the Authors

Tanya Santangelo is an Associate Professor in the Department of Leadership for Educational Equity and Excellence in the School of Education at Arcadia University. She received her Ph.D. in Special Education from the University of Maryland in 2005. Her two primary areas of research/interest are: developing, validating, and disseminating ef- fective practices for teaching and assessing writing; and increasing educators’ knowledge and use of evidence-based practices.

Leslie C. Novosel is an Assistant Professor in the Department of Special Education at the University of Hawaiʻi at Mānoa. She received her Ph.D. in Special Education from the University of Kansas in 2011. Her research interests include evidence- based practices, school-wide models of response to intervention, and literacy interventions for adolescents with learning disabilities.

LEARNING DISABILITIES RESEARCH 101

Bryan G. Cook is a Professor in the Department of Special Education at the University of Hawaiʻi at Mānoa. He received his Ph.D. in Special Education from the University of California at Santa Barbara in 1997. His research interests include evidence- based practices, meta-research in special education, and physical activity among children and youth with high-incidence disabilities.

Meredith Gapsis is a special education teacher who works in an inclusive high school English classroom. She is currently completing her doctoral degree in special education at Arcadia University. Her dissertation focuses on teachers’ use of Web-based resources to identify and implement evidence-based practices.

Copyright of Learning Disabilities Research & Practice (Wiley-Blackwell) is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.