4 EDUCATION DISCUSSIONS DUE IN 72 HOURS

Chapter 12

Assessment in the Differentiated School: Leading Data-Driven Reforms

Learning Objectives

After reading this chapter, you will be able to:

• Explain how school leaders may assist teachers in implementing the new Common Core assessments.

• Articulate the importance for change leaders to institute schoolwide, data-driven decision- making practices.

• Convey how leaders may use common formative assessments and action research to promote data-driven school reform in differentiated classrooms.

12

© Jetta Productions/Photodisc/Thinkstock

Not everything that counts can be counted and not everything that can be counted counts.

—Albert Einstein

Introduction Chapter 12

Introduction This notable quote by Albert Einstein represents the dilemma that has always existed with educational assessments. In this final chapter of the text, we make the case that assessment is what pulls everything together in 21st century schools. From teachers making differentiated instructional decisions about their students to school leaders making decisions about whole- school reforms, data reigns supreme. Or, if it does not, it surely needs to do so. To help the change leader understand critical issues in assessment, this chapter presents the following topics: (a) the current state of the Common Core, high-stakes assessments; (b) the need for change leaders to institute data-driven decision making from the school level to the classroom level; and (c) the opportunity to examine data gathered from common formative assessments and action research to ignite change for leaders and teachers.

V O I C E S F R O M T H E F I E L D

Jose Luis Vilson is a middle school math teacher in the Inwood/Washington Heights neighborhood in New York City. As a teacher-leader, he also serves as president emeritus of the Latino Alumni Network of Syracuse University and is a popular writer and social activist.

A few years ago, I attended EduCon (a popular conference for educators) in Philadelphia. Upon arriving, I noticed that there were only three people of color at the entire conference, and my fiancé and I accounted for two of them! I immediately identified this as a problem within education. Either nobody at this conference was reaching out to teachers of color, or there simply aren’t very many out there to begin with.

I began to collaborate with the conference’s organizer, and we had several conversations about race. These conversations focused on both the teachers in our profession and the students that we serve. In the United States, our classrooms are filled with diverse students, with different cultures and ethnicities. However, our teaching force is primarily white. Often, teachers are at a loss for how to address cultural issues that are apparent in the classroom.

Much of education reform addresses how to help underperforming students; but what they’re really getting at is how to address diverse students. People like to say they are “color blind” or that “race doesn’t matter,” but we now know from research that ignoring race doesn’t serve anyone and, in fact, can make it harder for diverse students to learn and succeed. With a deeper cultural understanding of where our students come from, how they live, and what their experience is like, teachers can help them achieve both academically and socially.

I took it upon myself to establish a network of connected educators, not only to help students but also to connect diverse teachers with each other. What began as a support group is quickly moving toward an action group, which will collaborate with policymakers and major conference planners to increase the representation of teachers of color in policymaking discussions at small and major con- ferences and throughout the profession.

Only when we address the imbalance of diverse teachers throughout our education system, can we begin to address the larger challenge of closing the achievement gap for our students of differ- ent cultures and ethnicities. Teacher leadership requires not only responding to problems as they become apparent, but also seeking out problems that might go unnoticed if not acted upon.

Understanding Common Core Assessments Chapter 12

Pre-Test 1. Which of the following is not true of newest assessments in English language arts

and math?

a. They are designed to measure readiness for college and careers.

b. They attempt to negate concerns related to current state tests.

c. They are based on advances in technology and cognition.

d. They measure students in kindergarten through fifth grades.

2. Mrs. Menchie is using formative assessment with her classes. Which practice would best exemplify this process?

a. She administers an exam at the end of the content unit to assign grades.

b. She gives an individual oral assessment and then adjusts her instruction.

c. She has students do homework and returns the feedback a few weeks later.

d. She plans for her next unit by reviewing the state standards.

3. Sean has researched different ways to implement technology to enhance student learning and plans to mention it during a professional learning community meeting to discuss the research findings. Which step of action research does this situation exemplify?

a. Planning

b. Acting

c. Developing

d. Reflecting

Answers 1. d. They measure students in kindergarten through fifth grades. The answer can be found

in Section 12.1.

2. b. She gives an individual oral assessment and then adjusts her instruction. The answer can be found in Section 12.2.

3. a. Planning. The answer can be found in Section 12.3.

12.1 Understanding Common Core Assessments “The way the Common Core comes to life is through the assessments,” observed Gene Wilhoit, executive director of the Council of Chief State School Officers. Wilhoit went on to say, “If we get this right, we will put our students on a course for a better future” (quoted in Changnon, 2012, p. 2). Wilhoit’s remarks point out both the enormous opportunities and great challenges that lie ahead for educators as a new system of assessments is being created for the nation’s schools. In a similar vein, Sarles (2013), addressing the new Common Core assessments, remarked, “For good or ill, [the assessments] are what our students and our teachers will be facing beginning in the school year 2014–2015” (p. 10). As we embark upon this sea change in assessment for K–12 education, school leaders will need to have a thorough grounding in what is coming.

Understanding Common Core Assessments Chapter 12

Review of the Common Core State Standards

In Chapter 1 and again in Chapter 5, we discussed the Common Core State Standards (CCSS), which many have called our nation’s newest and most significant school reform. Before we begin the analysis of the Common Core assessments, let’s do a brief review of the CCSS. In 2012, Wiggins and McTighe published a white paper called “From Common Core Standards to Curriculum: Five Big Ideas.” The paper articulated their recommendations for developing curriculum and assessments that reflect the intent and emphases of the CCSS:

Big Idea 1: The Common Core Standards have new emphases and require a careful reading. The CCSS are legitimately new and do not represent mere minor adaptations to our current curriculum and instructional practices. The initiative didn’t call for, nor does it support, a national curriculum.

Big Idea 2: Standards are not curricula. A standard is an outcome, not a claim about how to achieve an outcome (i.e., a curriculum). While curriculum and instruction must address established standards, it is important to keep the long-term educational ends in mind—the development of vital capabilities in the learner due to engaging and effective work.

Big Idea 3: Standards need to be “unpacked.” Education leaders need to “unpack,” or translate, the standards into an engaging, outcome-focused curriculum to ensure clarity for teachers and school leaders about the results. The authors recommend four broad catego- ries: (a) long-term transfer goals, (b) overarching understandings, (c) overarching essential questions, and (d) a set of recurring cornerstone tasks.

Big Idea 4: A coherent curriculum is mapped backward from desired performances. The key to avoiding an overly discrete and fragmented curriculum is to design backward from complex performances that require content. Curriculum should be framed and developed in terms of worthy outputs—that is, desired performances by the learner—not simply as a listing of content inputs.

Big Idea 5: The standards come to life through the assessments. A prevalent misconception about stan- dards in general is that they simply specify learning goals to be achieved. A more complete and accu- rate conception recognizes that standards also refer to the desired qualities of student work and the degree of rigor that must be assessed and achieved. (Adapted from Wiggins & McTighe, 2012)

An Examination of the New Assessments Starting in 2014, 45 states and the District of Columbia will replace their current state-based assessments with digital assessment tools that will measure students’ progress against the Common Core State Standards (Fink, 2013). With these new assessments comes an emphasis on “critical higher order thinking skills” over “subject/content knowledge” (Martin, 2012).

Concerns About the New Assessments States, school districts, and school leaders have some deep concerns about the Common Core assessments. Although there has been some attempt at piloting the assessments, leaders are concerned about the following: (a) how the standards were developed; (b) that they are being rolled out entirely too early with little to no piloting; (c) implications of the standards on urban students, students with disabilities, and English learners; and (d) the impact of the standards on teachers and administrators. There is some justification for this concern. For

Think About It

As you examined the five big ideas of the CCSS, which do you believe will be the most challenging for school leaders to implement? For teachers?

Understanding Common Core Assessments Chapter 12

example, when Kentucky, as a pilot state, administered the first Common Core–aligned tests in 2011–2012, “The number of students scoring ‘proficient’ in language arts and math fell by more than a third” (Fink, 2013, p. 36). Even though experts expected the drop because the Common Core holds students to a higher standard, educators throughout the nation are wondering how best to help teachers prepare their students for these tests and how to com- municate to parents and the community that test scores, especially in the first year, may drop.

An Overview of Smarter Balanced Assessments

The Smarter Balanced Assessments is a system of valid, reliable, and fair next- generation assessments aligned to the Common Core State Standards (CCSS) in English language arts/lit- eracy (ELA/literacy) and mathematics for grades 3–8 and grade 11. The system, which includes both summative assessments for accountability purposes and optional interim assessments for instructional use, will use computer-adaptive testing technologies, to the greatest extent pos- sible, to provide meaningful feedback and actionable data that teachers and other educators can use to help students succeed.

Smarter Balanced Assessments will go beyond multiple-choice questions to include extended- response and technology-enhanced items, as well as performance tasks that allow students to demonstrate critical-thinking and problem-solving skills. Performance tasks challenge stu- dents to apply their knowledge and skills to respond to complex, real-world problems. Smarter Balanced Assessments capitalize on the precision and efficiency of computer-adaptive testing (CAT). This approach represents a significant improvement over traditional paper-and-pencil assessments used in many states today, providing more accurate scores for all students across the full range of the achievement continuum. The Smarter Balanced Assessment System has the following components:

1. A summative assessment administered during the last 12 weeks of the school year. The summative assessment will consist of two parts: a computer adaptive test and perfor- mance tasks that will be taken on a computer, but will not be computer adaptive. The summative assessment will:

• accurately describe both student achievement and growth of student learning as part of program evaluation and school, district, and state accountability systems;

• provide valid, reliable, and fair measures of students’ progress toward, and attainment of, the knowledge and skills required to be college- and career-ready; and

• capitalize on the strengths of computer adaptive testing—efficient and precise mea- surement across the full range of achievement and quick turnaround of results.

2. Optional interim assessments administered at locally determined intervals. These assess- ments will provide educators with actionable information about student progress throughout the year. Like the summative assessment, the interim assessments will be computer adaptive and include performance tasks. The interim assessments will:

• help teachers, students, and parents understand whether students are on track, and identify strengths and limitations in relation to the Common Core State Standards;

• be fully accessible for instruction and professional development (non-secure); and

• support the development of state end-of-course tests.

3. Formative assessment practices and strategies that are the basis for a digital library of professional development materials, resources, and tools aligned to the Common Core State Standards. Research-based instructional tools will be available on-demand to help

Understanding Common Core Assessments Chapter 12

teachers address learning challenges and differentiate instruction. The digital library will include professional development materials related to all components of the assessment system, such as scoring rubrics for performance tasks.

4. A secure, online reporting system that provides assessment results to students, parents, teachers, and administrators. The reports will show student achievement and progress toward mastery of the Common Core State Standards. (Retrieved November 5, 2013, from http://www.smarterbalanced.org/smarter-balanced-assessments/.)

An Overview of PARCC

State leaders in the Partnership for Assessment of Readiness of College and Careers (PARCC) share one fundamental goal: to build their collective capacity to dramatically increase the rates at which students graduate from high school prepared for success in college and the workplace. The PARCC assessment has six priority purposes:

• Determine whether students are college and career ready or on track

• Assess the full range of the CCSS, including standards that are difficult to measure

• Measure the full range of student performance, including the performance of high- and low-performing students

• Provide data during the academic year to inform instruction, interventions, and profes- sional development

• Provide data for accountability, including measures of growth

• Incorporate innovative approaches throughout the assessment system

To address the priority purposes, PARCC has developed an assessment system comprised of multiple components. Each component will be delivered by computer to maximize the use of technology innovations and will include the following components:

1. Two summative, required assessment components designed to:

• Make “college- and career-readiness” and “on-track” determinations

• Measure the full range of standards and full performance continuum, and

• Provide data for accountability uses, including measures of growth.

2. Two non-summative, optional assessment components designed to:

• Generate timely information for informing instruction, interventions, and professional development during the school year.

3. In English language arts/literacy, an additional non-summative component will assess students’ speaking and listening skills. This component is required, but the score is not included in the summative score. This component will be locally scored. (Retrieved November 5, 2013, from http://www.parcconline.org/parcc-assessment.)

Think About It

How might the new assessments be an improvement over your state’s current assessment system?

Leading Data-Driven School Reform Chapter 12

The Good News

Joanne Weiss, the chief of staff to U.S. Secretary of Education Arne Duncan, said, “When we get assessment right, it helps families, teachers, schools, and systems tailor learning to students’ needs and make wise decisions. . . . Today, we stand on the cusp of the biggest advances in assessment in a generation, with assessments that are more useful and less intrusive, thanks in part to advances in education technology” (quoted in Sparks, 2013, p. 1). Perhaps the greatest benefit of this coalescence of states around a common set of academic standards and two comprehensive assessment systems is the creation of the criti- cal mass needed to accelerate research and development across the entire K–12 education enterprise (Doorey, 2012). To accomplish this goal, states, districts, and individual schools are going to need to move to a model of decision making based on data regarding student learning.

Think About It

How might change leaders best prepare teachers for these new Common Core assessments?

12.2 Leading Data-Driven School Reform Data-driven decision making (DDDM) refers to school leaders and teachers systematically collecting and analyzing multiple types of data (e.g., input, process, outcome, satisfaction) to guide a range of decisions to help improve the success of students and schools. Mandinach, Honey, Light, and Brunner (2008) defined input data as school expenditures, or the demo- graphics of the student population; process data as the quality of instruction or financial oper- ations; outcome data as dropout rates, student test scores; and satisfaction data as opinions from teachers, students, parents, or the community. Examination of these sources of data will lead to decisions that usually fall into two categories:

• Using data to inform identify or clarify: identifying goals or needs

• Using data to act: changing curriculum or reallocating resources

Marsh, Pane, and Hamilton (2006, p. 3) suggested a conceptual framework for DDDM, as illustrated in Figure 12.1. In this DDDM framework, once raw data is collected, it must be orga- nized and combined with an understanding of the context through a process of analysis and summary to yield usable information. This information then becomes actionable knowledge when it is weighed against the relative merits of possible solutions. Actionable knowledge can inform different types of decisions, such as setting goals, assessing progress toward attain- ing them, addressing individual or group needs (e.g., low-performing students), evaluating the effectiveness of certain practices, or reallocating resources. Once the decision to act has been made, new data can be collected to begin assessing the effectiveness of those actions. This continuous cycle of data collection (see Figure 12.1), analysis, and decision making is con- ducted to improve the success of students and schools. For a DDDM model to work, the data must be accurate and valid; otherwise, it may become misinformation that leads to invalid inferences and poor decisions.

Leading Data-Driven School Reform Chapter 12

Measuring What Matters

What types of data are administrators and teachers using in DDDM? Outcome data, includ- ing achievement test scores, and state tests are the most popular type of student data to examine, because these data summarize a student’s knowledge on a broad range of skills and topics aligned with standards. Given the high stakes attached to these results, it makes these test scores useful for guiding decision making. However, these tests are usually given in the spring, and the results don’t become available until the end of the school year. By then, it is too late for these data to be used to make decisions for the cohort of students who took the test. Those students have moved to different classes, different teachers, or even different schools. For this reason, schools also look to formative assessments (interim progress tests) in DDDM. These assessments are given more frequently throughout the school year to pro- vide diagnostic information that could be used for decision making. Formative assessment data are also collected through classroom observation, teacher-generated tests, assignments, and homework. Sometimes these outcome data are more revealing than the state test data, because the former can be used in a more timely manner to support and monitor change.

In addition, nonachievement student outcome measures are also used for DDDM. Data on student attendance, student mobility, and graduation rates provide evidence that can be used to evaluate school performance. Inferences can be made from these data on principal effec- tiveness and to inform instructional planning.

Making Sense of Data

With so much data being generated, educators first ask, “What exactly should we be keep- ing track of?” and then “What do we do with the data once we collect it? How do we tell ‘good’ data from ‘bad’ data?” Data is the raw form of information, and information is

f12.01_EDU675.ai

Types of decisions can be made based on the data.

Examples of decisions are:

• Create and determine the progress towards achieving goals

• Address the needs of an individual or the group

• Determine the effectiveness of a practice

• Reallocate resources based on outcomes

• Continue the process of improving

Data types include input, process, outcome, and satisfaction.

Useable information

Figure 12.1: Framework for DDDM

In data-driven decision making, educators systematically collect and analyze many types of data.

Source: Adapted from Marsh, J., Pane, J., & Hamilton, L. (2006). Making sense of data-driven decision making in education: Evidence from recent RAND research, page 3. Reprinted by permission.

Leading Data-Driven School Reform Chapter 12

something that is useful in the decision-making process. Information comes from data, but the important part is how to turn data into information that’s reliable. The first step is to generate usable information from the data through sense making. Sense making is generally defined as a dynamic process by which individuals and groups actively construct knowledge and interpret experiences and ideas (Weick, Sutcliffe, & Obstfeld, 2005). Sense making asks the question, “What do the data show?” This involves finding relationships between various pieces of data.

In looking at evidence (data), several strategies are used to determine these relationships, trends, and patterns: triangulation, mental models, and inference. School leaders also need to know how to look for disconfirming evidence. For example, numbers should never be shown in isolation; they need to be evaluated against some standard or benchmark to be meaningful. Graphic information is almost always easier to understand than numeric data, so aggregating data into charts and graphs is a good idea. Qualitative summaries are also effective in putting the numbers in perspective. In other words, use prose and a few key graphics to transform the data into information. Lipton and Wellman (2004) suggested a protocol for sense making that includes the following components and questions:

1. Engaging with the data

• What predictions will you make about the data?

• With what assumptions are we beginning this process?

• What questions do you want to ask about the data?

• What can we learn as a result of analyzing the data?

2. Analyzing the data

• What important points seem to pop out from the data?

• What patterns or trends are appearing?

• What are some similarities and differences between data from various sources?

• What are we seeing at the school level? The grade level? The class level?

• What seems to be surprising or unexpected?

• What are some things we have not yet explored? What other data do we need?

3. Generating theory

• What inferences, explanations, or conclusions can we make about the data?

• What are the data telling us about the current conditions? Student performance/ attitude? Learning environment? Curriculum/instruction/assessment/staff development?

Formal leaders, given their position in the school power structure, often have the authority to guide or direct sense-making processes by governing where and how sense making happens and by providing material support. The way that school leaders engage in framing tasks— diagnostic, prognostic, and motivating—will shape how teachers generate usable informa- tion from the data and the degree to which teachers believe it is a meaningful strategy for school improvement. How leaders frame DDDM is likely to influence the type of culture that is created around data use (Park, Daly, & Guerra, 2013). Institutional routines must allow time for this sort of data-based inquiry methodology. The focus of team meetings should be on the improvement of instruction through the analysis of student achievement data (Schmoker, 2008). School leaders also need to provide time for teams to meet and review performance

Leading Data-Driven School Reform Chapter 12

data on a consistent basis. In addition, building a professional framework around data analysis needs to lead to a meaningful change in practice for positive action to take place.

Park, Daly, and Guerra (2013) also reported that certain types of decisions are more likely to be informed by data than others. Across studies, they found that test scores (driven by federal requirements) are primarily used to develop school improvement plans (SIPs). Remember that data do not drive change; rather, data are used to support and monitor change. Data are used to confirm the progress a school is making toward its mission or to document a learner’s mastery of a content segment or skill. For example, data can be used to document a school’s progress toward accomplishing its mission. The school is not designed around a test; instead, it is designed around what has been learned from the data (Petrides, 2006).

School leaders and instructional leaders use data to become better managers by looking at several indicators of student performance, such as teacher practice and curricular fit, as a tool to plan the next action steps.

Turning Data into Action

The most prominent advocates of 21st century education stress the importance of examining the results of assessments to make sound decisions about next learning steps. According to Petrides (2006):

There is a tipping point, though, in convincing teachers to examine the impact of their own teaching practices. It arrives when teachers see that something they felt intrinsically is down on paper in a concrete way. To actually know why a lesson worked and how to repeat it is extremely validating. (p. 40)

To accomplish this, a generous allocation of time is needed to move data use to center stage. However, according to Hubbard, Datnow, and Pruyn (2013, in press), time alone is insufficient to turn data into action. Their study showed that,

The principal and teachers had constructed teaching practices and organized the school day in a way that kept English language arts and math data in an instructional silo. . . . Teachers were accustomed to teaching one subject, assessing student work, using the data to inform the teaching of that subject, and moving on to repeat the cycle with a different subject. Integrating content areas in an interdisciplinary way and understanding the useful- ness of using data across content areas was not only unfamiliar but uncomfortable terrain. (p. 31)

What kinds of supports are available to help teachers with data use? According to Marsh, Pane, and Hamilton (2006), the most common form of support for DDDM is workshops or training on how to examine test data. As districts continually add more reform initiatives to a teacher’s plate and as teacher’s pay becomes increasingly tied to student achievement, the importance of providing teachers and school leaders with the requisite knowledge and skills to turn data into action to affect change has never been more important (Hubbard, Datnow, & Pruyn, 2013).

Data collected by questioning students, examining their work, and observing instruction and materials used in classrooms can supplement test score data to inform school leaders and fac- ulty about current practices as they relate to best practices in teaching. For example, “Schools and even whole states could make steady gains on standardized tests without offering

Leading Data-Driven School Reform Chapter 12

students intellectually challenging tasks” (Schmoker, 2008, p. 71). Test prep activities, which were respon- sible for increasing test scores, might not necessarily go hand in hand with best practices to offer more intellectually challenging tasks to learners. In part, this disconnect explains why the Sacramento Bee reported that despite higher passing rates on the math portion of standardized test, more and more college students need remedial math courses in col- lege (Kollars, 2008).

Implications of DDDM for Teaching Practices

From a review of the literature, it is clear that not all educators have the necessary elements to implement successful DDDM: access to data that are timely and valid, the time to analyze the data collected, motivation to interpret the data into usable information, or the skills to develop action plans from the information. The accumulation of data does not, by itself, drive decisions or lead to school improvement. The process of translating data into information, and that information into action, is a labor-intensive, time-consuming practice. School leaders should consider how to provide needed support and infrastructure to facilitate data use (e.g., professional development on analyzing data and identifying and enacting solutions, online learning management systems to collect and organize raw data). Finally, we invite school lead- ers to access the following web link, which provides resources to support the effective use of educational data at the K–12 level: http://edadmin.edb.utexas.edu/datause/index.htm.

DDDM in Action: Response to Intervention

A good example of DDDM within a whole-school context is the model of Response to Intervention (RTI), discussed in Chapter 5. To review, RTI has been defined as the practice of (a) providing high-quality instruction or intervention matched to student needs and (b) using learning rate over time and level of performance to (c) make important educational decisions (Batsche et al., 2005).

Not all schools are implementing the rigorous RtI model. One of the biggest challenges for school leaders is related to time constraints. Where do school leaders find time for increased collaboration across various school professionals for data analysis or for preparing data mate- rials for presentation to the professional learning community? However, those that are imple- menting the model are finding that DDDM is at the heart of this important school reform. Basically, RTI is a set of systematic, increasingly intensive educational interventions, all based on data and designed to target an individual student’s specific learning challenges and to pro- vide a supplementary intervention within the context of the general education class (Bender & Waller, 2012).

The components of an RTI model are universal screening and progress monitoring based on data. Universal screening is the first step in identifying students who are at risk for learn- ing difficulties. Typically conducted three times a year, universal screening measures consist of assessments focused on target skills (e.g., phonological awareness) that are highly predictive of future outcomes (Jenkins, 2003). The instruction that occurs as a function of the outcomes of the assessments truly drives the changes in students identified as being at some level of risk for not meeting academic expectations.

Think About It

• What does DDDM mean for school leaders? For teachers?

• What could you do to further your own practices of DDDM?

Leading Data-Driven School Reform Chapter 12

DDDM in Action: Multitier System of Supports (MTSS)

Closely related to RtI is the Multitier System of Supports (MTSS). MTSS is a multistep pro- cess of providing instruction and support to promote the academic and behavioral success of all children. Individual students’ progress is monitored, and results are used to make decisions about further instruction and intervention. MTSS is most commonly used in addressing read- ing, math, and behavior, but it can also be used in other areas. The MTSS process is flexible and designed by school districts to meet the needs of students.

The MTSS process, like RtI, has three tiers. Each tier provides differing levels of support.

• In Tier I, all students receive high-quality curriculum and instruction in the general edu- cation classroom. The teacher assists all students.

• In Tier II, the school provides supplemental instructional support, usually in small groups, to students who need additional support to what they are receiving from the general curriculum.

• In Tier III, intense instructional support is provided to students with the greatest needs, with frequent progress monitoring. (http://www.usd250.org/)

Think About It

In both the RTI and MTSS models, progress is closely monitored, and changes in instruction are based on data collected from ongoing assessments. How might these models help close the achievement gap?

Progress Monitoring

If leaders are going to base decisions on use of student learning data, then from where does that data originate? Throughout the school year, formative assessment practices (discussed in Chapter 11) provide the data necessary to benchmark students’ progress. Student progress is monitored so that special attention can be paid to any student falling behind and adjustments can be made well before formal statewide testing occurs. Often student progress data is kept on benchmark assessments. Benchmark is a term that has been connected to high-stakes testing because performance on benchmark checks has been linked to the probability of suc- cess on high-stakes tests. However, a benchmark can also be a predetermined score (e.g., a test score that corresponds to the 25th percentile, below which a student qualifies for inter- vention). Progress monitoring uses a series of equivalent assessments or probes to document progress from one period to the next. Progress monitoring keeps track of students’ academic development through frequent data collection (e.g., weekly or monthly), with interpretation of the data at regular intervals and changes to instruction based on the interpretation of each student’s progress. This method of progress monitoring is, itself, a formative process to assess student academic performance and to evaluate the effectiveness of instruction. Progress monitoring involves repeated samples of student performance data over time (e.g., weekly quiz grades, lab assignments, writing prompt responses). When progress-monitoring data are charted over time in a graph, the visual representation can be used to provide feed- back for students on their performance and guidance for teachers in their instruction, just as

How Leaders Ignite Data-Based School Reform Chapter 12

the online Weight Watchers program uses the slope of the visual line of improvement to keep participants motivated.

The challenge of this process lies in maintaining equivalency of the probes. As content in a discipline changes, performance indicators change. Unequal probes will not work for accurate comparisons. However, certain aspects will work, such as key vocabulary, which can be stable in level of difficulty across study units. Key vocabulary may be “big picture” ideas critical to understanding details. Key words have been an effective indicator of both student perfor- mance and progress for more than two decades (Espin, Shin, & Busch, 2005).

12.3 How Leaders Ignite Data-Based School Reform Finally, what actions can be taken by school leaders when the progress monitoring dem- onstrates that students are either not engaged in the learning process or not achieving expected results? Two effective actions are the requirement that all teacher teams and grade-level professional learning communities (PLCs) use common formative assessments to analyze student learning data and the engagement in collaborative action research by teacher teams.

The Power of Common Formative Assessments

We discussed formative assessments in the previous chapter. Common formative assessments require teachers to give the same assessments. These assessments are given by teacher teams who teach the same content or grade level—that is, those teachers with “collective respon- sibility for the learning of a group of students who are expected to acquire the same knowl- edge and skills” (DuFour, DuFour, Eaker, & Many, 2010b, p. 2). Teachers who use common formative assessments have more in-depth discussions about proficiency, and, as they exam- ine the resulting data, they have a more focused discourse about what is needed to improve student learning (Graham & Ferriter, 2008). According to DuFour, DuFour, Eaker, and Many (2010a, p. 80), common formative assessments (a) promote an effective strategy for deter- mining whether the guaranteed curriculum is being taught and learned; (b) inform teacher practice by facilitating a systematic, collective response to students who experience difficulty; (c) offer a powerful tool for changing adult behavior and practice; and (d) promote efficiency for teachers.

As stated in earlier chapters, the process of change is not an easy one for educators. Therefore, change leaders should be transparent about the introduction of common formative assess- ments. Leaders should consider the following points:

• Start with a volunteer group of teachers. It is recommended that school leaders start with a volunteer group of teachers who already have a record of effective work in a PLC. Teachers can even embark upon collaborative action research (see the next sec- tion) to study the efficacy of using common formative assessments and report findings to their peers.

• In the beginning, keep the focus on process not on results. Getting teachers comfort- able with the process of designing common formative assessments is highly critical before they actually start analyzing student data. An idea that one school leader used successfully was to encourage teachers to use any common assessment they deemed appropriate. This decision on the part of the leader allowed the teachers to experience

How Leaders Ignite Data-Based School Reform Chapter 12

maximum ownership on the process. Then, gradually, the leader introduced more and more specific assessment measures that teachers might use to obtain data.

• Ensure teachers that they are not being evaluated. Edie Halcomb, in her book Getting Excited About Data (2004), shared six reasons for resistance to data. One of those, she asserted, is fear of evaluation. Ainsworth and Viegut (2006) added, “Teachers need to be able to experiment, reflect, dialogue, and become comfortable with the process of developing and scoring common formative assessments without fear of reprisal” (p. 65).

• Collaboratively design future professional development opportunities. In an open school environment, leaders welcome opportunities to collaborate with teachers

regarding their professional development needs. As teachers gain more confidence in using common formative assessments, they will have good ideas regarding their own learning needs. Once data are analyzed and teachers witness that many students are not learning key curricular standards, they will want to know more about differentiated instruction and research-based teaching strategies.

Action Research: A Data-driven Model of Reform

If you want truly to understand something, try to change it. —Kurt Lewin (1946)

School leaders may use action research to have teachers, and especially, teacher teams, become more effective in making data-driven decisions regarding student learning. By defini- tion, action research is:

Any systematic inquiry conducted by teacher researchers, principals, school counselors, or other stakeholders in the teaching/learning environment to gather information about how their particular school operates, how they teach, and how well their students learn. This information is gathered with the goals of gaining insights, developing reflective practice, effecting positive changes in the school environment and improving student outcomes and the lives of those involved. (Mills, 2011, p. 5)

The key word in this definition is systematic, because action research follows a problem- solving format in which the answer to one problem leads to asking the next question. Since data are always the focus of action research, it makes an excellent change model for school leaders toward DDDM.

Action research is relevant, persuasive, and authoritative (because it is data based); improves teachers’ practices and student outcomes; and demonstrates a level of commitment from the teacher researcher to critically examine the effect of his or her actions on student learning. For these reasons, action research is a powerful tool to use to guide decisions about curriculum and instruction. In addition, research suggests that teachers are more likely to be influenced by the professional practices and action research of their peers than they are to be influenced by journal articles or undergraduate or graduate courses (Reeves, 2008).

How School Leaders May Guide the Action Research Process School leaders need to be familiar with the action research process so that they may serve as a guide to teacher teams and PLCs. To begin, the action research cycle follows the same path you would take to solve any problem. There are four main “stages”:

Think About It

Looking at the many ways change leaders may assist teachers in using common formative assessments, which ones might be particularly effective in your school setting?

How Leaders Ignite Data-Based School Reform Chapter 12

1. Planning

2. Acting

3. Developing

4. Reflecting

Each stage has several steps, as outlined in Table 12.1.

Table 12.1: The steps of action research

Problem-Solving Approach Steps Taken in Action Research Planning Stage

1. Identify the problem.

It starts with identifying a problem you want to solve to improve your practice (and student learning).

The problem may stem from new federal regulations in education or simply from lack of student motivation.

Planning Stage

2. Collect information on the topic.

Review literature (guiding theories and previous research on the topic being investigated). The literature review builds a coherent argument leading up to the significance of the study and research question.

Make observations and jump-start your curiosity.

Planning Stage

3. Perform collaborative dialogues.

Talking about the problem with others brings the power of diverse voices to bear on the issue under study.

Planning Stage

4. Identify research question or hypothesis.

Is the question researchable?

Will the question produce data needed to answer it?

Can the question be answered in a reasonable time?

Is the question relevant to the targeted research context and guided by existing theories and research?

Planning Stage

5. Perform a needs assessment.

The needs assessment defines the gap between current practices or conditions and desired or wanted practices or conditions. The discrep- ancy between the current condition and desired condition identifies the need.

Planning Stage

6. Determine research method.

Action research is a set of procedures (as set forth by the steps in the left column) that we typically use qualitative and quantitative methods to measure the results of our actions.

Action research is inductive, which means that it begins with specific observations and concludes with broader generalizations.

Acting Stage

7. Develop action plan and collect data.

Conduct a needs assessment to provide baseline data on the issue being investigated. Develop a research question based on this information.

Identify the significance or potential learning outcomes from the study. Why is this particular issue worth investigating?

Develop an action plan: specific tasks (what will be done and by whom), timeline, and resource allocation.

Developing Stage

8. Analyze data.

Identify themes, trends, similarities, and difference in the data.

Construct a model of understanding (usable information) from the data.

Developing and Reflecting Stage

9. Conduct a discussion (reflection).

Identify continued professional development needs.

Think about where you have been, what you have learned, and where you are going.

Planning Stage

10. Select new research question or interest.

Implement recursive design (see Figure 12.1).

Case Study in Educational Leadership Chapter 12

Within this framework, action research clearly does not proceed in a linear fashion; rather, is a cyclical pro- cess with a recursive nature. Often, teacher research- ers will repeat some of the steps several times or go in a different order, depending on the meaning of the data they are collecting. Data are analyzed “as you go,” so that the results from the data collection and analysis drive the direction of future research.

Rigor in Action Research According to Mertler (2009, p. 25), there are numerous ways to provide rigor within the scope of teacher-led action research studies:

• Repetition of the cycle: One action research cycle is not enough; with each subsequent cycle, greater credibility is added to the findings (and more is learned).

• Prolonged engagement and persistent observation: Because teachers are with students every school day, they have the context to make extended observations and explore an issue in detail.

• Triangulation of data: Rigor is enhanced when multiple sources of data are examined to cross-check the accuracy of the data.

• Member checking: This means that participants have the opportunity to review the raw data, analyses, and final reports to verify that their beliefs, perspectives, and expe- riences were accurately represented.

Conducting rigorous research can be challenging. Measuring implementation fidelity, measur- ing performance, and documenting intervention, context, and changes all involve maintaining a balancing act between the teacher as a classroom teacher and the teacher as a researcher and change facilitator. There are many good books on action research to help guide teachers and leaders through the process:

1. Nancy Fichtman Dana and Diane Yendol-Hoppey, The reflective educator’s guide to profes- sional development: Coaching inquiry-oriented learning communities (Thousand Oaks, CA: Corwin Press, 2008).

2. Sandra M. Alber, A toolkit for action research (Lanham, MD: Rowman & Littlefield, 2010).

3. Gary L. Anderson, Kathryn Herr, and Ann Sigrid Nihlen, Studying your own school: An educator’s guide to practitioner action research (Thousand Oaks, CA: Corwin Press, 2007).

4. Karl H. Clauset, Dale W. Lick, and Carlene U. Murphy, Schoolwide action research for professional learning communities: Improving student learning through the whole-faculty study groups approach (Thousand Oaks, CA: Corwin Press, 2008).

12.4 Case Study in Educational Leadership The leadership at Dunn Elementary School believed in using teacher-based action research as an agent for improvements in the school’s performance. The principal had implemented the first tier of action research last year by asking all teachers to collect data per student on scores across tests, projects, and activities. At the end of the school year, one of the data sets

Think About It

Suppose that students are not progressing in essay writing as you had hoped. What might you do to systematically examine this issue through action research?

Case Study in Educational Leadership Chapter 12

gathered had demographics including the first name, age, grade, ethnicity, gender, and scores of students on all tests related to math, language, and spelling.

The principal also asked that teachers form teams across grade levels to look at the data and make recommendations for creating class rosters for the next year. The teachers of first, sec- ond, and third grade met to review the data and developed class rosters based on skill sets. The teams decided to create some blended classes across grade levels to try to address the deficits in skills.

Kearstin and Ann were coteachers in a blended first/second-grade classroom. Over the sum- mer, they met to review all the data they could on their class. The students had been assigned to the classroom based on math scores from the previous years. In the class were 11 boys and 14 girls. The class was diverse, with students identifying themselves as African American (4), white/European (13), Asian (2), and Latino (6).

The pre-test data indicated that all of these students scored between 50% and 70% mastery on most math skills. Particularly of concern were the scores in addition and subtraction up to 100. The teachers wanted to get these students to mastery (80%+) within 6 weeks.

Four of the students had individual education plans (IEPs) that included goals related to mas- tering addition and subtraction up to 100. These four students were strong in language skills. Additional research indicated that all of the students spoke English as their primary language, with 12 students also fluent in another language. The language skills of these 25 students were strong—all above 80% on last year’s tests.

From the research, the teachers realized the students as a group were at or above mastery in language, which indicated a potential strength area for teaching. In addition, the class’s math skills were below average, indicating a good place to conduct action research. They researched methods for teaching math using language as a basis for the lessons.

They found it difficult to identify a specific “programmed” math series that had language as its basis, so they created their own version. Every day once school began, they embedded journaling, word problems, and visual cues with words; used written instructions for addition and subtraction; and used talk-aloud techniques to help students hear and say the procedures for addition and subtraction. The plan included lessons that started with a word problem for the day and then direct verbal instruction on the lesson of the day. Next students became the teacher by repeating the day’s lesson in small groups, with practice that requires the students to quietly talk out loud as they work their problems.

During the first week of class, the teachers conducted an initial test to see if any students had dramatically changed over the summer, but there were no remarkable changes. Kearstin and Ann planned to spend a 6-week period using the language strategy and collect a minimum of five assessments to identify any growth of addition and subtraction skills. At the end of the 6 weeks, they sat down to review the data.

Table 12.2 outlines the test average scores after 6 weeks of instruction on addition and sub- traction and the combined score average per student in the room. An asterisk (*) was used to indicate students with IEP goals on math mastery. On first glance, they were pleased with the results.

Case Study in Educational Leadership Chapter 12

Table 12.2: Average test scores after 6-week program (percentage) POST

Student

Addition up to 100

Subtraction less than

100

Total Mastery

Score

50–59

60–69

70–79

80–89

90–100 Aaron 83 86 84.5 ×

Andrew 92 99 95.5 ×

Annabelle 78 79 78.5 ×

Bethany 81 99 90 ×

Bonnie 89 91 90 ×

Brinn* 95 95 95 ×*

Candace 100 80 90 ×

Catherine 78 83 80 ×

Christopher 83 87 75 ×

Dan 87 86 86.5 ×

Denise* 56 55 55.5 ×*

Elizabeth 83 85 84 ×

Galen 92 96 94 ×

Jesus 77 80 78.5 ×

Juanita 86 81 73.5 ×

Kendra 75 73 74 ×

Lonnie 85 84 84.5 ×

Marianne 98 97 97.5 ×

Martin 77 75 76 ×

Noe* 69 65 67 ×*

Nadira 80 81 80.5 ×

Randall 79 80 79.5 ×

Renee* 90 90 90 ×*

Steven 92 93 92.5 ×

Yuni 99 84 91.5 ×

However, they knew there were still important questions to ask. Using this table, they broke the data down further to identify any other trends. They had the following questions on test scores related to the 6-month language-based teaching:

• Did we meet the goal of getting all students up to mastery (80%) within 6 weeks?

• How many students are still scoring below the 80% mark after 6 weeks?

• Did any students remain at the same level after 6 weeks?

• Was there a difference in scores between the boys and girls in the class?

• How did the students compare based on ethnicity?

• Are there any negative trends from using this method?

• Are there any positive trends based on the data?

Case Study in Educational Leadership Chapter 12

• Our plan is to move toward multiplication and division. Can we move forward based on these scores, or should we stay with addition and subtraction with the entire class?

• Should using the language-based program continue?

Based on these questions, they created several other data sets to review and share with col- leagues (Tables 12.3–12.5). They held a meeting with the other first- through third-grade teachers to review the data and determine what steps to take next.

Table 12.3: Average pre- and post-test results

Average % on Tests 50–59 60–69 70–79 80–89 90–100 Pre-test Scores 8 7 10 0 0

Post-test Scores 1 1 7 6 10

Table 12.4: Pre- and post-test results by gender

Pre-test by Gender (percent)

50–59

60–69

70–79

80–89

90–100

Female 3 6 5 0 0

Male 5 1 5 0 0

Post-test by Gender (percent) Female 1 0 3 3 7

Male 0 1 4 3 3

Table 12.5: Pre- and post-test by ethnicity

Test Score Percentage Range

Ethnicity

50–59 60–69 70–79 80–89 90–99 PRE POST PRE POST PRE POST PRE POST PRE POST

African American 4 0 0 0 0 2 2 0

White/Euro 3 1 7 0 3 2 3 7

Asian 0 0 0 2 0 0 2

Latino 1 0 1 5 3 1 1

After reviewing the comments from their colleagues, Kearstin and Ann worked on the next version of their math teaching strategies. They decided that the strategy using language as a primary method for experiencing math problems had overall been successful. Pre-tests showed 0% of the students were at 80% or higher, while post-test scores showed that more than 50% of the students scored at 80% or higher. All students advanced at least 1 percent- age level. Both males and females improved scores. Improvements were noted across all eth- nicities. They recognized that some students were still scoring below the 80% desired score range and began mining the data on those particular students for insight for new strategies to help them progress.

Post-Test Chapter 12

Critical Thinking Questions 1. Review the data and provide answers to all the questions posed by the teachers. What trends do

the data reveal? Based on the answers to these questions, would you agree that the language- based strategy to teach addition and subtraction works?

2. What other questions would you want to ask about the data if you were helping the teachers review? Explain why having the answers to your questions would be helpful in improving the teachers’ strategy.

3. Given the data results, what would be your biggest concern about the strategy? Explain your reasoning.

Summary This final chapter was intended to assist the change leader in understanding critical issues in assessment. We discussed the current assessments for the Common Core State Standards— specifically, the PARCC and smarter-balanced assessment systems. This round of high-stakes testing, along with other current state and local measures, has strengthened the need for change leaders to institute data-driven decision making from the school to the classroom level. Finally, we encouraged school leaders to adopt schoolwide practices of using common formative assessments and action research to both assist in making data-driven decisions and rekindle excitement in schools for both leaders and teachers.

Post-Test 1. Which of the following is not true of newest assessments in English language arts and math?

a. They are designed to measure readiness for college and careers.

b. They attempt to negate concerns related to current state tests.

c. They are based on advances in technology and cognition.

d. They measure students in kindergarten through fifth grades.

2. Mrs. Menchie is using formative assessment with her classes. Which practice would best exemplify this process?

a. She administers an exam at the end of the content unit to assign grades.

b. She gives an individual oral assessment and then adjusts her instruction.

c. She has students do homework and returns the feedback a few weeks later.

d. She plans for her next unit by reviewing the state standards.

3. Sean has researched different ways to implement technology to enhance student learning and plans to mention it during a PLC meeting to discuss the research findings. Which step of action research does this situation exemplify?

a. Planning

b. Acting

c. Developing

d. Reflecting

Key Ideas Chapter 12

4. The Smarter Balanced Assessment Consortium System differs from other assessment systems in that it

a. is taken by students on computers at their schools.

b. allows students to retake parts, if locally approved.

c. has summative components.

d. focuses on science and social studies.

5. Ren has scored below minimum levels of proficiency and receives additional instruction in math. In the Response to Intervention model, she would

a. receive the most intensive intervention.

b. be classified as Tier III.

c. be classified as Tier II.

d. need support beyond what the model prescribes.

6. Brett is finding that his curiosity is in overdrive about developing a parent program for his school after seeing it at other schools. He would be in which stage of action research?

a. Planning: collect information

b. Acting: developing action plan

c. Planning: identify research hypothesis

d. Developing and reflecting: conduct discussion

Answers 1. d. They measure students in kindergarten through fifth grades. The answer can be found

in Section 12.1.

2. b. She gives an individual oral assessment and then adjusts her instruction. The answer can be found in Section 12.2.

3. a. Planning. The answer can be found in Section 12.3.

4, b. allows students to retake parts, if locally approved. The answer can be found in Section 12.1.

5. c. be classified as Tier II. The answer can be found in Section 12.2.

6. a. Planning: collect information. The answer can be found in Section 12.3.

Key Ideas • School leaders need to support teachers’ learning regarding the Common Core

assessments.

• Change leaders need to use principles of data-driven decision making when faced with school and classroom reforms.

• Data-driven decisions have the power to influence stakeholders.

• Common formative assessments and action research are powerful reform practices for teachers to engage in data-driven decision making.

Key Terms Chapter 12

Critical Thinking Questions 1. What are the two major assessment systems developed for the Common Core State

Standards? How do they differ?

2. Explain three ways that school leaders may assist teachers in implementing the new Common Core assessments.

3. Give two reasons change leaders need to make data-driven decisions.

4. What are the advantages of teams of teachers using common formative assessments?

5. Explain how school and teacher-leaders may use action research to propel school reforms.

Key Terms action research Measures change in a cyclical manner with the intention of overall positive growth throughout the process.

benchmark A predetermined score, below which a student qualifies for intervention.

common formative assessments Assessments given by teacher teams who teach the same content or grade level.

data-driven decision making (DDDM) The process by which administrators and teachers collect and analyze data to guide educational decisions.

multitier system of supports (MTSS) A multistep process of providing instruction and support to promote the academic and behavioral success of all children.

Partnership for Assessment of Readiness for College and Careers (PARCC) One of two consortia of states currently developing assessments for the Common Core State Standards.

progress monitoring A practice that regularly collects students’ learning outcomes over a period to assess student performance and to evaluate the effectiveness of instruction.

recursive design The spiraling cyclical process of action research such that researchers work simultaneously within several research steps and circle back to readdress issues and modify research questions based on reflection.

Response to Intervention (RTI) A three-tiered, data-driven approach that provides both prevention and intervention for students based on benchmark proficiencies.

Smarter Balanced Assessment A system of valid, reliable, fair assessments aligned to the Common Core State Standards

universal screening A component of RTI where tools, such as curriculum-based measures, checklists, or direct assessment, are used to identify levels of proficiency for each student in essential academic areas. Those students who are not meeting grade-level standards are identified and placed in the appropriate tier.

Additional Resources Chapter 12

Additional Resources Further Readings

Halcomb, E. (2004). Getting excited about data: Combining people, passion, and proof to maximize student achievement. Thousand Oaks, CA: Corwin Press.

Leithwood, K., Aitken, R., & Jantz, D. (2006). Making schools smarter: Leading with evi- dence (3rd ed.). Thousand Oaks, CA: Corwin Press.

McNiff, J., Lomax, P., & Whitehead, J. (2006). All you need to know about action research. Thousand Oaks, CA: Corwin Press.

Mills, G. (2007). Action research: A guide for the teacher researcher (3rd ed.). Upper Saddle River, NJ: Merrill.

Santoyo, P. (2010). Driven by assessment: A practical guide to improve instruction. New York, NY: John Wiley & Sons.

Videos

• How the Common Core is changing assessment: http://www.youtube.com/ watch?v=N6kdyqeoiSI

• Common Core assessments—Unpacking standards: http://www.youtube.com/ watch?v=A33RQ7uAeNA

• Action research—First steps: http://www.youtube.com/watch?v=MtV2t2lkgJw

Weblinks

• Achieve the Goal: http://www.achievethecore.org—a website provided by Student Achievement Partners, a nonprofit organization founded by the writers of the CCSS to provide updated and high-quality CCSS resources

• Center for Collaborative Action Research: http://cadres.pepperdine.edu/ccar/define.html

• Action Research—CCAR Interact: http://ccar.wikispaces.com

• Action Learning, Action Research Association: http://www.alara.net.au/public/home